Gamechanger in the M&E Industry: Real-Time Virtual Production
Gamechanger in the M&E Industry: Real-Time Virtual Production
By : Priyan Parab
After being intricately involved in the world of Real-Time Virtual Production for the past few months, I thought it would be a wise idea to give you all some insights about this blooming field. Being introduced to the world of a stage-like digital environment that optimizes the entire workflow of production and post-production pipelines can be of great benefit to individuals in the Media & Entertainment (M&E) sector.
This article focuses on explaining Virtual Production in a simplified manner. Irrespective of whether you are from the industry or not, you should not face any difficulties in understanding this invention. In case you do, please do not hesitate to get in touch with me!
The rise of these innovative technologies under the umbrella of Virtual Production will surely disrupt the market, while simultaneously keeping your current workflow intact with an added advantage to increase efficiency and productivity. The range of new possibilities it brings within the production procedures of any film, ad-film, television, live events, theatre, etc. are sure to complement the industry with widespread flexibility.
According to E&Y, the M&E sector in India had an upward growth chart by 9% in 2019 and reached a whopping figure of INR 1.82 trillion. Statistical research states these numbers would further rise up to INR 2.41 trillion by 2022. One cannot help but note here that the revenue-wise top 4 sectors are – Television, Digital Media (OTT), Filmed Entertainment and VFX respectively. Pre-production, production and post-production are critical elements in each of these 4 sectors.
What is Real-Time Virtual Production?
Imagine if the entire production of any of the 4 sectors mentioned above could be done indoors? Completely eliminating the need for outdoor shooting. All this while the final output of the shot is visible in real-time on your screen. Yes, this is absolutely possible! By stepping into the domain of Real-Time Virtual Production, you can not only have access to the final product of the scene in real-time but also save upon millions of dollars that would otherwise be invested in production and post-production costs.
Zero Density (ZD), a real-time virtual production equipment company, provides a solution known as Reality Virtual Studio platform. This is offered in India by Cineom, a leading system integrator company in the Indian M&E markets. The only two mandatory requirements are a studio with green/blue/black screen for keying and appropriate tracking devices. Once you have these along with a Reality Engine, trust me, there is no limit to executing your imagination.
This product amazingly not only keys the object of focus at an accuracy of 99.9% but also provides real-time 3D compositing. This means the manpower requirements are lowered as the high-tech system does the majority of the job. There’s a lot more to this product – all the reflections, refractions and formulated shadows of the subjects and objects are recorded in real-time. What this aims to reward the user is with accurate photo-realistic effects.
Wondering how trustworthy and effective can it be? The news and sports broadcasting industry have successfully reaped maximum benefits from this product. Star Sports India is currently using more than 10 Reality Engines to run their post-match analysis of IPL (Indian Premier League) and other events. Even the FIFA 2018 World Cup held in Russia used a similar concept of high-end Virtual Production for a top-notch viewing experience for their viewers. In my opinion, in the foreseeable future, this technology shall be adopted by the film & ad-film markets to revolutionize the way production works.
This must be a question lingering in your mind right now. My understanding from various projects I have seen & experienced has taught me how this product can simplify the tasks of directors and the DOP (Director of Photography). In films, this technology comes in as a time/money/effort saviour in Previz (Pre-Visualization). It gives the directors and DOP a vision of their final product when they try to shoot something on a green screen. All the background sets, including lighting, shadows, attachments, etc. can be clearly accessible and visible to the film crew in real-time while they are shooting.
Can it be shot in 4K? Oh, absolutely! One can record in 4K UHD HDR, as the Reality Engine from ZD is compatible with all such crucial formats involved in the production. An immense focus is laid on the different kinds of data that can be recorded and stored while shooting real-time –
- Background data
- Foreground data
- Tracking data
- Alpha Channels
- Real-time final Compositing
- Keyed Output (with variations)
The whole purpose of providing the user with such vital data is so that they have hands-on access to several elements of their shoot. This data also gives them the flexibility to accomplish any additional VFX-related goals.
The functionality of Reality Engine in the case of ad-films allows systems to export the real-time output and use it directly for the project at hand. To put things into perspective, let’s speak about an ad-film for a brand where the actor has to perform a stunt or walk on a world-renowned high rise building. In normal cases, this would be a herculean task due to permissions, stuntman training, costs and safety involved. Making an individual walk on the Eiffel Tower or World Trade Centre for real-life production requires months of planning. Rather than going through the above-mentioned hassle, you must opt to create a virtual set. These virtual sets can be created on Maya, 3Ds-max or an unreal engine (it has to be an FBX file), and then directly imported onto the Reality Engine by ZD. Simply turn on the virtual production system & your cameras, and you are good to begin shooting.
My recent experience doing Virtual Production
There are no limits to the benefits of Virtual Production. Not too long ago, I attended an event (which I was a part of) at Famous Studios. Cineom and Centroid India focused on some of the most important advancements in the domain of Virtual Production that was displayed there. It was for the 1st time in the industry that Cineom, ZD and Centroid teams could integrate the output of motion capture. We could literally include a CG (Computer-Generated) character into the Reality Engine via a live link. When viewed as an output, you could see the CG augmented character fighting with the real character, both being keyed at the same time! That too while both characters are present in the ongoing virtual set! Think about it, if we could achieve this in real-time, then what not is possible?
Centroid also provides some wondrous additional solutions that can be implemented during production and post-production:
- Real-time performance capture
- Virtual camera
- Real-time augmenting of CG elements in the real world
If you find yourself wondering which tracking devices, cameras or lenses to use, then I would love to tell you that any professional broadcast or film camera & lens can work just fine. Just make sure the lens used is capable of recording and storing lens data. I would suggest a few brands such as Arri, Angenieux, BMD, Fujinon, Panasonic (4K PTZ camera as they have in-built tracking), Sony, etc. Speaking of tracking devices – I would suggest a highly economical and user-friendly model known as Stype Redspy.
To quickly summarize, here are the elements you would need to do Virtual Production:
- Virtual Set (FBX file)
- Reality Engine (by Zero Density)
- Tracking System (Stype Redspy)
- Cameras, lenses and lights
- Cyclorama & other shooting accessories.
Here are some youtube links for your further understanding. 1- Virtual Production in Movies, 2- Virtual Production for TV & Broadcast, 3- Virtual Production for Ad-films & 4. Virtual Production for Live & Launch events.
Do you have any more technical questions or general queries? Please feel free to get in touch with me or approach Cineom team as they have a complete turnkey Virtual Production Solution.