Virtual production is a filmmaking method in which actors perform in front of large LED walls that display computer-generated environments rendered in real time by a game engine, with camera tracking so the background moves correctly with the lens. The environment, its light and the actors are captured together, in camera, on the day.
It is often discussed as a rival to AI-assisted production, but the two solve different problems at different stages, with very different cost structures. This guide explains how virtual production works, which films and series used it, its disadvantages and costs, and how it compares and combines with AI-assisted production.
Key takeaways
- Virtual production captures live performances against real-time rendered environments on LED walls, producing visual effects shots in camera rather than in post.
- Its defining advantage is interactive light: the actors and practical set are lit by the environment they appear to be in.
- Use grew quickly after The Mandalorian, and Wikipedia reports LED stages rising from three in 2019 to about 300 by October 2022.
- The main drawbacks are high fixed costs, front-loaded asset work, moiré and color-rendition issues on camera.
- Virtual production is a way of shooting; AI-assisted production is a way of working across the whole pipeline.
- The two combine well: AI-assisted previs and concepting before the shoot, the LED volume on the day.
Table of contents
- What is virtual production?
- How an LED volume and real-time engine work
- What movies used virtual production?
- What are the disadvantages of virtual production?
- Is virtual production expensive?
- What is AI-assisted production?
- Virtual production vs AI production: side by side
- Where the two work best together
- Choosing the right mix for your next project
- Frequently asked questions
What is virtual production?
Virtual production replaces a green screen with a live, rendered world that the camera can see. Wikipedia describes on-set virtual production as a technique “in which LED panels are used as a backdrop for a set, on which video or computer-generated imagery can be displayed in real-time,” and notes that it became widespread after the first season of The Mandalorian in 2019, which used Epic Games’ Unreal Engine.
Frame.io’s primer puts the recipe in one line: real-time game engine animation, LED screens and tracked cameras, combined to deliver visual effects shots live and in camera. The term is sometimes used more broadly for any workflow that blends real-time computer graphics with live action, but when producers ask “what is virtual production?” today, they almost always mean the LED volume.
The practical difference from green screen is what everyone on set can see. The director and cinematographer frame against the actual background, the actors react to a world instead of a colored wall, and the light from the screens falls on faces, costumes and props the way the environment would light them.
How an LED volume and real-time engine work
An LED volume is a stage wrapped in high-resolution LED panels, driven by a real-time engine that re-renders the background from the camera’s exact position many times per second. Four parts have to work together:
- The LED wall and ceiling. For The Mandalorian, Industrial Light & Magic describes a “20′ high by 270-degree semicircular LED video wall and ceiling with a 75′-diameter performance space,” where practical set pieces were combined with digital extensions on the screens.
- The real-time engine. Software such as Unreal Engine renders the environment so that perspective and parallax shift as the camera moves.
- Camera tracking. The camera’s position and lens data feed the engine, so the part of the wall the lens sees is rendered from the correct viewpoint.
- The environment team. Artists who build the virtual sets, and operators who adjust lighting, time of day or set dressing on the stage.
ILM says the system delivered accurate interactive light on the actors and practical sets and allowed real-time in-camera composites, and that over 50 percent of The Mandalorian Season One was filmed this way.
Key insight: Virtual production’s real advantage is not the background. It is the light. An actor lit by the environment they appear to be standing in is extremely hard to fake convincingly in post, and that is why good volume work reads as real.
What movies used virtual production?
LED-based virtual production has been used on major series and features since 2019. Wikipedia’s list of on-set virtual production credits includes:
- Series: The Mandalorian, House of the Dragon, Star Trek: Discovery and Ahsoka.
- Features: The Batman, Top Gun: Maverick, Avatar: The Way of Water, Barbie and Dune: Part Two.
The ideas behind it are older. Wikipedia notes that Avatar (2009), The Jungle Book (2016) and The Lion King (2019) used related in-camera and real-time techniques before LED volumes spread, and that the number of LED stages grew from three in 2019 to about 300 by October 2022, most of them built during or after COVID-19 lockdowns.
What are the disadvantages of virtual production?
Virtual production’s disadvantages are cost, preparation time and a set of on-camera technical problems that need specialist crew. None of them is a reason to avoid it, but each has to be planned for.
- Front-loaded work. Frame.io’s guide warns that assets such as models, characters and 3D environments “must be completely camera-ready before production starts.” Decisions that used to be deferred to post must be made in pre-production.
- Moiré. When the camera sensor’s pattern conflicts with the spacing between LEDs, interference patterns can appear. Pixel pitch, focus and camera distance all matter; the same Frame.io guide notes that The Mandalorian used 2.8mm panels and that denser panels cost more.
- Color rendition. Research by Chloe LeGendre, Lukas Lepicovsky and Paul Debevec found that LED panels “produce problematic color shifts when used as lighting” because of their narrow-band red, green and blue output, affecting skin tones and costume colors unless the stage is carefully calibrated.
- Not every shot is final. Frame.io notes that about half of the shots in The Mandalorian Season One were final in camera, so post-production VFX still has plenty to do.
- Specialist crew. Volumes need real-time operators, engineers and environment artists, and the cinematographer now lights in coordination with a rendered world.
Is virtual production expensive?
Yes, at small scale. A full LED volume carries high fixed costs: stage time, panels, processing, real-time environment builds and specialist crew. Those costs are largely independent of how much footage you shoot, so the economics improve with volume. A single day on a volume is expensive; many days that reuse the same environments spread the build cost.
That is why premium episodic series with recurring sets are the strongest fit, and why Frame.io points out that smaller setups, such as a single LED wall, can be a budget-friendly alternative with trade-offs in interactive lighting. For a mid-budget drama in contemporary settings, real locations are often cheaper and better.
Key insight: Virtual production moves money from post into pre-production. If your team cannot lock environments before the shoot, the volume will cost more and deliver less.
What is AI-assisted production?
AI-assisted production uses generative and analytical AI tools at many points in the pipeline rather than at one stage: script development, AI storyboarding, previs, concept and environment design, VFX on shot footage, editorial assistance, versioning and delivery. Artists direct the tools and approve the results.
Its strengths are the opposite of virtual production’s. It has low fixed costs and variable costs that scale with output, it makes iteration cheap, and it suits productions that cannot justify volume time. Its weaknesses are shot-to-shot consistency in final imagery, supervision time and unresolved rights questions around training data, all of which need to be planned and budgeted.
The two approaches also change crews differently. A volume adds on-set technical roles that barely existed a decade ago, and it makes departments that used to work in sequence work at the same time. AI-assisted production needs supervision capacity more than new job titles: experienced people reviewing and directing generated output. Productions that hand that review to junior staff because the tools look easy tend to get weak results.
Virtual production vs AI production: side by side
The clearest distinction is that virtual production is a way of shooting, while AI-assisted production is a way of working. The comparison below shows where each fits.
| Factor | Virtual production | AI-assisted production |
|---|---|---|
| What it is | Live performance captured against real-time rendered environments on LED walls | AI tools used across development, pre-production, production and post |
| Where it sits | On the stage, during principal photography | Throughout the pipeline, strongest in pre-production and post |
| Cost structure | High fixed costs; improves with many days on reused environments | Low fixed costs; variable costs in iteration and supervision |
| Defining strength | Real interactive light on actors, composites seen on set | Fast, cheap iteration and exploration of options |
| Main risks | Moiré, color shifts, front-loaded asset work, stage cost | Consistency across shots, supervision time, rights and provenance |
| Best suited to | Premium series and features with recurring or impossible environments | Previs, concepting, VFX on shot footage, versioning, budget-constrained ambition |
Neither fixes a weak script, removes the need for craft judgment, or is automatically cheaper. Both relocate where experience is applied rather than reducing the need for it.
Where the two work best together
The productions getting the most from both use them at different stages rather than choosing one. Three combinations come up most often.
- AI in pre-production, the volume on the day. Environment concepts and previs are generated and iterated cheaply, then the chosen direction is built properly for the LED wall. This directly addresses virtual production’s biggest pain point: environments must be decided early.
- Volume for principal photography, AI for everything after. Shoot the hero material on the stage, then use AI-assisted tools for VFX cleanup, versions and the large volume of promotional and platform-specific cuts a release needs, including vertical versions through tools such as Reframe.
- AI for exploration, real-time for execution. Test twenty environment directions in a day, then build the one that works in the engine.
By project type, the decision usually looks like this: premium episodic with recurring environments is a strong virtual production case; a feature with a few impossible locations can use a volume selectively; mid-budget contemporary drama rarely needs one; and high-volume short-form or vertical content is realistically an AI-assisted and conversion job, because volume economics do not work at that cadence.
Choosing the right mix for your next project
Four questions settle most projects. What is the scale, and will environments be reused? Does actor-environment light interaction matter to the look? How much creative exploration does the project need before the shoot? How much derivative material will the release require? Most productions of any size end up using both approaches. The useful planning question is where each one sits, not which one wins.
If you are genuinely unsure, keep your options open. Design environments as assets rather than sets, so they can serve as a volume build, a generated reference or a post element. Keep previs tool-agnostic so the plan survives a change of method. And decide the lighting approach early, because that choice constrains which method remains available.
If you are planning the pre-production side, our walkthrough of a script-to-screen AI workflow and our guide to AI filmmaking show how the pieces connect. Big Idea, CineVision’s AI-assisted production platform, supports that work from script to storyboard to pre-viz and render, with prompt-based control over locations, characters and props, consistent characters and environments across shots, and prompted VFX on footage you have already shot, whether it came from a location, a green screen or an LED volume.
