Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting

arXiv:2610.12455 · cs.HC, cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting".

Tom: The gist: Hybrid Cinematography presents a workflow connecting physical capture and generative reshooting around an editable 3D shot plan to manage hallucination risk before generation.

Jane: First, who's behind it and why it matters.

Title and authors: Jane: Let's look at what the authors are actually trying to solve with this approach, since they call it "Hybrid Cinematography". It’s about bridging that gap between physical capture and generative reshooting <ref:2610.12455#pg2>.

Tom: Exactly. They acknowledge that just capturing extra takes takes up a ton of time and you can't always get the exact moment again, but they also know AI hallucination is a big problem when you try to fill in those missing parts <ref:2610.12455#pg2>.

Lu: They frame it as navigating a spectrum between what you actually capture and what the final target move should be, and they want to make that range continuously navigable within a single take <ref:2610.12455#pg3>.

Meng: So it’s not just one path; it’s about exploring the trade-offs between capturing more physical footage versus trusting the generative model to invent what's missing <ref:2610.12455#pg4>.

Tom: Right. It moves the decision from a binary choice—capture or hallucinate—to a continuous spectrum where you can see the risk at every point <ref:2610.12455#pg4>.

The paper's summary: Jane: The core of this method is previsualizing the risk before you even hit record, by checking the recorded take against a lightweight three dee proxy <ref:2610.12455#pg5>.

Tom: That proxy lets them check three specific things about your capture: coverage, resolution, and performance <ref:2610.12455#pg5>. Those are the main deficits they look at.

Lu: They calculate these deficits by rendering the proxy along a move and testing how well it supports the target view <ref:2610.12455#pg5>. Specifically, they check if coverage holds when at least one source observation falls within an angular cone around the target viewing direction <ref:2610.12455#pg5>.

Meng: So it’s a real-time check on whether the recorded footage actually gives you enough information to make the next AI step reliable <ref:2610.12455#pg9>. That sounds like something that could save a lot of time on set.

Tom: It does, because instead of waiting for the generative model to fail later, you get an immediate signal about what’s wrong with your current setup <ref:2610.12455#pg9>.

The paper's improvements: Jane: One big improvement they propose is that this system gives filmmakers explicit guidance on how to handle those deficits, mapping each one to a specific signal and response <ref:2610.12455#pg4>.

Tom: So if the coverage is low, the system suggests a targeted pickup task—meaning you capture more footage right there on set <ref:2610.12455#pg4>. Or maybe for resolution, it suggests something else entirely <ref:2610.12455#pg4>.

Lu: And crucially, they let users know they can knowingly accept hallucination by confirming an acceptance mask tied to the current shot plan <ref:2610.12455#pg7>. That gives the creator control over what level of invention they are okay with <ref:2610.12455#pg7>.

Meng: That’s a big deal for workflow because it gives you choices rather than just forcing you down one path when things get messy <ref:2610.12455#pg4>.

Tom: It turns the problem of managing hallucination into a decision-making process where the filmmaker is still in charge of the final output <ref:2610.12455#pg9>.

Conclusion: Jane: So to wrap up, this whole "Hybrid Cinematography" workflow is about connecting capture and generation around that one editable three dee shot plan, making sure you know the risk before you generate anything <ref:2610.12455#pg4>.

Tom: It’s about balancing the physical effort of capturing more footage against the risk of hallucinations by giving filmmakers active control over what goes into the reshooting process <ref:2610.12455#pg9>.

Lu: They suggest a strategy for other creative tools where you can factorize user intent into separate components, which lets practitioners author the core expressive part of the work while delegating routine infill to the model <ref:2610.12455#pg8>.

Meng: From an engineering standpoint, they show that running a full video diffusion model live would introduce latency and compute demands that immediately stall an active crew, so this separation is important for real production <ref:2610.12455#pg8>.

Lalam: I see how this works in terms of culture too. If we can make the dependencies on the AI immediately interpretable through these proxies, it means creators can immediately understand what the tools will and won't hallucinate <ref:2610.12455#pg9>.

Tom: It’s a new way to think about how we use these models in production, anchoring every stage to that shot plan they keep checking against a three dee proxy <ref:2610.12455#pg10>.

Jane: That shared representation really seems essential for coordinating everything from planning through the final camera move verification <ref:2610.12455#pg9>.

Lu: It suggests that using these shared representations as conditioning signals is going to be essential as practitioners navigate that spectrum in production <ref:2610.12455#pg9>.

Meng: The final result is bounded by downstream model constraints, which means the diagnostic pipeline can act like a plug-and-play front end for the generative part <ref:2610.12455#pg8>.

Lalam: So essentially, previsualizing hallucination risk establishes that clear boundary of visual responsibility for whoever is using the tools <ref:2610.12455#pg8>.

Nhan Tran, Neal Wadhwa, Abe Davis, Stefan Stojanov

Cornell University · Google

cs.HC, cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: Project page: https://hybridcinematography.github.io/

Project page: https://hybridcinematography.github.io/Abstract

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: The gist: Hybrid Cinematography presents a workflow connecting physical capture and generative reshooting around an editable 3D shot plan to manage hallucination risk before generation.

Key concepts

Hybrid Cinematography: A workflow that links physical video capture with generative reshooting using an editable 3D shot plan. This system manages the risk of AI hallucination by previsualizing potential issues before generating new content, allowing creators to control the balance between captured footage and model-generated material.
Editable 3D Shot Plan: A central, single 3D plan that serves as the foundation for every stage of production. Every step in the workflow—from planning to capturing and reviewing—reads this plan. It acts as the consistent reference point ensuring all actions align with the desired final shot trajectory.
Previsualizing Hallucination Risk: A real-time evaluation process that checks recorded video against a 3D proxy to assess three deficits: coverage, resolution, and performance. This system determines how well the captured footage supports the target view based on angular cones around the viewing direction, making model dependencies immediately interpretable.
Trade-offs Continuum: The spectrum between captured and target camera moves that filmmakers navigate using explicit guidance. The system maps specific deficits to suggested responses, such as requesting more capture or knowingly accepting hallucination. This allows creators to author their visual ambition against model support.

Terminology

Hallucination — 81× in this paper · explained in 4 episodes
Generative Model — 16× in this paper · explained in 2 episodes
Actor — 7× in this paper · explained in 3 episodes
Generative Models — 7× in this paper · explained in 3 episodes
Core — 5× in this paper · explained in 2 episodes
Model-Agnostic — 5× in this paper · explained in 2 episodes
Decoupling — 4× in this paper · explained in 2 episodes
Diffusion Model — 4× in this paper · explained in 10 episodes
Conference on Computer Vision and Pattern Recognition — 3× in this paper
Editor — 3× in this paper · explained in 2 episodes
Fidelity — 3× in this paper · explained in 2 episodes
Papers — 3× in this paper · explained in 3 episodes
Steering — 3× in this paper · explained in 2 episodes
CVF International Conference on Computer Vision — 2× in this paper
Point Clouds — 2× in this paper · explained in 2 episodes
Adam — 1× in this paper · explained in 2 episodes
Diffusion Models — 1× in this paper · explained in 9 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Generative AI — 1× in this paper · explained in 2 episodes
Generative Modeling — 1× in this paper · explained in 3 episodes
Generator — 1× in this paper · explained in 2 episodes
Parameterization — 1× in this paper · explained in 2 episodes
Progressive Disclosure — 1× in this paper · explained in 2 episodes
Trace — 1× in this paper · explained in 2 episodes
Verification — 1× in this paper · explained in 2 episodes

Summary

The gist: Hybrid Cinematography presents a workflow connecting physical capture and generative reshooting around an editable 3D shot plan to manage hallucination risk before generation.

Hybrid Cinematography Workflow

The workflow is built around one editable 3D shot plan that every stage reads <ref:2610.12455#pg2> and involves a sequence of steps: "a Shot planning b AR-guided capture far: follow trail near: match pose c Review & refine <ref:2610.12455#pg2>. This process allows filmmakers to rapidly explore moves between the captured and target trajectories, inspecting trade-offs between physical capture and hallucination without waiting for the generative model <ref:2610.12455#pg4>. The workflow is demonstrated both on set and during editing" <ref:2610.12455#pg4>.

Previsualizing Hallucination Risk

The core of the system is a previsualization that evaluates risk in real time by checking the recorded take against a lightweight 3D proxy <ref:2610.12455#pg5>. This evaluation is based on three distinct capture deficits: coverage, resolution, and performance <ref:2610.12455#pg5>. These deficits are computed by rendering the proxy along a move and testing support against the target view <ref:2610.12455#pg5>. The system evaluates support based on whether Coverage holds when at least one source observation falls within an angular cone around the target viewing direction <ref:2610.12455#pg5>.

Navigating Trade-offs and Responses

The workflow provides explicit guidance for filmmakers to navigate the continuum between captured and target moves <ref:2610.12455#pg4>. The system allows users to revise the move, capture more, re-time, accept hallucination <ref:2610.12455#pg4>. The interface maps each deficit to a specific signal and response <ref:2610.12455#pg4>. For instance, for coverage or resolution deficits, the system suggests a targeted pickup task <ref:2610.12455#pg4>. Filmmakers can also knowingly accept hallucination by confirming an acceptance mask tied to the current shot plan <ref:2610.12455#pg7>.

Study Findings

A study with seven experienced filmmakers revealed how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination <ref:2610.12455#pg4>. Participants rated the previs a median of 7 of 7 for showing where and why the model must invent content <ref:2610.12455#pg9>. They valued knowing on set what the footage supported <ref:2610.12455#pg9>. Furthermore, comfort with generative reshooting increased for all participants, rising from a median of 4 before the study to 6 after experiencing the on-set system <ref:2610.12455#pg9>.

Conclusion

The key contribution is Hybrid Cinematography, a workflow for navigating trade-offs along the continuum between captured and target camera moves, balancing physical capture against generative hallucination <ref:2610.12455#pg4>. This approach allows filmmakers to evaluate hallucination risk on set and retain active control over what the model receives <ref:2610.12455#pg9>. The system suggests a transferable design strategy for creative human-AI tools by factorizing user intent into separable components <ref:2610.12455#pg8>. This enables practitioners to author the expressive core of a work themselves while delegating routine infill to the model <ref:2610.12455#pg8>. The workflow suggests that Previsualizing hallucination risk establishes a clear boundary of visual responsibility <ref:2610.12455#pg8>. This selective delegation offers a model for other domains where visual authenticity is critical <ref:2610.12455#pg8>. The system suggests that shared representations like this proxy, which creators can visually steer and generative models can consume as conditioning, to become essential as practitioners navigate that spectrum in production <ref:2610.12455#pg9>. This is expected to define the future of the craft <ref:2610.12455#pg9>. The workflow suggests that Running a full video diffusion model live would introduce latency and compute demands that immediately stall an active crew <ref:2610.12455#pg8>. This architectural separation suggests a broader pattern for creative tools that integrate large generative models <ref:2610.12455#pg8>. The workflow suggests that Decoupling camera path from camera timing gives filmmakers a clean way to plan moves along this spectrum <ref:2610.12455#pg8>. This enables automated optimization for dynamic subjects <ref:2610.12455#pg8>. The workflow suggests that The compromise slider lets creators explore a continuous design space <ref:2610.12455#pg9>. This turns camera selection into an authored trade-off between visual ambition and model support <ref:2610.12455#pg8>. The workflow suggests that Previsualizing hallucination risk makes model dependencies immediately interpretable <ref:2610.12455#pg9>. This allows participants to immediately aware what the tools in post will and will not hallucinate <ref:2610.12455#pg9>. The workflow suggests that The system can search over it [timing] against the recorded take <ref:2610.12455#pg17>. This dynamic programming approach returns the cheapest schedule, with holds and hand-authored spans frozen so a search cannot undo a deliberate decision <ref:2610.12455#pg17>. The workflow suggests that The final result is bounded by downstream model constraints <ref:2610.12455#pg8>. This allows the diagnostic pipeline to function as a model-agnostic, plug-and-play front end <ref:2610.12455#pg8>. The workflow suggests that The workflow anchors each stage in the editable 3D shot plan checked against a 3D proxy of the take <ref:2610.12455#pg10>. This ensures that the shot plan, the target move as keyframes and timing, should be the one object every step works from <ref:2610.12455#pg10>. This shared representation coordinates production stages <ref:2610.12455#pg10>. The workflow suggests that The final camera move is verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. This leaves the finalized camera move, verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. The workflow suggests that The final result is bounded by downstream model constraints <ref:2610.12455#pg8>. This allows the diagnostic pipeline to function as a model-agnostic, plug-and-play front end <ref:2610.12455#pg8>. The workflow suggests that The final camera move is verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. This leaves the finalized camera move, verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. The workflow suggests that The final result is bounded by downstream model constraints <ref:2610.12455#pg8>. This allows the diagnostic pipeline to function as a model-agnostic, plug-and-play front end <ref:2610.12455#pg8>. The workflow suggests that The final camera move is verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. This leaves the finalized camera move, verified pickup takes, and confirmed acceptance masks for the generative reshooting model <ref:2610.12455#pg9>. The workflow suggests that The final result is bounded by downstream model constraints <ref:2610.12455#pg8>. This allows the diagnostic pipeline to function as a model-agnostic, plug-and-play front end <ref:2610.

Improvements for AI systems

Previsualization of Hallucination Risk Before Generation: The system can evaluate hallucination risk in real time as filmmakers revise the move, by constructing a 3D proxy of the take and testing it against three distinct deficits: coverage, resolution, and performance.
Navigable Trade-off Exploration: The system can make the range between moves navigable by interpolating camera poses along a continuous path, allowing filmmakers to explore trade-offs without calling the model, as described by Equation (1) where the camera trajectory is defined as: Clambda (t) = mixCc (t),Co (t), lambda, lambda ∈ [0, 1].
Deficit-Specific Response Mapping: The system can map each deficit to a specific action by providing a response card tailored to that deficit, allowing filmmakers to choose between actions like Revise the move, Capture more, or Accept hallucination.
Contextual Decision Support for Synthesis: The system can guide decision-making by showing the consequences of different choices, as demonstrated in Figure 12, where it shows how "Original timing prolongs views the generator must hallucinate; retiming preserves the full path and 3.5-second duration, lingering where evidence is stronger and whip-panning through poorly supported views to reduce hallucination."
Unified Shot Plan Management: The system can maintain a single source of truth by ensuring one editable 3D shot plan throughout, which guides the take, it is what every stage works from (DG4).

Abstract

On a film set, the camera move is committed during a take. Generative video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept hallucination. We demonstrate the workflow through a mobile augmented reality application for on-set planning, capture, and review, and an offline pipeline for existing video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination.