AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models".
Dev: Here's the gist of the paper. AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional demonstration data, policy fine-tuning, or architectural modification.
Rosa: This approach preserves the pre-trained capabilities of Vision-Language Action Models (VLAs) and enables robust task execution across diverse setups including different extrinsics, intrinsics, and freely moving handheld cameras.
Dev: Pruning redundant reasoning tokens The core idea of the framework is to virtually adjust test-time camera observations to match the training camera configuration in real-time.
Rosa: This is achieved by using a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters.
Paper summary: Rosa: So, we're looking at this paper now, AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models. The main idea here is that you can take a vision language action model and use it in totally different camera setups without needing any extra training data or changing the model itself.
Dev: Right, so the core thesis is to virtually adjust what the camera sees at test time to match how it was trained, all in real-time. It claims this lets you handle different camera positions, intrinsics, and even handheld cameras while keeping those pre-trained capabilities intact.
Taro: But that sounds like a lot of work for the robot. How does this actually work without needing to retrain anything? And what kind of robustness are they talking about when the world misbehaves?
Rosa: Well, it says they use a feed-forward novel view synthesis model to generate high-quality target view images that handle both extrinsic and intrinsic parameters for the test camera. This adaptation runs in real-time and plugs directly into any Vision Language Action Model.
Dev: The math behind it is that they transform the test time observation, which we call I test t, using a module F to get a synthetic image called I train t, where I train t equals F of I test t, C test, C train.
Taro: So you're taking the current view and projecting it to look like it came from the training camera configuration using this synthesis model? That sounds like a clever trick for geometry.
Rosa: Exactly. They are drawing inspiration from novel view synthesis to generate precise projective warping of three dimensional geometry, which they say is more robust than just traditional image warping when the multi-view geometry is inaccurate.
Dev: The point they make is that this model can handle various changes in pose, focal length, and lighting without having to modify the original Vision Language Action Model. It’s plug-and-play for any RGB based policy.
Taro: But if it's plugging into an existing VLA, what's the risk? Does this approach inject extra features or change how the model reasons internally?
Rosa: It preserves the pre-trained capabilities of those VLAs by only adjusting the input observation, not modifying the policy itself. They say this avoids catastrophic forgetting associated with retraining, which is a big deal for deployment.
Dev: The performance claims are pretty strong on their benchmark, LIBERO. They state that their method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional three dimensional aware features for visual input.
Taro: I saw they mention something about agent camera perturbations achieving a ninety nine point zero percent success rate on unseen agent camera viewpoint across all LIBERO suites. That’s a high bar for generalization in the real world.
Paper summary: Rosa: And they also showed that on the most challenging LIBERO Long suite with wrist camera perturbations, their method, called Ours-pi, achieved an eighty eight point six percent average success rate when compared to data augmentation and GeoAwareVLA.
Dev: The ablation study on viewpoint adaptation methods is interesting too. They found that the feed-forward novel view synthesis approach significantly outperforms geometric baselines on both image synthesis quality and task success rate across all perturbation levels.
Taro: Why did they find that? Because depth based projection showed improvement over no adaptation and homography, but it had these non photorealistic artifacts from point cloud projection when there were large viewpoint shifts.
Rosa: That’s because the feed-forward novel view synthesis leverages learned priors to accurately reconstruct three dimensional geometry from multi-view images while naturally synthesizing photorealistic images, which they link to that superior image quality.
Dev: The paper notes that the camera adaptation module F is flexible with respect to how many input and output cameras there are, meaning N and M can be different in some cases. This flexibility is important for deployment scenarios where calibration happens once during setup.
Taro: But what about the practical side? They mention that feed-forward novel view synthesis introduces a latency of approximately thirty milliseconds per frame, which could be an issue if you're dealing with extremely dynamic camera movements.
Rosa: That’s a known trade-off. The authors acknowledge that this latency might pose challenges in very fast, dynamic scenarios and requires more GPU memory during inference to run smoothly.
Dev: So, the paper says it can operate under dynamic camera conditions by generalizing not just to changes in camera extrinsics but also to variations in other parameters like intrinsics and image characteristics across different camera models.
Taro: It sounds like they are tackling a lot of real-world variation without needing any new demonstrations or retraining cycles, which is a huge practical win for deploying VLAs on robots.
Rosa: The conclusion is that this zero shot camera adaptation framework effectively generalizes to novel viewpoints in real world manipulation scenarios while maintaining task success rates comparable to those achieved with the training viewpoint.
Dev: The title, AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models, points directly at how they are achieving this without the usual heavy lifting of demonstration data or policy fine tuning.
Taro: I think what this means for people using these models is that you don't have to spend all your time collecting massive datasets just to make the model work in a slightly different camera position on a new robot setup.
Rosa: It simplifies the deployment pipeline significantly, making it much more practical for researchers and engineers working outside of perfectly controlled lab settings.
Dev: So, in short, they've found a way to adapt models to arbitrary camera configurations in real time just by using this novel view synthesis approach, which is much more efficient than fine tuning the entire VLA.
Conclusion: Rosa: So, we're looking at AnyCamVLA, and the title says zero shot camera adaptation for viewpoint robust vision language action models. Basically, they figured out how to make a robot model work in a camera position it was never trained on just by adjusting the picture in real time.
Dev: Right. They’re talking about this framework that virtually adjusts test time camera observations to match the training camera configuration without needing any extra training data or policy fine tuning.
Taro: So, no new robot demonstrations required for every new viewpoint? That sounds like a huge win for getting these models out into the real world quickly.
Rosa: Exactly. The core idea is using a feed forward novel view synthesis model to generate those target view images in real time, handling both the camera position and its internal settings.
Dev: And that’s how they build the training signal; they transform the test time observation by feeding it through this module F, so the policy runs on that synthetic image instead of the actual one.
Taro: But what about when things go wrong? If you're driving or cooking and your camera suddenly moves in a weird way, how does this system handle that surprise?
Rosa: Well, they show strong performance on the LIBERO benchmark. They got ninety nine point zero percent success rate on unseen agent camera viewpoints across all those tests.
Dev: And they also achieved eighty eight point six percent average success rate on the most difficult suite involving wrist camera perturbations. That’s solid generalization for a system that hasn't seen that specific setup before.
Taro: What I'm really interested in is the trade off, since we know this runs in real time. They admitted there’s about thirty milliseconds of latency per frame, and it needs more GPU memory during inference.
Rosa: Yeah, that latency is something to watch for when you deploy it on a very fast dynamic system. But they say it still works under dynamic camera conditions, handling changes in intrinsics and image characteristics too.
Dev: So the big picture here is that this method bypasses the major headaches of getting viewpoint generalization across many different tasks or risking catastrophic forgetting from policy fine tuning.
Taro: It makes deployment way more practical because you don't need massive datasets just to make it work in a slightly different camera setup.
Rosa: It really simplifies the entire pipeline, moving us closer to deploying these vision language models in messy real-world scenarios without needing perfect lab conditions or endless training cycles.
Dev: Next up, we're going to look at how they compared this novel view synthesis approach against other geometric methods and why that choice matters for quality.
Seoul National University · Massachusetts Institute of Technology
cs.RO
Submitted: 2026-03-06
Updated: 2026-11-11
Comments: Accepted to IROS 2026
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional…
Key concepts
- Zero-Shot Camera Adaptation
- This is a framework that allows an existing VLA, trained on one set of cameras, to work successfully with entirely new camera views it has never seen before. It achieves this by virtually transforming the test-time images to resemble the training conditions in real-time, eliminating the need for new demonstrations or retraining.
- Novel View Synthesis Model
- This is a feed-forward model used to generate high-quality target view images from a source image and camera parameters. It is crucial because it handles both changes in camera position (extrinsic) and internal lens settings (intrinsic), enabling the virtual adjustment of test observations.
- Policy Inference Formulation
- The method reformulates how the VLA policy makes decisions. Instead of feeding raw test images, it feeds the synthesized, adapted images into the frozen VLA policy. This keeps the original visual reasoning capabilities intact while adapting to arbitrary camera configurations at inference time.
- LVSM Fine-tuning for Domain Adaptation
- This refers to a one-time process where a smaller model (LVSM) is fine-tuned using multi-view data. This pre-adaptation step creates a universal view adapter that can be plugged into any VLA, making the overall adaptation highly efficient and practical.
Terminology
Summary
AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional demonstration data, policy fine-tuning, or architectural modification. p. 0 In this paper, we propose a zero-shot camera
adaptation framework without additional demonstration data,
policy fine-tuning, or architectural modification. Our key idea
is to virtually adjust test-time camera observations to match the
training camera configuration in real-time. At each control step, images from a test-time
camera are synthesized into the training camera viewpoint at 30 Hz, while
the frozen VLA policy runs at 10 Hz. Without any policy fine-tuning, our
framework enables robust task execution across diverse setups—including
different extrinsics, intrinsics, and freely moving handheld cameras such as
an iPhone, ZED, and RealSense.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The core idea of the framework is to virtually adjust test-time camera observations to match the training camera configuration in real-time. This is achieved by using a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. p. 0 Our key idea
is to virtually adjust test-time camera observations to match the
training camera configuration in real-time. For that, we use a
recent feed-forward novel view synthesis model which outputs
high-quality target view images, handling both extrinsic and
intrinsic parameters. Such a model handles only visual input, which
is significantly smaller than the conventional VLAs. The
viewpoint adaptation runs in real-time and can be plugged
into the input of any VLAs to robustly process realistic
variations.Where the paper says it
Where the paper says it
How the training signal is built
The framework operates by transforming test-time observations using a camera adaptation module F, such that ˆI train t = F(I test t, C test, C train). The adapted policy inference then becomes at = πθ(F(I test t, C test, C train), l). p. 2 ,
N and M can differ. The adapted policy inference then
becomes:
at = πθ(F(I
test
t
, C
test
, C
train), l). The adapted policy inference then
becomes:
at = πθ(F(I
test
t
, C
test
, C
train), l). (3)
This formulation allows us to leverage the pre-trained policy
without modification while adapting to arbitrary camera
configurations at test time. (3)
This formulation allows us to leverage the pre-trained policy
without modification while adapting to arbitrary camera
configurations at test time. We implement the camera adaptation module F using
a feed-forward novel view synthesis model [24], which
can generate photorealistic images from arbitrary viewpoints
given a set of source images and their corresponding camera
parameters.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The method offers several key benefits, including requiring no additional robot demonstrations or policy fine-tuning, which reduces computational cost and the risk of catastrophic forgetting associated with retraining. It operates in a plug-and-play manner, making it applicable to any RGB-based VLA without architectural modifications. p. 2 First, it requires no additional robot demonstrations or policy fine-tuning, reducing the computational cost
and risk of catastrophic forgetting associated with retraining. Second, it operates in a plug-and-play manner, making it
applicable to any RGB-based VLA without architectural
modifications. Second, it operates in a plug-and-play manner, making it
applicable to any RGB-based VLA without architectural
modifications. Finally, unlike representation-centric methods
that inject additional features, our approach preserves the
original visual input modality, fully leveraging the pretrained visual-linguistic reasoning capabilities of VLMs.Where the paper says it
Where the paper says it
Performance and comparison
Extensive experiments on the LIBERO benchmark consistently show that the method outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. For agent camera perturbations, Ours-OV achieves 99. p. 3 To investigate the effectiveness of our method under
different camera configurations, we also conduct wrist camera perturbations on LIBERO-Long, the most challenging
suite with long-horizon tasks. As shown in Table II, our
method achieves 88. 2) Viewpoint Robustness Evaluation: Table I shows the
success rate on agent camera perturbations across all
LIBERO suites. Our methods consistently achieve the best
performance across all suites and perturbation levels, demonstrating robust viewpoint generalization. To investigate the effectiveness of our method under
different camera configurations, we also conduct wrist camera perturbations on LIBERO-Long, the most challenging
suite with long-horizon tasks. As shown in Table II, our
method achieves 88. As shown in Table II, our
method achieves 88. 6% average success rate, outperforming
both data augmentation and GeoAwareVLA. Even when finetuning with all 10 tasks, the forgetting problem persists,
suggesting that simply increasing the amount of viewpointaugmented data cannot fully prevent the degradation of
previously learned behaviors. These results highlight two fundamental challenges with
the fine-tuning approach: (1) achieving viewpoint generalization across diverse tasks requires demonstrations from a
large number of tasks, making data collection expensive and
Input Views w/o FT w/ FT GT
Sim Real
Fig.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Ablation on Viewpoint Adaptation Methods
An ablation study comparing feed-forward novel view synthesis with alternative geometric methods on LIBERO-Long reveals that the feed-forward novel view synthesis approach significantly outperforms geometric baselines on image synthesis quality and task success rate across all perturbation levels. While depth-based projection shows considerable improvement over no adaptation and homography, the non-photorealistic artifacts from point cloud projection under large viewpoint shifts limit the visual understanding of the VLA. p. 5 As shown in Table III, our method significantly outperforms geometric baselines on image synthesis quality and
task success rate across all perturbation levels. While depthbased projection shows considerable improvement over no
adaptation and homography, the non-photorealistic artifacts
from point cloud projection under large viewpoint shifts
limit the visual understanding of the VLA, despite providing
geometrically correct views. While depthbased projection shows considerable improvement over no
adaptation and homography, the non-photorealistic artifacts
from point cloud projection under large viewpoint shifts
limit the visual understanding of the VLA, despite providing
geometrically correct views. In contrast, feed-forward novel
view synthesis leverages learned priors to accurately reconstruct 3D geometry from multi-view images while naturally
synthesizing photorealistic images, which is supported by the
superior image quality to baselines. 4%
on original view), demonstrating that VLAs can effectively
process synthesized views when they are visually consistent
with training data. The critical role of fine-tuning LVSM is evident from the
poor performance of the pre-trained LVSM without finetuning, as illustrated in Figure 4.Where the paper says it
Where the paper says it
Where the paper says it
Real-World Validation
In real-world experiments, the zero-shot camera adaptation framework effectively improves performance on novel viewpoints, maintaining task success rates comparable to those achieved with the training viewpoint. This demonstrates that the framework effectively generalizes to novel viewpoints in real-world manipulation scenarios. p. 6 even a slight change in viewpoint is sufficient to degrade
task performance. In contrast, our method maintains task
success rates comparable to those achieved with the training
viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints
in real-world manipulation scenarios. The
pose of each handheld camera is estimated using ArUco
markers [50] attached to the work table. This experiment
demonstrates that our method can successfully operate under
dynamic camera conditions in real-time, generalizing not
only to changes in camera extrinsics but also to variations
in other camera parameters such as intrinsics and image
characteristics across different camera models. even a slight change in viewpoint is sufficient to degrade
task performance. In contrast, our method maintains task
success rates comparable to those achieved with the training
viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints
in real-world manipulation scenarios. even a slight change in viewpoint is sufficient to degrade
task performance. In contrast, our method maintains task
success rates comparable to those achieved with the training
viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints
in real-world manipulation scenarios. The
pose of each handheld camera is estimated using ArUco
markers [50] attached to the work table. This experiment
demonstrates that our method can successfully operate under
dynamic camera conditions in real-time, generalizing not
only to changes in camera extrinsics but also to variations
in other camera parameters such as intrinsics and image
characteristics across different camera models.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Limitations
The method has limitations, as it can fail in cases when the reconstruction quality of novel view synthesis degrades, for instance, when the source view is limited to a single camera and the target viewpoint is far from the source view, or large occluded regions are present in the input images. Additionally, feed-forward novel view synthesis introduces a latency of approximately 30 ms per frame, which may pose challenges in extremely dynamic scenarios and requires additional GPU memory during inference. p. 6 Our
method can fail in cases when the reconstruction quality
of novel view synthesis degrades, for instance, when the
source view is limited to a single camera and the target
viewpoint is far from the source view, or large occluded
regions are present in the input images. Also, feed-forward
novel view synthesis introduces a latency of approximately
30 ms per frame, which may pose challenges in extremely
dynamic scenarios, and requires additional GPU memory
during inference. Also, feed-forward
novel view synthesis introduces a latency of approximately
30 ms per frame, which may pose challenges in extremely
dynamic scenarios, and requires additional GPU memory
during inference. One open challenge lies in target viewpoint selection when the training camera configuration varies
across demonstrations. While our primary approach is zero-shot adaptation with
frozen policies, we note that the one-time LVSM fine-tuning
for domain adaptation is significantly more efficient than
VLA fine-tuning. First, LVSM is substantially smaller (171M
parameters) compared to VLAs (7B for OpenVLA-OFT,
3. More
importantly, LVSM fine-tuning only requires multi-view images without any robot action labels, making data collection
orders of magnitude easier and cheaper than collecting expert
demonstrations for VLA training. This makes our approach
highly practical: a single LVSM model fine-tuned once on
multi-view data can serve as a universal view adapter for any Training Views
Test Views
Our camera setting Objects used in experiments
Fig. In contrast, our zero-shot adaptation
framework circumvents both issues entirely. By keeping the
policy frozen and adapting only the visual input, our method
requires no additional robot demonstrations or training,
eliminating data collection and computational training costs.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Zero-Shot Camera Adaptation Framework: The system can perform robust task execution across diverse setups, including different camera extrinsics, intrinsics, and freely moving handheld cameras such as an iPhone, ZED, and RealSense without additional demonstration data or policy fine-tuning.
-
Real-time View Synthesis Integration: The framework integrates a feed-forward novel view synthesis model to virtually adjust test-time camera observations to match the training camera configuration in real-time, handling both extrinsic and intrinsic parameters at approximately 30 FPS while allowing the frozen VLA policy to run at 10 Hz.
-
Enhanced Viewpoint Robustness: The method consistently outperforms baselines on benchmarks like LIBERO by enabling robust generalization, even under large viewpoint shifts (up to 15 cm translation and 60° rotation in camera extrinsics), showing
the best task success rate over baselines.
-
Plug-and-Play Deployment: The approach operates in a plug-and-play manner, making it applicable to any RGBbased policy without architectural modifications, preserving the pre-trained capabilities of VLAs and avoiding the
catastrophic forgetting
associated with fine-tuning.
Abstract
Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
Sources
- RT-1: Robotics Transformer for Real-World Control at Scale
- Octo: An Open-Source Generalist Robot Policy
- OpenVLA: An Open-Source Vision-Language-Action Model
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- Evaluating Real-World Robot Manipulation Policies in Simulation
- LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
- 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
- GeoAware-VLA: Implicit Geometry Aware Vision-Language-Action Model
- Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learning
- FP3: A 3D Foundation Policy for Robotic Manipulation
- How Do VLAs Effectively Inherit from VLMs?
- From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors
- Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Grasping
- Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- Stereo Magnification: Learning View Synthesis using Multiplane Images
- Decoupled Weight Decay Regularization
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation