AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models

arXiv:2603.05868 · cs.RO · Submitted 2026-03-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models".

Dev: Here's the gist of the paper. AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional demonstration data, policy fine-tuning, or architectural modification.

Rosa: This approach preserves the pre-trained capabilities of Vision-Language Action Models (VLAs) and enables robust task execution across diverse setups including different extrinsics, intrinsics, and freely moving handheld cameras.

Dev: Pruning redundant reasoning tokens The core idea of the framework is to virtually adjust test-time camera observations to match the training camera configuration in real-time.

Rosa: This is achieved by using a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters.

Paper summary: Rosa: So, we're looking at this paper now, AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models. The main idea here is that you can take a vision language action model and use it in totally different camera setups without needing any extra training data or changing the model itself.

Dev: Right, so the core thesis is to virtually adjust what the camera sees at test time to match how it was trained, all in real-time. It claims this lets you handle different camera positions, intrinsics, and even handheld cameras while keeping those pre-trained capabilities intact.

Taro: But that sounds like a lot of work for the robot. How does this actually work without needing to retrain anything? And what kind of robustness are they talking about when the world misbehaves?

Rosa: Well, it says they use a feed-forward novel view synthesis model to generate high-quality target view images that handle both extrinsic and intrinsic parameters for the test camera. This adaptation runs in real-time and plugs directly into any Vision Language Action Model.

Dev: The math behind it is that they transform the test time observation, which we call I test t, using a module F to get a synthetic image called I train t, where I train t equals F of I test t, C test, C train.

Taro: So you're taking the current view and projecting it to look like it came from the training camera configuration using this synthesis model? That sounds like a clever trick for geometry.

Rosa: Exactly. They are drawing inspiration from novel view synthesis to generate precise projective warping of three dimensional geometry, which they say is more robust than just traditional image warping when the multi-view geometry is inaccurate.

Dev: The point they make is that this model can handle various changes in pose, focal length, and lighting without having to modify the original Vision Language Action Model. It’s plug-and-play for any RGB based policy.

Taro: But if it's plugging into an existing VLA, what's the risk? Does this approach inject extra features or change how the model reasons internally?

Rosa: It preserves the pre-trained capabilities of those VLAs by only adjusting the input observation, not modifying the policy itself. They say this avoids catastrophic forgetting associated with retraining, which is a big deal for deployment.

Dev: The performance claims are pretty strong on their benchmark, LIBERO. They state that their method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional three dimensional aware features for visual input.

Taro: I saw they mention something about agent camera perturbations achieving a ninety nine point zero percent success rate on unseen agent camera viewpoint across all LIBERO suites. That’s a high bar for generalization in the real world.

Paper summary: Rosa: And they also showed that on the most challenging LIBERO Long suite with wrist camera perturbations, their method, called Ours-pi, achieved an eighty eight point six percent average success rate when compared to data augmentation and GeoAwareVLA.

Dev: The ablation study on viewpoint adaptation methods is interesting too. They found that the feed-forward novel view synthesis approach significantly outperforms geometric baselines on both image synthesis quality and task success rate across all perturbation levels.

Taro: Why did they find that? Because depth based projection showed improvement over no adaptation and homography, but it had these non photorealistic artifacts from point cloud projection when there were large viewpoint shifts.

Rosa: That’s because the feed-forward novel view synthesis leverages learned priors to accurately reconstruct three dimensional geometry from multi-view images while naturally synthesizing photorealistic images, which they link to that superior image quality.

Dev: The paper notes that the camera adaptation module F is flexible with respect to how many input and output cameras there are, meaning N and M can be different in some cases. This flexibility is important for deployment scenarios where calibration happens once during setup.

Taro: But what about the practical side? They mention that feed-forward novel view synthesis introduces a latency of approximately thirty milliseconds per frame, which could be an issue if you're dealing with extremely dynamic camera movements.

Rosa: That’s a known trade-off. The authors acknowledge that this latency might pose challenges in very fast, dynamic scenarios and requires more GPU memory during inference to run smoothly.

Dev: So, the paper says it can operate under dynamic camera conditions by generalizing not just to changes in camera extrinsics but also to variations in other parameters like intrinsics and image characteristics across different camera models.

Taro: It sounds like they are tackling a lot of real-world variation without needing any new demonstrations or retraining cycles, which is a huge practical win for deploying VLAs on robots.

Rosa: The conclusion is that this zero shot camera adaptation framework effectively generalizes to novel viewpoints in real world manipulation scenarios while maintaining task success rates comparable to those achieved with the training viewpoint.

Dev: The title, AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models, points directly at how they are achieving this without the usual heavy lifting of demonstration data or policy fine tuning.

Taro: I think what this means for people using these models is that you don't have to spend all your time collecting massive datasets just to make the model work in a slightly different camera position on a new robot setup.

Rosa: It simplifies the deployment pipeline significantly, making it much more practical for researchers and engineers working outside of perfectly controlled lab settings.

Dev: So, in short, they've found a way to adapt models to arbitrary camera configurations in real time just by using this novel view synthesis approach, which is much more efficient than fine tuning the entire VLA.

Conclusion: Rosa: So, we're looking at AnyCamVLA, and the title says zero shot camera adaptation for viewpoint robust vision language action models. Basically, they figured out how to make a robot model work in a camera position it was never trained on just by adjusting the picture in real time.

Dev: Right. They’re talking about this framework that virtually adjusts test time camera observations to match the training camera configuration without needing any extra training data or policy fine tuning.

Taro: So, no new robot demonstrations required for every new viewpoint? That sounds like a huge win for getting these models out into the real world quickly.

Rosa: Exactly. The core idea is using a feed forward novel view synthesis model to generate those target view images in real time, handling both the camera position and its internal settings.

Dev: And that’s how they build the training signal; they transform the test time observation by feeding it through this module F, so the policy runs on that synthetic image instead of the actual one.

Taro: But what about when things go wrong? If you're driving or cooking and your camera suddenly moves in a weird way, how does this system handle that surprise?

Rosa: Well, they show strong performance on the LIBERO benchmark. They got ninety nine point zero percent success rate on unseen agent camera viewpoints across all those tests.

Dev: And they also achieved eighty eight point six percent average success rate on the most difficult suite involving wrist camera perturbations. That’s solid generalization for a system that hasn't seen that specific setup before.

Taro: What I'm really interested in is the trade off, since we know this runs in real time. They admitted there’s about thirty milliseconds of latency per frame, and it needs more GPU memory during inference.

Rosa: Yeah, that latency is something to watch for when you deploy it on a very fast dynamic system. But they say it still works under dynamic camera conditions, handling changes in intrinsics and image characteristics too.

Dev: So the big picture here is that this method bypasses the major headaches of getting viewpoint generalization across many different tasks or risking catastrophic forgetting from policy fine tuning.

Taro: It makes deployment way more practical because you don't need massive datasets just to make it work in a slightly different camera setup.

Rosa: It really simplifies the entire pipeline, moving us closer to deploying these vision language models in messy real-world scenarios without needing perfect lab conditions or endless training cycles.

Dev: Next up, we're going to look at how they compared this novel view synthesis approach against other geometric methods and why that choice matters for quality.

Seoul National University · Massachusetts Institute of Technology

cs.RO

Submitted: 2026-03-06

Updated: 2026-11-11

Comments: Accepted to IROS 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional…

Key concepts

Zero-Shot Camera Adaptation
This is a framework that allows an existing VLA, trained on one set of cameras, to work successfully with entirely new camera views it has never seen before. It achieves this by virtually transforming the test-time images to resemble the training conditions in real-time, eliminating the need for new demonstrations or retraining.
Novel View Synthesis Model
This is a feed-forward model used to generate high-quality target view images from a source image and camera parameters. It is crucial because it handles both changes in camera position (extrinsic) and internal lens settings (intrinsic), enabling the virtual adjustment of test observations.
Policy Inference Formulation
The method reformulates how the VLA policy makes decisions. Instead of feeding raw test images, it feeds the synthesized, adapted images into the frozen VLA policy. This keeps the original visual reasoning capabilities intact while adapting to arbitrary camera configurations at inference time.
LVSM Fine-tuning for Domain Adaptation
This refers to a one-time process where a smaller model (LVSM) is fine-tuned using multi-view data. This pre-adaptation step creates a universal view adapter that can be plugged into any VLA, making the overall adaptation highly efficient and practical.

Terminology

Summary

AnyCamVLA proposes a zero-shot camera adaptation framework that virtually adjusts test-time camera observations to match training camera configurations in real-time without requiring additional demonstration data, policy fine-tuning, or architectural modification. p. 0

Where the paper says it

In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time.

This approach preserves the pre-trained capabilities of Vision-Language Action Models (VLAs) and enables robust task execution across diverse setups including different extrinsics, intrinsics, and freely moving handheld cameras. p. 0
Where the paper says it

At each control step, images from a test-time camera are synthesized into the training camera viewpoint at 30 Hz, while the frozen VLA policy runs at 10 Hz. Without any policy fine-tuning, our framework enables robust task execution across diverse setups—including different extrinsics, intrinsics, and freely moving handheld cameras such as an iPhone, ZED, and RealSense.

Pruning redundant reasoning tokens

The core idea of the framework is to virtually adjust test-time camera observations to match the training camera configuration in real-time. This is achieved by using a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. p. 0

Where the paper says it

Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters.

The adaptation runs in real-time and can be plugged into the input of any VLAs to robustly process realistic variations. p. 1
Where the paper says it

Such a model handles only visual input, which is significantly smaller than the conventional VLAs. The viewpoint adaptation runs in real-time and can be plugged into the input of any VLAs to robustly process realistic variations.

How the training signal is built

The framework operates by transforming test-time observations using a camera adaptation module F, such that ˆI train t = F(I test t, C test, C train). The adapted policy inference then becomes at = πθ(F(I test t, C test, C train), l). p. 2

Where the paper says it

, N and M can differ. The adapted policy inference then becomes: at = πθ(F(I test t , C test , C train), l).

This formulation allows the frozen VLA policy to be leveraged while adapting to arbitrary camera configurations at test time. p. 2
Where the paper says it

The adapted policy inference then becomes: at = πθ(F(I test t , C test , C train), l). (3) This formulation allows us to leverage the pre-trained policy without modification while adapting to arbitrary camera configurations at test time.

The module F is implemented using a feed-forward novel view synthesis model [24]. p. 2
Where the paper says it

(3) This formulation allows us to leverage the pre-trained policy without modification while adapting to arbitrary camera configurations at test time. We implement the camera adaptation module F using a feed-forward novel view synthesis model [24], which can generate photorealistic images from arbitrary viewpoints given a set of source images and their corresponding camera parameters.

Where the gains come from

The method offers several key benefits, including requiring no additional robot demonstrations or policy fine-tuning, which reduces computational cost and the risk of catastrophic forgetting associated with retraining. It operates in a plug-and-play manner, making it applicable to any RGB-based VLA without architectural modifications. p. 2

Where the paper says it

First, it requires no additional robot demonstrations or policy fine-tuning, reducing the computational cost and risk of catastrophic forgetting associated with retraining. Second, it operates in a plug-and-play manner, making it applicable to any RGB-based VLA without architectural modifications.

Unlike representation-centric methods that inject additional features, this approach preserves the original visual input modality and fully leverages the pretrained visual-linguistic reasoning capabilities of VLMs. p. 2
Where the paper says it

Second, it operates in a plug-and-play manner, making it applicable to any RGB-based VLA without architectural modifications. Finally, unlike representation-centric methods that inject additional features, our approach preserves the original visual input modality, fully leveraging the pretrained visual-linguistic reasoning capabilities of VLMs.

Performance and comparison

Extensive experiments on the LIBERO benchmark consistently show that the method outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. For agent camera perturbations, Ours-OV achieves 99. p. 3

Where the paper says it

To investigate the effectiveness of our method under different camera configurations, we also conduct wrist camera perturbations on LIBERO-Long, the most challenging suite with long-horizon tasks. As shown in Table II, our method achieves 88.

0% success rate on unseen agent camera viewpoint across all LIBERO suites, demonstrating robust viewpoint generalization. p. 3
Where the paper says it

2) Viewpoint Robustness Evaluation: Table I shows the success rate on agent camera perturbations across all LIBERO suites. Our methods consistently achieve the best performance across all suites and perturbation levels, demonstrating robust viewpoint generalization.

Furthermore, on the most challenging LIBERO-Long suite with wrist camera perturbations, Ours-π achieves 88. p. 3
Where the paper says it

To investigate the effectiveness of our method under different camera configurations, we also conduct wrist camera perturbations on LIBERO-Long, the most challenging suite with long-horizon tasks. As shown in Table II, our method achieves 88.

6% average success rate, outperforming both data augmentation and GeoAwareVLA. p. 3
Where the paper says it

As shown in Table II, our method achieves 88. 6% average success rate, outperforming both data augmentation and GeoAwareVLA.

The zero-shot adaptation framework circumvents the issues of achieving viewpoint generalization across diverse tasks requiring demonstrations from a large number of tasks and the risk of catastrophic forgetting inherent in policy fine-tuning. p. 4
Where the paper says it

Even when finetuning with all 10 tasks, the forgetting problem persists, suggesting that simply increasing the amount of viewpointaugmented data cannot fully prevent the degradation of previously learned behaviors. These results highlight two fundamental challenges with the fine-tuning approach: (1) achieving viewpoint generalization across diverse tasks requires demonstrations from a large number of tasks, making data collection expensive and Input Views w/o FT w/ FT GT Sim Real Fig.

Ablation on Viewpoint Adaptation Methods

An ablation study comparing feed-forward novel view synthesis with alternative geometric methods on LIBERO-Long reveals that the feed-forward novel view synthesis approach significantly outperforms geometric baselines on image synthesis quality and task success rate across all perturbation levels. While depth-based projection shows considerable improvement over no adaptation and homography, the non-photorealistic artifacts from point cloud projection under large viewpoint shifts limit the visual understanding of the VLA. p. 5

Where the paper says it

As shown in Table III, our method significantly outperforms geometric baselines on image synthesis quality and task success rate across all perturbation levels. While depthbased projection shows considerable improvement over no adaptation and homography, the non-photorealistic artifacts from point cloud projection under large viewpoint shifts limit the visual understanding of the VLA, despite providing geometrically correct views.

In contrast, feed-forward novel view synthesis leverages learned priors to accurately reconstruct 3D geometry from multi-view images while naturally synthesizing photorealistic images, which is supported by the superior image quality to baselines. p. 5
Where the paper says it

While depthbased projection shows considerable improvement over no adaptation and homography, the non-photorealistic artifacts from point cloud projection under large viewpoint shifts limit the visual understanding of the VLA, despite providing geometrically correct views. In contrast, feed-forward novel view synthesis leverages learned priors to accurately reconstruct 3D geometry from multi-view images while naturally synthesizing photorealistic images, which is supported by the superior image quality to baselines.

The critical role of fine-tuning LVSM is evident from the poor performance of the pre-trained LVSM without finetuning, as illustrated in Figure 4. p. 5
Where the paper says it

4% on original view), demonstrating that VLAs can effectively process synthesized views when they are visually consistent with training data. The critical role of fine-tuning LVSM is evident from the poor performance of the pre-trained LVSM without finetuning, as illustrated in Figure 4.

Real-World Validation

In real-world experiments, the zero-shot camera adaptation framework effectively improves performance on novel viewpoints, maintaining task success rates comparable to those achieved with the training viewpoint. This demonstrates that the framework effectively generalizes to novel viewpoints in real-world manipulation scenarios. p. 6

Where the paper says it

even a slight change in viewpoint is sufficient to degrade task performance. In contrast, our method maintains task success rates comparable to those achieved with the training viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints in real-world manipulation scenarios.

Moreover, the method can successfully operate under dynamic camera conditions by generalizing not only to changes in camera extrinsics but also to variations in other camera parameters such as intrinsics and image characteristics across different camera models. p. 6
Where the paper says it

The pose of each handheld camera is estimated using ArUco markers [50] attached to the work table. This experiment demonstrates that our method can successfully operate under dynamic camera conditions in real-time, generalizing not only to changes in camera extrinsics but also to variations in other camera parameters such as intrinsics and image characteristics across different camera models.

The framework maintains task success rates comparable to those achieved with the training viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints in real-world manipulation scenarios. p. 6
Where the paper says it

even a slight change in viewpoint is sufficient to degrade task performance. In contrast, our method maintains task success rates comparable to those achieved with the training viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints in real-world manipulation scenarios.

p. 6
Where the paper says it

even a slight change in viewpoint is sufficient to degrade task performance. In contrast, our method maintains task success rates comparable to those achieved with the training viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints in real-world manipulation scenarios.

The method can successfully operate under dynamic camera conditions in real-time, generalizing not only to changes in camera extrinsics but also to variations in other camera parameters such as intrinsics and image characteristics across different camera models. p. 6
Where the paper says it

The pose of each handheld camera is estimated using ArUco markers [50] attached to the work table. This experiment demonstrates that our method can successfully operate under dynamic camera conditions in real-time, generalizing not only to changes in camera extrinsics but also to variations in other camera parameters such as intrinsics and image characteristics across different camera models.

The framework maintains task success rates comparable to those achieved with the training viewpoint, demonstrating that our zero-shot camera adaptation framework effectively generalizes to novel viewpoints in real-world manipulation scenarios.

Conclusion and Limitations

The method has limitations, as it can fail in cases when the reconstruction quality of novel view synthesis degrades, for instance, when the source view is limited to a single camera and the target viewpoint is far from the source view, or large occluded regions are present in the input images. Additionally, feed-forward novel view synthesis introduces a latency of approximately 30 ms per frame, which may pose challenges in extremely dynamic scenarios and requires additional GPU memory during inference. p. 6

Where the paper says it

Our method can fail in cases when the reconstruction quality of novel view synthesis degrades, for instance, when the source view is limited to a single camera and the target viewpoint is far from the source view, or large occluded regions are present in the input images. Also, feed-forward novel view synthesis introduces a latency of approximately 30 ms per frame, which may pose challenges in extremely dynamic scenarios, and requires additional GPU memory during inference.

An open challenge lies in target viewpoint selection when the training camera configuration varies across demonstrations. p. 6
Where the paper says it

Also, feed-forward novel view synthesis introduces a latency of approximately 30 ms per frame, which may pose challenges in extremely dynamic scenarios, and requires additional GPU memory during inference. One open challenge lies in target viewpoint selection when the training camera configuration varies across demonstrations.

The one-time LVSM fine-tuning for domain adaptation is significantly more efficient than VLA fine-tuning because LVSM is substantially smaller and requires less computational resources. p. 5
Where the paper says it

While our primary approach is zero-shot adaptation with frozen policies, we note that the one-time LVSM fine-tuning for domain adaptation is significantly more efficient than VLA fine-tuning. First, LVSM is substantially smaller (171M parameters) compared to VLAs (7B for OpenVLA-OFT, 3.

This makes our approach highly practical: a single LVSM model fine-tuned once on multi-view data can serve as a universal view adapter for any Training Views Test Views. p. 5
Where the paper says it

More importantly, LVSM fine-tuning only requires multi-view images without any robot action labels, making data collection orders of magnitude easier and cheaper than collecting expert demonstrations for VLA training. This makes our approach highly practical: a single LVSM model fine-tuned once on multi-view data can serve as a universal view adapter for any Training Views Test Views Our camera setting Objects used in experiments Fig.

The method requires no additional robot demonstrations or training, eliminating data collection and computational training costs. p. 4
Where the paper says it

In contrast, our zero-shot adaptation framework circumvents both issues entirely. By keeping the policy frozen and adapting only the visual input, our method requires no additional robot demonstrations or training, eliminating data collection and computational training costs.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Zero-Shot Camera Adaptation Framework: The system can perform robust task execution across diverse setups, including different camera extrinsics, intrinsics, and freely moving handheld cameras such as an iPhone, ZED, and RealSense without additional demonstration data or policy fine-tuning.

  2. Real-time View Synthesis Integration: The framework integrates a feed-forward novel view synthesis model to virtually adjust test-time camera observations to match the training camera configuration in real-time, handling both extrinsic and intrinsic parameters at approximately 30 FPS while allowing the frozen VLA policy to run at 10 Hz.

  3. Enhanced Viewpoint Robustness: The method consistently outperforms baselines on benchmarks like LIBERO by enabling robust generalization, even under large viewpoint shifts (up to 15 cm translation and 60° rotation in camera extrinsics), showing the best task success rate over baselines.

  4. Plug-and-Play Deployment: The approach operates in a plug-and-play manner, making it applicable to any RGBbased policy without architectural modifications, preserving the pre-trained capabilities of VLAs and avoiding the catastrophic forgetting associated with fine-tuning.

Abstract

Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/

Sources

Related papers