RoboEdit: Turning Human Manipulation Videos into Scalable Robot Experience

arXiv:2608.18948 · cs.RO · Submitted 2026-08-19 · Read on arXiv

cs.RO

Submitted: 2026-08-19

Updated: 2026-09-27

Comments: 22 pages, 14 figures, 7 tables, including supplementary material. Revised version with additional experiments and implementation details

Project page: https://roboedit.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Adaptation — 15× in this paper · explained in 1 episode
• Consistency — 8× in this paper · explained in 3 episodes
• Benchmark — 7× in this paper · explained in 2 episodes
• Regularization — 7× in this paper · explained in 3 episodes
• Alignment — 6× in this paper · explained in 1 episode
• Fidelity — 6× in this paper · explained in 2 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Computer Vision and Pattern Recognition — 3× in this paper
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Affordance — 2× in this paper · explained in 1 episode
• Core — 2× in this paper · explained in 2 episodes
• Data Augmentation — 2× in this paper · explained in 7 episodes
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• In Conference on Robot Learning — 2× in this paper
• Ablation Studies — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Abundance — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Flow Matching — 1× in this paper · explained in 7 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• In IEEE International Conference on Computer Vision — 1× in this paper
• Latent Representation — 1× in this paper · explained in 2 episodes
• Latent Representations — 1× in this paper · explained in 3 episodes
• LoRA Adapters — 1× in this paper · explained in 3 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Objective Function — 1× in this paper · explained in 2 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes

Sources

• Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation
• HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• VINO: A Unified Visual Generator with Interleaved OmniModal Context
• H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videos
• Project Aria: A New Tool for Egocentric Multi-Modal AI Research
• GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities
• DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
• OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing
• LoRA: Low-Rank Adaptation of Large Language Models
• Hand Pose Estimation via Latent 2.5D Heatmap Regression
• AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks
• Kinematic Motion Retargeting for Contact-Rich Anthropomorphic Manipulations
• H2R: A Human-to-Robot Data Augmentation for Robot Pre-training from Videos
• OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation
• Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance
• EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing
• TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding
• LIV: Language-Image Representations and Rewards for Robotic Control
• DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies