WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses
cs.RO, cs.CV
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: 41 pages, 35 figures, 11 tables
Code: https://github.com/isaac-sim/IsaacSim
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart
Abstract
Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi 0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.
Sources
- PaLM-E: An Embodied Multimodal Language Model
- RT-1: Robotics Transformer for Real-World Control at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- OpenVLA: An Open-Source Vision-Language-Action Model
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
- Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
- AerialVLN: Vision-and-Language Navigation for UAVs
- AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions
- Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology
- Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
- CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs
- UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generation
- RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviour
- Open X-Embodiment: Robotic Learning Datasets and RT-X Models
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigation
- Aerial Vision-and-Dialog Navigation
- UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies