FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception
eess.SP, cs.CV, cs.DC, cs.LG, cs.RO
Submitted: 2026-09-24
Updated: 2026-09-24
Comments: Paper is currently under review. Authors' version posted for personal use and not for redistribution
License: http://creativecommons.org/licenses/by/4.0/
The gist: Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constraints.
Terminology
Abstract
Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constraints. However, corruption of transmitted intermediate features creates a deployment mismatch for clean-trained split interfaces, while stronger channel-aware codecs can impose substantial onboard cost. We present FreshLatent, a lightweight channel-aware latent adapter that trains a power-normalized encoder-decoder through wireless corruption while keeping the surrounding VLM frozen. We formulate deployment around a mission-conditioned perception requirement and embedded interface cost, linking channel quality and communication budget to the operating conditions under which perception remains usable. At 0 dB and the tightest communication budget, FreshLatent improves gIoU and cIoU over clean split compression by 20.79 and 20.87 points, respectively. At the most adverse evaluated SNR (0 dB), across all three communication budgets, FreshLatent recovers 63.5-69.1% of the gIoU improvement achieved by a much heavier, range-trained feature-JSCC codec. On an NVIDIA Jetson AGX Xavier in 10-W mode, FreshLatent uses 37-40x fewer encoder parameters, 7.7-9.9x lower edge-interface latency, and 8.8-10.0x lower edge-interface energy than the heavier codec. Together, these results show that lightweight channel-aware adaptation can recover a substantial fraction of the robustness of a much larger communication interface while broadening quality-valid operation under constrained wireless conditions.
Sources
- AVERY: Intent-Driven Adaptive VLM Split Computing via Embodied Self-Awareness for Efficient Disaster Response Systems
- A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
- Progressive Semantic Communication for Efficient Edge-Cloud Vision-Language Models
- LISA: Reasoning Segmentation via Large Language Model
- Decoupled Weight Decay Regularization
- MemoGuard: An Adaptive Runtime for Guarding Against Memory Traps in Communication-Limited Robot Navigation
Related papers
- What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals
- Runtime Assurance Under Measurement Attack: Necessary and Sufficient Observability Conditions for Learned Control in Radio Access Networks
- Physics-Constrained Deep Learning Model for Contactless Blood Pressure Monitoring from Triaxial Bodyseismography
- SpellerSSL: Self-Supervised Learning with P300 Aggregation for Speller BCIs
- Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Review
- Continuous Orthogonal Mode Decomposition: Haptic Signal Prediction in Tactile Internet