A Two-Stage Shape- and Motion-Aware Framework for mmWave-based Human Mesh Recovery
cs.CV
Submitted: 2026-05-08
Updated: 2026-10-27
Comments: accepted by WACV'27
License: http://creativecommons.org/licenses/by/4.0/
The gist: Millimeter-wave (mmWave) radar has emerged as a promising sensing modality for human perception, offering robustness under challenging environmental conditions and strong privacy preservation.
Terminology
Abstract
Millimeter-wave (mmWave) radar has emerged as a promising sensing modality for human perception, offering robustness under challenging environmental conditions and strong privacy preservation. However, recovering accurate 3D human meshes from radar is difficult for two reasons: radar tensors contain significant environmental clutter, and each frame observes only part of the body surface. These challenges demand opposite operations, removing measured energy that does not belong to the body versus inferring geometry the sensor never measured. Yet previous works typically entangle both within a single network, with the mesh objective as the main supervision. We instead reformulate radar-based mesh recovery as two explicitly supervised sub-problems. A signal-level separation stage isolates body-related voxels through coarse-to-fine localization and voxel-wise segmentation, trained with visibility derived voxel labels. A structure-level reconstruction stage then recovers the body from this partial observation using a dual-branch network that jointly models per-frame shape and inter-frame motion. Experiments show that our method reduces mesh error by up to 9.1% over the strongest baseline while using over 10 times fewer parameters. Source code can be found via https://github.com/haipham03/mmTSM-Mesh
Sources
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs