Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback".
Dev: Here's the gist of the paper. Wi-Fi sensing enables contactless human sensing, while most existing systems rely on Channel State Information, which is accessible only on a limited range of Wi-Fi chipsets <ref:2611.16185#pg5> In contrast, Beamforming Feedback Information (BFI) is standardized in modern Wi-Fi and readily accessible on a wide range of commodity devices, offering new opportunities for human sensing <ref:2611.16185#pg3> This paper presents BeamLens, a Wi-Fi sensing system that extracts fine-grained spatial information from BFI to enable versatile human sensing <ref:2611.16185#pg4> Extensive evaluations demonstrate that BeamLens achieves 90.16% accuracy in human activity recognition, a mean per-joint position error of 8.64 cm in 3D human pose estimation, and 70.64% intersection over union in human silhouette estimation <ref:2611.16185#pg4> System Architecture and Information Extraction BeamLens is designed as a Wi-Fi sensing system that exploits fine-grained spatial information embedded in BFI to enable versatile human sensing, including 3D human pose estimation, human silhouette estimation, and human activity recognition <ref:2611.16185#pg3> The system operates through four modules: two-view BFI probing, fine-grained spatial information extraction, spatial resolution enhancement, and two-view spatial information fusion <ref:2611.16185#pg4> In the two-view BFI probing module, BeamLens deploys two AP-STA links to provide an azimuth view and an elevation view of the human body <ref:2611.16185#pg5> The fine-grained spatial information extraction module jointly estimates the angle of departure (AoD) and relative time of flight (RToF) from BFI by leveraging spatial diversity across multiple transmitting antennas and frequency diversity across multiple subcarriers <ref:2611.16185#pg5> Spatial Resolution Enhancement The spatial resolution of BFI-based Wi-Fi sensing is enhanced by combining a sparse physical antenna array with a uniform virtual antenna array <ref:2611.16185#pg4> The sparse physical antenna array is constructed by rearranging the transmitting antennas based on the minimum redundancy principle to enlarge the antenna array aperture without requiring additional hardware <ref:2611.16185#pg5> Furthermore, a uniform virtual antenna array is constructed from the pairwise separations of its antennas to increase the effective degrees of freedom and enable separation of more multipath components <ref:2611.16185#pg5> Self-Supervised Representation Learning To fuse complementary spatial information captured from two views and temporal information obtained through continuous BFI measurements, BeamLens employs a self-supervised learning framework that jointly exploits cross-view correspondence and temporal consistency <ref:2611.16185#pg4> This framework utilizes two objectives: temporally aware cross-view contrastive learning to align the azimuth and elevation views, and temporal masked reconstruction to capture the continuity of human movements <ref:2611.16185#pg4> The self-supervised learning objective is defined as LSSL = Lcv + β Lrec, where Lcv aligns the two views at each moment and captures temporal consistency <ref:2611.16185#pg4> Performance and Generalization Extensive experiments demonstrate that BeamLens achieves 90.16% accuracy in human activity recognition, a mean per-joint position error of 8.64 cm in 3D human pose estimation, and 70.64% intersection over union in human silhouette estimation <ref:2611.16185#pg4> The system exhibits strong generalization to unseen environments and subjects <ref:2611.16185#pg4> This is because BeamLens learns motion representations from unlabeled BFI during pretraining, which capture how the human body moves rather than the body shape or motion habits of a specific person <ref:2611.16185#pg5> The performance degradation under Non-Line-of-Sight (NLoS) conditions is small, indicating that BeamLens remains effective even when the direct path between the AP and the STA is blocked <ref:2611.16185#pg5> Ablation Study Contributions The ablation study confirms that all components contribute to versatile human sensing, as removing any component degrades performance <ref:2611.16185#pg5> Using only a single view leads to the largest degradation in accuracy, because most of the activities mainly involve horizontal movements captured by the azimuth view, whereas the elevation view mainly captures vertical movements <ref:2611.16185#pg4> The sparse physical antenna array and uniform virtual antenna array contribute to improved angular resolution of the heatmaps <ref:2611.16185#pg5> The two pretraining objectives are complementary, as the contrastive loss aligns the two views at each moment while the reconstruction loss captures how the motion evolves over time <ref:2611.16185#pg4> The results demonstrate that BeamLens substantially reduces the labeling effort required for versatile human sensing <ref:2611.16185#pg5> BeamLens presents a BFI-based Wi-Fi sensing system that extracts fine-grained spatial information to enable versatile human sensing <ref:2611.16185#pg4> BeamLens jointly estimates AoD and RToF from BFI to extract fine-grained spatial information and leverages two AP-STA links to capture complementary azimuth and elevation views <ref:2611.16185#pg4> To enhance spatial resolution, we design a sparse physical antenna array and construct a uniform virtual antenna array <ref:2611.16185#pg5> Furthermore, we develop a self-supervised representation learning framework that integrates complementary spatial information across views and captures temporal dependencies to learn a general-purpose backbone for diverse human sensing tasks <ref:2611.16185#pg4> Extensive experiments demonstrated the effectiveness and versatility of BeamLens, achieving 90.16% accuracy in human activity recognition, an MPJPE of 8.64 cm in 3D human pose estimation, and 70.64% IoU in human silhouette estimation <ref:2611.16185#pg4> BeamLens also demonstrated strong generalization, highlighting its potential for practical and versatile Wi-Fi-based human sensing <ref:2611.
Paper summary: Rosa: So we're looking at this paper, "Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback." Essentially, they're tackling a problem where most current Wi-Fi sensing just uses Channel State Information, which is only available on a few specific chipsets.
Dev: Yeah, but this paper points to Beamforming Feedback Information or BFI. That’s standardized in modern Wi-Fi and you can actually grab it with regular stuff like laptops without needing special firmware changes <ref:2611.16185#pg2>.
Rosa: Exactly. The thesis here is that BFI opens up a lot of opportunities because it's widely accessible, and this paper introduces BeamLens, a system that extracts very fine-grained spatial information from that BFI to do more than just basic motion detection.
Dev: They claim BeamLens can support three specific human sensing tasks: three dee human pose estimation, human silhouette estimation, and even human activity recognition <ref:2611.16185#pg3>. That’s a lot of capability packed into one system.
Taro: What I find interesting is how they claim to get that fine-grained spatial information by jointly estimating the angle of departure and the relative time of flight from the BFI <ref:2611.16185#pg3>. That sounds like a big computational lift.
Rosa: It is, but they handle that by using both spatial diversity across multiple transmitting antennas and frequency diversity across multiple subcarriers <ref:2611.16185#pg3>. They’re leveraging the physical setup of the Wi-Fi system itself to get better data.
Dev: And they don't stop there; they also have this spatial resolution enhancement method where they combine a sparse physical antenna array with a uniform virtual antenna array <ref:2611.16185#pg3>. That’s their way of improving the angular resolution and separating those tricky multipath components.
Taro: So, when you're driving or walking, you’d expect some signal interference, and they are trying to clean up that physical measurement by using both the real antenna arrangement and a virtual one built from the pairwise separations <ref:2611.16185#pg3>. How does that translate to something useful in the field?
Rosa: It translates into better accuracy for things like three dee human pose estimation, which they report achieves a mean per-joint position error of eight point sixty-four centimeters <ref:2611.16185#pg4>. That's a pretty solid number when you're trying to map out someone’s body in three dimensions.
Dev: And they also have this self-supervised representation learning framework <ref:2611.16185#pg3>. They use two objectives here: temporally aware cross-view contrastive learning and temporal masked reconstruction <ref:2611.16185#pg4>.
Paper summary: Taro: What does that self-supervised part actually do? It seems like they're trying to fuse the information from both views—the azimuth and elevation views—and also capture how the movement changes over time <ref:2611.16185#pg3>.
Rosa: Right. They define this as a loss function called LSSL, which is the sum of the cross-view alignment loss, Lcv, and the temporal reconstruction loss, Lrec <ref:2611.16185#pg4>. It’s about aligning the views at each moment while making sure it looks continuous over time.
Dev: So what's the result on those three tasks? They report ninety point sixteen percent accuracy for human activity recognition, an error of eight point sixty-four centimeters for pose estimation, and a seventy point six four percent intersection over union in human silhouette estimation <ref:2611.16185#pg4>.
Taro: Those numbers sound good on paper, but what happens when the world misbehaves? What's the caveat?
Rosa: Well, they showed strong generalization to unseen environments and subjects <ref:2611.16185#pg4>. They pretrain this system using unlabeled BFI data to learn motion representations that focus on how the human body moves, not just what a specific person looks like or how they usually move <ref:2611.16185#pg5>.
Dev: That generalization is important. They also noted that the performance drop when things aren't in perfect line of sight, Non-Line-of-Sight conditions, is small <ref:2611.16185#pg5>. This means it still works even if there’s a wall blocking the direct path between the Wi-Fi access point and your device.
Taro: So, for someone listening right now who's maybe trying to use this while driving, does that mean they can trust it when they have some signal blockage?
Rosa: It suggests that BeamLens is quite robust in those scenarios because it learns general motion patterns rather than just relying on a perfect direct line of sight <ref:2611.16185#pg5>. That's what the authors are claiming about its practical usability.
Dev: Looking at the ablation study, they confirmed every single part contributes to that versatility, but removing any component drops the performance significantly <ref:2611.16185#pg5>. They specifically pointed out that using only a single view causes the biggest drop in accuracy because most activities involve horizontal movements captured by the azimuth view, while vertical movements are mostly seen in the elevation view <ref:2611.16185#pg4>.
Paper summary: Taro: That makes sense. If you only have one angle, you miss half of what's happening with the person's movement <ref:2611.16185#pg4>. They also noted that the sparse physical antenna array and the uniform virtual antenna array both help improve the angular resolution of those heatmaps <ref:2611.16185#pg5>.
Rosa: So, to summarize, they developed BeamLens by jointly estimating angle of departure and relative time of flight from BFI using multiple antennas and frequencies <ref:2611.16185#pg3>. They use that data to build these enhanced spatial representations through self-supervised learning <ref:2611.16185#pg4>.
Dev: The final conclusion is that BeamLens successfully extracts fine-grained spatial information from BFI to enable versatile human sensing <ref:2611.16185#pg4>. They achieved ninety point sixteen percent accuracy in activity recognition, an MPJPE of eight point sixty-four centimeters for pose estimation, and a seventy point six four percent intersection over union for silhouette estimation <ref:2611.16185#pg4>.
Taro: It seems the implication here is that we can start moving away from just detecting if someone is moving at all, toward actually understanding their three dee pose and activities in a way that's more robust to environmental noise <ref:2611.16185#pg1>.
Rosa: That’s the big picture. The authors are showing that by using standardized BFI feedback instead of older Channel State Information, we can build a system that works across different devices and gives us richer spatial data for human sensing <ref:2611.16185#pg2>.
Dev: So, to wrap up this paper on "Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback," the authors developed BeamLens by jointly estimating angle of departure and relative time of flight from BFI to extract fine-grained spatial information and leveraged two AP-STA links to capture complementary azimuth and elevation views <ref:2611.16185#pg4>. They enhanced spatial resolution by combining a sparse physical antenna array with a uniform virtual antenna array <ref:2611.16185#pg3>. And they built this self-supervised representation learning framework that integrates complementary spatial information across different views and captures temporal dependencies between observations to learn a general-purpose backbone for diverse downstream human sensing tasks <ref:2611.16185#pg4>.
Taro: The results show the system achieves ninety point sixteen percent accuracy in human activity recognition, a mean per-joint position error of eight point sixty-four centimeters in three dee human pose estimation, and a seventy point six four percent intersection over union in human silhouette estimation <ref:2611.16185#pg4>.
Paper summary: Rosa: And it's important to remember that BeamLens also demonstrated strong generalization to unseen environments and subjects <ref:2611.16185#pg4>. This is because the system learns motion representations from unlabeled BFI during pretraining, which capture how the human body moves rather than the body shape or motion habits of a specific person <ref:2611.16185#pg5>.
Dev: They also highlighted that performance degradation under Non-Line-of-Sight conditions is small, indicating that BeamLens remains effective even when the direct path between the AP and the STA is blocked <ref:2611.16185#pg5>. That’s a key point for real-world deployment.
Taro: So, what does this mean for us in terms of future work? What's next after this system is built?
Rosa: The ablation study showed that all components matter, so they confirmed that removing any part degrades the performance <ref:2611.16185#pg5>. They also noted that using only a single view leads to the largest degradation in accuracy because most activities mainly involve horizontal movements captured by the azimuth view, whereas the elevation view mainly captures vertical movements <ref:2611.16185#pg4>.
Dev: The combination of those two pretraining objectives is also key; the contrastive loss aligns the two views at each moment while the reconstruction loss captures how the motion evolves over time <ref:2611.16185#pg4>. It's a balanced approach to learning both spatial and temporal aspects of movement.
Taro: It really does show that this system substantially reduces the labeling effort required for versatile human sensing <ref:2611.16185#pg5>. That’s a big deal for how we actually train these sensing models in the future.
Rosa: So, to conclude on this paper, "Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback," it presents a BFI-based Wi-Fi sensing system that extracts fine-grained spatial information to enable versatile human sensing <ref:2611.16185#pg4>. It jointly estimates angle of departure and relative time of flight from BFI to extract fine-grained spatial information and leverages two AP-STA links to capture complementary azimuth and elevation views <ref:2611.16185#pg4>.
Dev: They enhanced spatial resolution by designing a sparse physical antenna array and constructing a uniform virtual antenna array <ref:2611.16185#pg3>. Furthermore, they developed this self-supervised representation learning framework that integrates complementary spatial information across different views and captures temporal dependencies between observations to learn a general-purpose backbone for diverse human sensing tasks <ref:2611.16185#pg4>.
Taro: Extensive experiments demonstrated the effectiveness and versatility of BeamLens, achieving ninety point sixteen percent accuracy in human activity recognition, an MPJPE of eight point sixty-four centimeters in three dee human pose estimation, and a
Conclusion: Rosa: So we're wrapping up this deep dive on BeamLens, which is titled "Enabling Versatile Human Sensing through Wi-Fi Beamforming Feedback."
Dev: Yeah, the authors are focusing on using this Wi-Fi feedback information to do a lot more than just basic sensing.
Rosa: They’ve shown how you can take that BFI data and extract very detailed spatial info to tackle things like three dee human pose estimation and activity recognition.
Dev: It seems they've put together this whole architecture using two AP-STA links to get both an azimuth view and an elevation view of the person.
Rosa: Exactly, and they use some clever techniques, like a sparse physical antenna array combined with a virtual one, to really boost that spatial resolution of the heatmaps.
Dev: And the core of it is this self-supervised learning framework they built to fuse those two views together while also tracking how the movement evolves over time.
Rosa: The results show they hit ninety point one six percent accuracy for activity recognition and a mean per-joint position error of eight point six four centimeters for that three dee pose estimation.
Dev: That's a solid performance figure, but they also make it clear that this system generalizes well to people and environments it hasn't seen before.
Rosa: They trained the model on unlabeled BFI data first, which means it learns how the body moves generally, not just what one specific person looks like or how they usually move.
Dev: And they also said that performance drops aren't huge even when there is a lot of signal interference, non-line-of-sight conditions.
Rosa: That’s what makes it potentially useful for real life scenarios where you can't guarantee a perfect line of sight between the access point and the person.
Dev: So, what does all this mean for us? It suggests we can start moving beyond just knowing if someone is walking to actually understanding their three dee posture and what they are doing in a much more versatile way.
Rosa: And that’s where we're headed next—we need to look at how this kind of BFI sensing might fit into actual wearable devices.
University of South Florida · University of Miami
eess.SP
Submitted: 2026-11-12
Updated: 2026-11-12
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 90/100
The gist: Wi-Fi sensing enables contactless human sensing, while most existing systems rely on Channel State Information…
Key concepts
- Beamforming Feedback Information (BFI)
- BFI is a standardized piece of information in modern Wi-Fi that allows devices to understand the directionality of the signal. Unlike channel state information, BFI is accessible on many devices and provides crucial spatial data needed for sensing human movements.
- Two-View BFI Probing
- This module uses two separate Wi-Fi links (AP-STA) to capture two different perspectives—an azimuth view and an elevation view—of the human body. This dual perspective is essential for building a complete 3D understanding of the subject's position and movement.
- Self-Supervised Representation Learning
- This framework uses learning techniques to fuse spatial data from both views with temporal data over time. It learns how human movements evolve by aligning the two views at each moment and reconstructing continuous motion, creating a general model for sensing.
Terminology
Summary
Wi-Fi sensing enables contactless human sensing, while most existing systems rely on Channel State Information, which is accessible only on a limited range of Wi-Fi chipsets <ref:2611. p. 016185#pg5> In contrast, Beamforming Feedback Information (BFI) is standardized in modern Wi-Fi and readily accessible on a wide range of commodity devices, offering new opportunities for human sensing <ref:2611. p. 016185#pg3> This paper presents BeamLens, a Wi-Fi sensing system that extracts fine-grained spatial information from BFI to enable versatile human sensing <ref:2611. p. 016185#pg4> Extensive evaluations demonstrate that BeamLens achieves 90. p. 016% accuracy in human activity recognition, a mean per-joint position error of 8. p. 0 p. 064 cm in 3D human pose estimation, and 70. p. 0 p. 0 p. 064% intersection over union in human silhouette estimation <ref:2611. p. 0 p. 016185#pg4>
System Architecture and Information Extraction
BeamLens is designed as a Wi-Fi sensing system that exploits fine-grained spatial information embedded in BFI to enable versatile human sensing, including 3D human pose estimation, human silhouette estimation, and human activity recognition <ref:2611.16185#pg3> The system operates through four modules: two-view BFI probing, fine-grained spatial information extraction, spatial resolution enhancement, and two-view spatial information fusion <ref:2611. p. 416185#pg4> In the two-view BFI probing module, BeamLens deploys two AP-STA links to provide an azimuth view and an elevation view of the human body <ref:2611. p. 416185#pg5> The fine-grained spatial information extraction module jointly estimates the angle of departure (AoD) and relative time of flight (RToF) from BFI by leveraging spatial diversity across multiple transmitting antennas and frequency diversity across multiple subcarriers <ref:2611. p. 116185#pg5>
Spatial Resolution Enhancement
The spatial resolution of BFI-based Wi-Fi sensing is enhanced by combining a sparse physical antenna array with a uniform virtual antenna array <ref:2611.16185#pg4> The sparse physical antenna array is constructed by rearranging the transmitting antennas based on the minimum redundancy principle to enlarge the antenna array aperture without requiring additional hardware <ref:2611. p. 116185#pg5> Furthermore, a uniform virtual antenna array is constructed from the pairwise separations of its antennas to increase the effective degrees of freedom and enable separation of more multipath components <ref:2611. p. 116185#pg5>
Self-Supervised Representation Learning
To fuse complementary spatial information captured from two views and temporal information obtained through continuous BFI measurements, BeamLens employs a self-supervised learning framework that jointly exploits cross-view correspondence and temporal consistency <ref:2611.16185#pg4> This framework utilizes two objectives: temporally aware cross-view contrastive learning to align the azimuth and elevation views, and temporal masked reconstruction to capture the continuity of human movements <ref:2611. p. 116185#pg4> The self-supervised learning objective is defined as LSSL = Lcv + β Lrec, where Lcv aligns the two views at each moment and captures temporal consistency <ref:2611. p. 116185#pg4>
Performance and Generalization
Extensive experiments demonstrate that BeamLens achieves 90.16% accuracy in human activity recognition, a mean per-joint position error of 8.64 cm in 3D human pose estimation, and 70.64% intersection over union in human silhouette estimation <ref:2611.16185#pg4> The system exhibits strong generalization to unseen environments and subjects <ref:2611. p. 016185#pg4> This is because BeamLens learns motion representations from unlabeled BFI during pretraining, which capture how the human body moves rather than the body shape or motion habits of a specific person <ref:2611. p. 216185#pg5> The performance degradation under Non-Line-of-Sight (NLoS) conditions is small, indicating that BeamLens remains effective even when the direct path between the AP and the STA is blocked <ref:2611.16185#pg5>
Ablation Study Contributions
The ablation study confirms that all components contribute to versatile human sensing, as removing any component degrades performance <ref:2611.16185#pg5> Using only a single view leads to the largest degradation in accuracy, because most of the activities mainly involve horizontal movements captured by the azimuth view, whereas the elevation view mainly captures vertical movements <ref:2611.16185#pg4> The sparse physical antenna array and uniform virtual antenna array contribute to improved angular resolution of the heatmaps <ref:2611. p. 216185#pg5> The two pretraining objectives are complementary, as the contrastive loss aligns the two views at each moment while the reconstruction loss captures how the motion evolves over time <ref:2611. p. 516185#pg4> The results demonstrate that BeamLens substantially reduces the labeling effort required for versatile human sensing <ref:2611. p. 216185#pg5>
Conclusion
BeamLens presents a BFI-based Wi-Fi sensing system that extracts fine-grained spatial information to enable versatile human sensing <ref:2611.16185#pg4> BeamLens jointly estimates AoD and RToF from BFI to extract fine-grained spatial information and leverages two AP-STA links to capture complementary azimuth and elevation views <ref:2611. p. 016185#pg4> To enhance spatial resolution, we design a sparse physical antenna array and construct a uniform virtual antenna array <ref:2611. p. 016185#pg5> Furthermore, we develop a self-supervised representation learning framework that integrates complementary spatial information across views and captures temporal dependencies to learn a general-purpose backbone for diverse human sensing tasks <ref:2611. p. 216185#pg4> Extensive experiments demonstrated the effectiveness and versatility of BeamLens, achieving 90. p. 216% accuracy in human activity recognition, an MPJPE of 8. p. 164 cm in 3D human pose estimation, and 70.64% IoU in human silhouette estimation <ref:2611. p. 116185#pg4> BeamLens also demonstrated strong generalization, highlighting its potential for practical and versatile Wi-Fi-based human sensing <ref:2611. p. 0
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
BeamLens can perform versatile human sensing tasks including 3D human pose estimation, human silhouette estimation, and human activity recognition by jointly estimating angle of departure (AoD) and relative time of flight (RToF) from Beamforming Feedback Information (BFI).
-
The system enhances spatial resolution by combining a sparse physical antenna array with a uniform virtual antenna array to improve angular resolution and multipath separation capability.
-
The system enables versatile sensing by using a self-supervised learning framework that jointly exploits cross-view correspondence and temporal consistency to learn a general-purpose backbone for diverse sensing tasks.
-
The system can remove environment dependent reflections by applying background subtraction to the heatmaps, retaining only the reflections from the human body, and compensates for transmit chain phase offsets using known LoS path information.
-
The system improves performance under data scarcity by requiring only half of the labeled data to match or exceed the performance of Scratch trained with all labels, demonstrating a substantial reduction in labeling effort needed for versatile human sensing.
Sources
- DensePose From WiFi
- Decoupled Weight Decay Regularization
- Representation Learning with Contrastive Predictive Coding
Related papers
- Open-H-RF: A Large-Scale Ultrasound Raw Channel Dataset Initiative
- Scalable Attention for 5G NR Channel Estimation
- Low-Altitude Wireless Networks: A Comprehensive Survey
- Sensing-Assisted OFDM with Index Modulation: A Waveform for Optical Wireless Integrated Communication and Localisation
- Iteration-Free Candidate Pruning and an Index-Domain VLSI Architecture for Sort-Free MIMO Box Decoding up to 1024-QAM
- A Foundation Model for Large-Scale Wireless Network Planning, Operation and Optimization