Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors

arXiv:2608.19357 · eess.SP · Submitted 2026-08-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "Untangling Dense Crowds with mmWave Radar".

Dev: Here's the gist of the paper. In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations.

Rosa: The Core Problem of Spatio-Temporal Entanglement Natural crowded environments exhibit several characteristics that make recovering individual spatial footprints particularly challenging These challenges include Crowd Size and Local Density, where many individuals traverse the sensing field of view (FOV) simultaneously, often moving in close proximity.

Dev: Observation Ambiguity from Merging and Occlusion arises because radar returns from multiple individuals may thus overlap, or disappear due to blockage.

Rosa: Interaction-Rich Motion means pedestrian motion is inherently interdependent, as individuals continuously adjust their movement in response to other pedestrians and obstacles.

Title and authors: Rosa: So we're looking at this paper called "Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors." It sounds like they're tackling a really tough problem in crowded areas.

Dev: Yeah, it's about recovering individual footprints and how pedestrians move when they can merge or block each other for long periods. Rosa, what’s the basic idea behind this title?

Rosa: Well, it suggests they aren't just trying to track individuals in a crowd; they're trying to understand the spatial behaviors of those individuals by using learned crowd semantics and modeling how radar observations get messed up because of merging and occlusion.

Dev: That sounds like a big leap from just tracking people, doesn't it? It suggests they’re moving beyond simple detection into understanding the actual flow within the crowd.

Taro: From my side, I'm thinking about what that means for real-world failure modes. If you can model those sustained merging and occlusion events, it gives us a much better picture of when and how tracking just breaks down in practice.

Rosa: Exactly. It's not just about getting a better track; it’s about creating a system that understands the underlying crowd dynamics even when things get messy.

Dev: And this approach is interesting because it seems to link what's happening on a large scale, like the overall crowd flow, down to how individual radar signals are actually observed at a very small scale.

The paper's summary: Rosa: So the core of this paper is that they use a macro-to-micro modeling approach. On the macro level, they learn spatial usage patterns of the crowd from those radar point clouds by creating something called a continuous spatial flow field, which describes how the crowd generally moves across an area.

Dev: And then at the micro level, they build a physics-informed model for that radar observability loss caused by merging and occlusion. That model then guides the reasoning about individual states to figure out what each person is doing.

Taro: I like that structure because it addresses the entanglement issue directly by modeling both sides of it—the crowd's movement and the physical way those movements interfere with each other's signals.

Rosa: Right, and then they propose this semantic-guided multi-hypothesis reasoning framework. This is where they explicitly model all the ways an observation could happen: one person making one observation, multiple people making one observation through merging, or a person not being seen at all due to occlusion.

Dev: That sounds like a systematic way to score competing possibilities instead of just picking the most likely guess out of thin air. It's principled reasoning over those many-to-one and one-to-none mappings.

Rosa: Precisely. They use these learned crowd semantics and that loss model together to generate, score, and propagate hypotheses until they can untangle the individual footprints from the noisy point clouds, even when the observations are entangled for a long time.

The paper's improvements: Dev: So what are the actual improvements they claim? They focus on making it robust during prolonged loss of observability. They use that macro-to-micro structure and the multi-hypothesis reasoning to recover footprints even when things are entangled for extended periods.

Taro: I'm interested in how they handle those persistent loss cases. The paper suggests treating occlusion as a specific hypothesis class where an individual produces no return due to blockage, and they score that using something called phi star i t, which is the worst-case likelihood of that blockage occurring.

Rosa: And they also introduced some new metrics to quantify how entangled the crowd is. They have the Observability Factor and the Merge Factor. These are meant to give a principled way to measure observation ambiguity in dense crowds, something that was missing before.

Dev: The paper claims they robustly recover individual space usage patterns with an average median DTW of thirty-one centimeters across their experiments, which is pretty solid for spatial accuracy <ref:2608.19357#pg1>.

Taro: And the ablation study shows that just having the learned crowd semantics isn't enough; you need all three parts working together—the semantics, the loss model, and the reasoning procedure—to get good results.

Conclusion: Rosa: So to wrap up, this paper on "Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors" shows that combining learned crowd semantics with a physics-informed model of observation loss allows for recovering individual footprints even when the system is dealing with long periods of merging and occlusion.

Dev: And their method uses a principled, semantic-guided multi-hypothesis reasoning foundation to score competing hypotheses over many-to-one and one-to-none mappings, which gives them better tracking accuracy than some state of the art methods.

Taro: What stands out for me is how they treat occlusion not just as a gap in data, but as a hypothesis class that can be scored based on the worst-case likelihood of blockage occurring.

Rosa: And they also provide those novel metrics, like the Observability Factor and the Merge Factor, to give us a principled way to measure how tangled the observations are in dense crowds.

Dev: This work suggests mmWave radar is viable for fine-grained crowd analytics because it can handle these complex, interaction-rich environments without needing person identification.

Taro: It’s a solid foundation for understanding pedestrian behavior, especially in those prolonged scenarios where simple tracking just fails.

Aaditya Prakash Kattekola, Anurag Pallaprolu, Yasamin Mostofi

University of California Santa Barbara

eess.SP

Submitted: 2026-08-19

Updated: 2026-11-03

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 88/100

The gist: In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem…

Key concepts

Spatio-Temporal Entanglement
This is the core problem where radar data from different individuals becomes intertwined across both space and time. It happens because pedestrians frequently merge or block each other for extended periods, making it extremely difficult to separate and identify each person's unique movement path from the resulting point cloud observations.
Macro-to-Micro Modeling Paradigm
This approach bridges high-level crowd understanding with detailed individual tracking. At the macro level, the system learns general spatial usage patterns of the entire crowd using a flow field. At the micro level, it uses a physics model to predict how specific radar observations are lost due to merging or occlusion, guiding individual state recovery.
Principled Semantic-Guided Multi-Hypothesis Reasoning
This is the method used to resolve observation ambiguity. It systematically generates and scores competing scenarios—such as one person being observed, multiple people merging into one signal, or one person being completely blocked—to find the most consistent explanation for the radar data.
Observability Factor and Merge Factor
These are novel metrics proposed to quantify how entangled the crowd observations are. The Observability Factor measures how many detections remain clear of merging or occlusion relative to the total crowd size, while the Merge Factor tracks how often observations from different individuals appear close together in subsequent time steps.

Terminology

Summary

In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. p. 0

Where the paper says it

Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors Aaditya Prakash Kattekola, Anurag Pallaprolu, and Yasamin Mostofi Abstract—In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations.

p. 0
Where the paper says it

Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors Aaditya Prakash Kattekola, Anurag Pallaprolu, and Yasamin Mostofi Abstract—In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations.

The Core Problem of Spatio-Temporal Entanglement

Natural crowded environments exhibit several characteristics that make recovering individual spatial footprints particularly challenging These challenges include Crowd Size and Local Density, where many individuals traverse the sensing field of view (FOV) simultaneously, often moving in close proximity. Observation Ambiguity from Merging and Occlusion arises because radar returns from multiple individuals may thus overlap, or disappear due to blockage. p. 1

Where the paper says it

Observation Ambiguity from Merging and Occlusion: As individuals move in close proximity, observations from different pedestrians frequently merge or become occluded. Radar returns from multiple individuals may thus overlap, or disappear due to blockage.

Interaction-Rich Motion means pedestrian motion is inherently interdependent, as individuals continuously adjust their movement in response to other pedestrians and obstacles. p. 1
Where the paper says it

As a result, a single observation may correspond to multiple individuals, while at other times an individual may produce no observable return. Interaction-Rich Motion: Pedestrian motion is inherently interdependent, as individuals continuously adjust their movement in response to other pedestrians and obstacles.

These properties give rise to what we term spatio-temporal entanglement in this paper. p. 1
Where the paper says it

Natural Space Usage: Pedestrians exhibit natural patterns of space usage, rather than following scripted routes. These properties give rise to what we term spatio-temporal entanglement in this paper.

Spatio-Temporal Entanglement is defined as the phenomenon in which radar point cloud observations associated with different individuals become intertwined across space and time, sometimes for prolonged durations, making it difficult to disentangle and recover each individual’s footprint from mmWave point clouds. p. 1
Where the paper says it

These properties give rise to what we term spatio-temporal entanglement in this paper. Spatio-Temporal Entanglement: We define this as the phenomenon in which radar point cloud observations associated with different individuals become intertwined across space and time, sometimes for prolonged durations, making it difficult to disentangle and recover each individual’s footprint from mmWave point clouds.

Prolonged Observability Loss occurs because merging and occlusion among individuals often persist over extended time intervals rather than occurring sporadically. p. 1
Where the paper says it

Spatio-Temporal Entanglement: We define this as the phenomenon in which radar point cloud observations associated with different individuals become intertwined across space and time, sometimes for prolonged durations, making it difficult to disentangle and recover each individual’s footprint from mmWave point clouds. Prolonged Observability Loss: In realistic scenarios, merging and occlusion among individuals often persist over extended time intervals rather than occurring sporadically, giving rise to prolonged spatio-temporal entanglement and extended periods of observability loss.

The Macro-to-Micro Modeling Paradigm

The proposed framework adopts a macro-to-micro modeling paradigm that bridges crowd semantics and radar observability. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds. p. 0

Where the paper says it

Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion.

Specifically, we capture such patterns through a continuous spatial flow field vˆ(x): R squared → R squared, which encodes the dominant motion tendencies at each location x in the FOV. p. 4
Where the paper says it

Although individual radar detections are sparse and intermittent, aggregate observations reveal persistent patterns of how space is collectively used by the crowd. Building on prior work in radar-based flow field estimation [36], [37], we capture such patterns through a continuous spatial flow field vˆ(x) : R 2 → R 2 , which encodes the dominant motion tendencies at each location x in the FOV.

This learned macroscopic semantics serves as our baseline for predicting how individuals move through the space. p. 4
Where the paper says it

, a few seconds). The learned macroscopic semantics vˆ(x) serve as our baseline for predicting how individuals move through the space, as we discuss next.

At the microscopic level, we develop a physics-informed model of radar observability loss caused by merging and occlusion. p. 0
Where the paper says it

Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion.

This model guides downstream reasoning over individual states. p. 2
Where the paper says it

At the macroscopic level, it learns spatial usage patterns of the crowd from radar point clouds, while at the microscopic level it develops a physics-informed model of radar observability loss caused by merging and occlusion, guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework.

Principled Semantic-Guided Multi-Hypothesis Reasoning

Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework. This method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between predicted states and observations. p. 2

Where the paper says it

c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework. The method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between predicted states and observations.

The framework systematically generates, scores, and propagates competing hypotheses to resolve observation ambiguity. p. 1
Where the paper says it

High-level overview of the proposed methodology for untangling individual footprints in dense crowds. p. 1

Where the paper says it

High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).

The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).

We consider three canonical classes of correspondence hypotheses: (i) a baseline hypothesis in which each observation corresponds to a single individual, (ii) merge hypotheses in which multiple individuals jointly generate a single observation, and (iii) occlusion hypotheses in which an individual produces no observable return due to blockage. p. 5
Where the paper says it

The resulting reasoning process forms a dynamic hypothesis tree that maintains multiple competing explanations of individual motion until sufficient evidence emerges to disambiguate them, with ambiguities typically resolving within a short temporal horizon as new observations arrive. More specifically, we consider three canonical classes of correspondence hypotheses within this framework: (i) a baseline hypothesis in which each observation corresponds to a single individual, (ii) merge hypotheses in which multiple individuals jointly generate a single observation, and (iii) occlusion hypotheses in which an individual produces no observable return due to blockage.

Quantifying Entanglement and Performance Metrics

We propose novel metrics to quantify the degree of spatiotemporal crowd entanglement. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. p. 3

Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

p. 3
Where the paper says it

Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.

e., not occluded or merged) to the total crowd size, averaged over time. p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

p. 3
Where the paper says it

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. , not occluded or merged) to the total crowd size, averaged over time.

The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time. p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

p. 3
Where the paper says it

merging and occlusion where multiple individuals may yield a single merged observation, or an occluded individual may produce none at all. The Merge Factor, on the other hand, is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

Our framework robustly recovers individual space usage patterns, with an average median DTW of 31 cm across all experiments. p. 2
Where the paper says it

Our experiments involve crowds of up to (and including) 10 individuals spanning six categories of pedestrian space usage behaviors, and 4 different environments including both indoor and outdoors. Our framework robustly recovers individual space usage patterns, with an average median DTW of 31 cm across all experiments, demonstrating significant improvement over the state-of-the-art.

Ablation and State-of-the-Art Comparison

Our framework is built upon three key components: learned crowd semantics, a principled model of radar observation loss caused by occlusion and merging, and a multi-hypothesis reasoning procedure. Disabling all modules reverts the system to a classical state-space estimator, leading to incomplete recovery of footprints during sustained loss of observability. p. 2

Where the paper says it

Our framework robustly recovers individual space usage patterns, with an average median DTW of 31 cm across all experiments, demonstrating significant improvement over the state-of-the-art. The recovered footprints exhibit strong structural agreement with camera-derived ground truth, validating the framework’s effectiveness in untangling individual behaviors in challenging, interaction-rich crowds, that can experience several instances of prolonged observability loss.

Enabling only macroscopic semantics captures the general shape of the crowd footprints but remains inaccurate due to entire motion paths being missed. The proposed framework delivers substantial performance gains over Multi-Hypothesis Tracker (MHT) across all experimental scenarios. p. 0
Where the paper says it

Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments.

Specifically, MHT achieves an average median DTW error of 95. p. 9
Where the paper says it

These results demonstrate that systematic reasoning that integrates principled modeling of radar observability with learned crowd semantics can effectively resolve severe observation ambiguities and reliably untangle individual footprints of a crowd. Similarly, our framework achieves an average median DTW of 31 cm, with all experiments falling well within the biacromial shoulder width of 0.

94 cm, compared with only 30. p. 0
Where the paper says it

The proposed foundation substantially outperforms the state-of-the-art across all experiments, achieving an average median DTW of 30. 5 cm compared to 95.

5 cm for the proposed framework. p. 1
Where the paper says it

High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).

Conclusion and Future Directions

This paper addresses the challenge of recovering individual spatial footprints from spatio-temporally entangled mmWave radar point clouds. Overall, the results demonstrate the viability of mmWave radar as a sensing modality for fine-grained crowd analytics and individual behavior understanding in dense, interaction-rich environments. p. 2

Where the paper says it

RELATED WORK RF sensing has emerged as a promising modality for crowd analytics, as it does not depend on lighting conditions, alleviates privacy concerns, and avoids infrastructural overhead associated with alternative modalities discussed in Sec. Prior RF-based approaches to crowd sensing span a hierarchy of inference granularity, from coarse statistics to fine-grained individual footprint recovery.

Future work could explore incorporating multiple radar viewpoints to enhance robustness in scenarios involving very prolonged loss of observability. p. 1
Where the paper says it

The second category, on the other hand, aims to recover individual footprints, but existing systems are mainly limited to small number of occupants (five or less) and spatially-sparse scenarios where pedestrian interactions are minimal and their paths remain largely well separated, typically following simple and prescribed routes [41]–[45]. More importantly, the existing work does not consider challenging cases where sustained merging or occlusion results in prolonged observability loss.

We note that our framework recovers individual spatial footprints without relying on person identification. p. 0
Where the paper says it

Over the past decade, WiFi signals have been widely leveraged for sensing tasks beyond communication [21], including imaging [22]–[24], person identification [25]–[27], and activity recognition [28]–[30]. While WiFi-based solutions have been explored for crowd analytics, the inherently low spatial resolution of commodity WiFi hardware confines such approaches to occupancy estimation [31], or motion inference of simple scenarios [32], insufficient for the individual-level footprint recovery that dense crowd settings demand.

--- Page 1 ---

Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors

Abstract—In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. p. 0

Where the paper says it

This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints.

At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion. p. 0
Where the paper says it

Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion.

Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-toone and oneto-none observation mappings, enabling untangling of individual spatial footprints. p. 0
Where the paper says it

At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion. Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints.

We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments. p. 0
Where the paper says it

Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments.

Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the groundtruth. p. 0
Where the paper says it

We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the groundtruth.

The recovered footprints further enable accurate zonal analysis of pedestrian space usage. p. 0
Where the paper says it

Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the groundtruth. The recovered footprints further enable accurate zonal analysis of pedestrian space usage.

Finally, we present an ablation study and further compare against state-of-the-art. p. 0
Where the paper says it

The recovered footprints further enable accurate zonal analysis of pedestrian space usage. Finally, we present an ablation study and further compare against state-of-the-art.

--- Page 2 ---

Fig. 1. High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C). p. 1

Where the paper says it

High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).

--- Page 3 ---

We next summarize our key contributions. a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally entangled mmWave radar point clouds in dense crowds, even when such entanglements persist over long periods of time. p. 1

Where the paper says it

We next summarize our key contributions. Statement of Contributions: a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally entangled mmWave radar point clouds in dense crowds, even when such entanglements persist over long periods of time.

By jointly leveraging learned crowd semantics, observability loss modeling, and principled reasoning over radar observability loss, our framework systematically untangles overlapping radar observations to recover individual spatial footprints and behaviors. p. 1
Where the paper says it

Statement of Contributions: a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally entangled mmWave radar point clouds in dense crowds, even when such entanglements persist over long periods of time. By jointly leveraging learned crowd semantics, observability loss modeling, and principled reasoning over radar observability loss, our framework systematically untangles overlapping radar observations to recover individual spatial footprints and behaviors.

b) The proposed framework adopts a macro-to-micro modeling paradigm that bridges crowd semantics and radar observability modeling. p. 1
Where the paper says it

By jointly leveraging learned crowd semantics, observability loss modeling, and principled reasoning over radar observability loss, our framework systematically untangles overlapping radar observations to recover individual spatial footprints and behaviors. b) The proposed framework adopts a macro-to-micro modeling paradigm that bridges crowd semantics and radar observability

At the macroscopic level, it learns spatial usage patterns of the crowd from radar point clouds, while at the microscopic level it develops a physics-informed model of radar observability loss caused by merging and occlusion. p. 2
Where the paper says it

At the macroscopic level, it learns spatial usage patterns of the crowd from radar point clouds, while at the microscopic level it develops a physics-informed model of radar observability loss caused by merging and occlusion, guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework.

c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework. p. 2
Where the paper says it

At the macroscopic level, it learns spatial usage patterns of the crowd from radar point clouds, while at the microscopic level it develops a physics-informed model of radar observability loss caused by merging and occlusion, guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework.

The method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between predicted states and observations. p. 2
Where the paper says it

c) Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework. The method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between predicted states and observations.

--- Page 4 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. p. 3

Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

p. 3
Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

p. 3
Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

p. 3
Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

p. 3
Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

p. 3
Where the paper says it

The IF signals from all TX-RX pairs are reorganized into a 2D virtual planar array, and we denote the discretized IF signal at virtual array element (p, q) by s (p,q) IF [m, n], where m, n index chirps and ADC samples respectively. Point cloud extraction proceeds in two stages.

First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

p. 3
Where the paper says it

Point cloud extraction proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles.

Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

p. 3
Where the paper says it

This step effectively suppresses static clutter and avoids CFAR-style thresholding [70] over entire range–angle spectra, yielding a set of candidate sensing depths over time. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges.

Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

p. 3
Where the paper says it

The resulting cluster centroids then provide radar observations that serve as inputs for our proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived directly from radar observations to quantify the extent of spatio-temporal entanglement across our experiments.

The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

--- Page 5 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

--- Page 6 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

--- Page 7 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

--- Page 8 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

--- Page 9 ---

We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved system can recover individual footprints even during prolonged observability loss by jointly leveraging learned crowd semantics and a physics-informed model of radar observability loss caused by sustained merging and occlusion.

  2. Improved system can perform zonal analysis of pedestrian space usage by using the recovered footprints to characterize how individuals dwell in different parts of the space over time, as demonstrated in Experiment 9.

  3. Improved system can achieve better tracking accuracy than state-of-the-art methods by employing a multi-hypothesis reasoning procedure that systematically evaluates competing hypotheses via principled reasoning over many-to-one and one-to-none observation mappings.

  4. Improved system can quantify the degree of spatio temporal crowd entanglement by proposing novel metrics, specifically the Observability Factor and the Merge Factor, which provide a principled measure of observation ambiguity in dense crowds.

  5. Improved system can maintain robustness during severe observability loss by treating occlusion as a hypothesis class where an individual produces no observable return due to blockage, scored using a probability based on the worst-case occlusion likelihood phi star i t.

  6. Improved system can resolve ambiguity in entangled observations by integrating learned semantic motion dynamics with radar evidence through a dynamic hypothesis tree that propagates competing explanations forward in time until sufficient evidence emerges.

  7. Improved system can utilize a macro-to-micro modeling paradigm where at the macroscopic level, it learns spatial usage patterns from radar point clouds via a continuous spatial flow field v hat x, and at the microscopic level, it reconstructs individual footprints guided by this learned semantics.

Abstract

In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion. Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and one-to-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the ground-truth. The recovered footprints further enable accurate zonal analysis of pedestrian space usage. Finally, we present an ablation study and further compare against state-of-the-art. The proposed foundation substantially outperforms the state-of-the-art across all experiments, achieving an average median DTW of 0.31 m compared to 0.96 m, highlighting the framework's robustness to frequent and prolonged merging and occlusion.

Sources

Related papers