Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Untangling Dense Crowds with mmWave Radar".
Dev: Here's the gist of the paper. In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations.
Rosa: The Core Problem of Spatio-Temporal Entanglement Natural crowded environments exhibit several characteristics that make recovering individual spatial footprints particularly challenging These challenges include Crowd Size and Local Density, where many individuals traverse the sensing field of view (FOV) simultaneously, often moving in close proximity.
Dev: Observation Ambiguity from Merging and Occlusion arises because radar returns from multiple individuals may thus overlap, or disappear due to blockage.
Rosa: Interaction-Rich Motion means pedestrian motion is inherently interdependent, as individuals continuously adjust their movement in response to other pedestrians and obstacles.
Title and authors: Rosa: So we're looking at this paper called "Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors." It sounds like they're tackling a really tough problem in crowded areas.
Dev: Yeah, it's about recovering individual footprints and how pedestrians move when they can merge or block each other for long periods. Rosa, what’s the basic idea behind this title?
Rosa: Well, it suggests they aren't just trying to track individuals in a crowd; they're trying to understand the spatial behaviors of those individuals by using learned crowd semantics and modeling how radar observations get messed up because of merging and occlusion.
Dev: That sounds like a big leap from just tracking people, doesn't it? It suggests they’re moving beyond simple detection into understanding the actual flow within the crowd.
Taro: From my side, I'm thinking about what that means for real-world failure modes. If you can model those sustained merging and occlusion events, it gives us a much better picture of when and how tracking just breaks down in practice.
Rosa: Exactly. It's not just about getting a better track; it’s about creating a system that understands the underlying crowd dynamics even when things get messy.
Dev: And this approach is interesting because it seems to link what's happening on a large scale, like the overall crowd flow, down to how individual radar signals are actually observed at a very small scale.
The paper's summary: Rosa: So the core of this paper is that they use a macro-to-micro modeling approach. On the macro level, they learn spatial usage patterns of the crowd from those radar point clouds by creating something called a continuous spatial flow field, which describes how the crowd generally moves across an area.
Dev: And then at the micro level, they build a physics-informed model for that radar observability loss caused by merging and occlusion. That model then guides the reasoning about individual states to figure out what each person is doing.
Taro: I like that structure because it addresses the entanglement issue directly by modeling both sides of it—the crowd's movement and the physical way those movements interfere with each other's signals.
Rosa: Right, and then they propose this semantic-guided multi-hypothesis reasoning framework. This is where they explicitly model all the ways an observation could happen: one person making one observation, multiple people making one observation through merging, or a person not being seen at all due to occlusion.
Dev: That sounds like a systematic way to score competing possibilities instead of just picking the most likely guess out of thin air. It's principled reasoning over those many-to-one and one-to-none mappings.
Rosa: Precisely. They use these learned crowd semantics and that loss model together to generate, score, and propagate hypotheses until they can untangle the individual footprints from the noisy point clouds, even when the observations are entangled for a long time.
The paper's improvements: Dev: So what are the actual improvements they claim? They focus on making it robust during prolonged loss of observability. They use that macro-to-micro structure and the multi-hypothesis reasoning to recover footprints even when things are entangled for extended periods.
Taro: I'm interested in how they handle those persistent loss cases. The paper suggests treating occlusion as a specific hypothesis class where an individual produces no return due to blockage, and they score that using something called phi star i t, which is the worst-case likelihood of that blockage occurring.
Rosa: And they also introduced some new metrics to quantify how entangled the crowd is. They have the Observability Factor and the Merge Factor. These are meant to give a principled way to measure observation ambiguity in dense crowds, something that was missing before.
Dev: The paper claims they robustly recover individual space usage patterns with an average median DTW of thirty-one centimeters across their experiments, which is pretty solid for spatial accuracy <ref:2608.19357#pg1>.
Taro: And the ablation study shows that just having the learned crowd semantics isn't enough; you need all three parts working together—the semantics, the loss model, and the reasoning procedure—to get good results.
Conclusion: Rosa: So to wrap up, this paper on "Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors" shows that combining learned crowd semantics with a physics-informed model of observation loss allows for recovering individual footprints even when the system is dealing with long periods of merging and occlusion.
Dev: And their method uses a principled, semantic-guided multi-hypothesis reasoning foundation to score competing hypotheses over many-to-one and one-to-none mappings, which gives them better tracking accuracy than some state of the art methods.
Taro: What stands out for me is how they treat occlusion not just as a gap in data, but as a hypothesis class that can be scored based on the worst-case likelihood of blockage occurring.
Rosa: And they also provide those novel metrics, like the Observability Factor and the Merge Factor, to give us a principled way to measure how tangled the observations are in dense crowds.
Dev: This work suggests mmWave radar is viable for fine-grained crowd analytics because it can handle these complex, interaction-rich environments without needing person identification.
Taro: It’s a solid foundation for understanding pedestrian behavior, especially in those prolonged scenarios where simple tracking just fails.
Aaditya Prakash Kattekola, Anurag Pallaprolu, Yasamin Mostofi
University of California Santa Barbara
eess.SP
Submitted: 2026-08-19
Updated: 2026-11-03
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 88/100
The gist: In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem…
Key concepts
- Spatio-Temporal Entanglement
- This is the core problem where radar data from different individuals becomes intertwined across both space and time. It happens because pedestrians frequently merge or block each other for extended periods, making it extremely difficult to separate and identify each person's unique movement path from the resulting point cloud observations.
- Macro-to-Micro Modeling Paradigm
- This approach bridges high-level crowd understanding with detailed individual tracking. At the macro level, the system learns general spatial usage patterns of the entire crowd using a flow field. At the micro level, it uses a physics model to predict how specific radar observations are lost due to merging or occlusion, guiding individual state recovery.
- Principled Semantic-Guided Multi-Hypothesis Reasoning
- This is the method used to resolve observation ambiguity. It systematically generates and scores competing scenarios—such as one person being observed, multiple people merging into one signal, or one person being completely blocked—to find the most consistent explanation for the radar data.
- Observability Factor and Merge Factor
- These are novel metrics proposed to quantify how entangled the crowd observations are. The Observability Factor measures how many detections remain clear of merging or occlusion relative to the total crowd size, while the Merge Factor tracks how often observations from different individuals appear close together in subsequent time steps.
Terminology
Summary
In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. p. 0 Untangling Dense Crowds with mmWave Radar:
From Crowd Semantics to Individual Spatial
Behaviors
Aaditya Prakash Kattekola, Anurag Pallaprolu, and Yasamin Mostofi
Abstract—In this paper, we present a novel framework for
recovering individual footprints and pedestrian spatial behaviors
in dense crowds. This is a highly challenging problem, as
pedestrians can merge or block one another often and for
prolonged periods in crowded areas, leading to strongly entangled
observations. Untangling Dense Crowds with mmWave Radar:
From Crowd Semantics to Individual Spatial
Behaviors
Aaditya Prakash Kattekola, Anurag Pallaprolu, and Yasamin Mostofi
Abstract—In this paper, we present a novel framework for
recovering individual footprints and pedestrian spatial behaviors
in dense crowds. This is a highly challenging problem, as
pedestrians can merge or block one another often and for
prolonged periods in crowded areas, leading to strongly entangled
observations.Where the paper says it
Where the paper says it
The Core Problem of Spatio-Temporal Entanglement
Natural crowded environments exhibit several characteristics that make recovering individual spatial footprints particularly challenging These challenges include Crowd Size and Local Density, where many individuals traverse the sensing field of view (FOV) simultaneously, often moving in close proximity. Observation Ambiguity from Merging and Occlusion arises because radar returns from multiple individuals may thus overlap, or disappear due to blockage. p. 1 Observation Ambiguity from Merging and Occlusion:
As individuals move in close proximity, observations from
different pedestrians frequently merge or become occluded. Radar returns from multiple individuals may thus overlap, or
disappear due to blockage. As a result, a single observation
may correspond to multiple individuals, while at other times
an individual may produce no observable return. Interaction-Rich Motion: Pedestrian motion is inherently
interdependent, as individuals continuously adjust their movement in response to other pedestrians and obstacles. Natural Space Usage: Pedestrians exhibit natural patterns
of space usage, rather than following scripted routes. These properties give rise to what we term spatio-temporal
entanglement in this paper. These properties give rise to what we term spatio-temporal
entanglement in this paper. Spatio-Temporal Entanglement: We define this as the phenomenon in which radar point cloud observations associated
with different individuals become intertwined across space and
time, sometimes for prolonged durations, making it difficult
to disentangle and recover each individual’s footprint from
mmWave point clouds. Spatio-Temporal Entanglement: We define this as the phenomenon in which radar point cloud observations associated
with different individuals become intertwined across space and
time, sometimes for prolonged durations, making it difficult
to disentangle and recover each individual’s footprint from
mmWave point clouds. Prolonged Observability Loss: In realistic scenarios, merging and occlusion among individuals often persist over extended time intervals rather than occurring sporadically, giving
rise to prolonged spatio-temporal entanglement and extended
periods of observability loss.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The Macro-to-Micro Modeling Paradigm
The proposed framework adopts a macro-to-micro modeling paradigm that bridges crowd semantics and radar observability. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds. p. 0 Our approach jointly leverages learned crowd
semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of
the crowd from radar point clouds, while at the microscopic
level we develop a physics-informed model of radar observability
loss caused by sustained merging and occlusion. Although individual radar detections are sparse and
intermittent, aggregate observations reveal persistent patterns
of how space is collectively used by the crowd. Building on
prior work in radar-based flow field estimation [36], [37],
we capture such patterns through a continuous spatial flow
field vˆ(x) : R
2 → R
2
, which encodes the dominant motion
tendencies at each location x in the FOV. , a
few seconds). The learned macroscopic semantics vˆ(x) serve as our
baseline for predicting how individuals move through the
space, as we discuss next. Our approach jointly leverages learned crowd
semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of
the crowd from radar point clouds, while at the microscopic
level we develop a physics-informed model of radar observability
loss caused by sustained merging and occlusion. At the macroscopic level, it learns spatial usage
patterns of the crowd from radar point clouds, while at the
microscopic level it develops a physics-informed model of
radar observability loss caused by merging and occlusion,
guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we
propose a principled semantic-guided multi-hypothesis reasoning framework.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Principled Semantic-Guided Multi-Hypothesis Reasoning
Building on this macro-to-micro modeling structure, we propose a principled semantic-guided multi-hypothesis reasoning framework. This method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between predicted states and observations. p. 2 c) Building on this macro-to-micro modeling structure, we
propose a principled semantic-guided multi-hypothesis reasoning framework. The method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between
predicted states and observations. High-level overview of the proposed methodology for untangling individual footprints in dense crowds. p. 1 High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial
semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates
competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C). The resulting reasoning process
forms a dynamic hypothesis tree that maintains multiple
competing explanations of individual motion until sufficient
evidence emerges to disambiguate them, with ambiguities
typically resolving within a short temporal horizon as new
observations arrive. More specifically, we consider three canonical classes of
correspondence hypotheses within this framework: (i) a baseline hypothesis in which each observation corresponds to a
single individual, (ii) merge hypotheses in which multiple
individuals jointly generate a single observation, and (iii)
occlusion hypotheses in which an individual produces no
observable return due to blockage.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Quantifying Entanglement and Performance Metrics
We propose novel metrics to quantify the degree of spatiotemporal crowd entanglement. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i. p. 3 Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. The
Observability Factor is defined as the ratio of the number of
observations detected at a given time step (i. , not occluded
or merged) to the total crowd size, averaged over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. merging and occlusion where multiple individuals may yield
a single merged observation, or an occluded individual may
produce none at all. The Merge Factor, on the other hand,
is defined as the number of observations at time step t that
fall within a human radius [72] of at least one observation at
time step t + 1, normalized by the crowd size and averaged
over time. Our experiments involve crowds of up to (and including) 10 individuals spanning
six categories of pedestrian space usage behaviors, and 4
different environments including both indoor and outdoors. Our framework robustly recovers individual space usage
patterns, with an average median DTW of 31 cm across
all experiments, demonstrating significant improvement over
the state-of-the-art.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Ablation and State-of-the-Art Comparison
Our framework is built upon three key components: learned crowd semantics, a principled model of radar observation loss caused by occlusion and merging, and a multi-hypothesis reasoning procedure. Disabling all modules reverts the system to a classical state-space estimator, leading to incomplete recovery of footprints during sustained loss of observability. p. 2 Our framework robustly recovers individual space usage
patterns, with an average median DTW of 31 cm across
all experiments, demonstrating significant improvement over
the state-of-the-art. The recovered footprints exhibit strong
structural agreement with camera-derived ground truth, validating the framework’s effectiveness in untangling individual
behaviors in challenging, interaction-rich crowds, that can experience several instances of prolonged observability loss. Building on
this macro-micro structure, we introduce a semantic-guided
multi-hypothesis reasoning foundation that evaluates competing
hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18
real-world experiments using an off-the-shelf mmWave radar,
involving crowds of up to (and including) 10 individuals across
six categories of diverse pedestrian behaviors, and four environments. These results demonstrate that systematic reasoning
that integrates principled modeling of radar observability with
learned crowd semantics can effectively resolve severe observation ambiguities and reliably untangle individual footprints
of a crowd. Similarly, our framework achieves an average median DTW
of 31 cm, with all experiments falling well within the biacromial shoulder width of 0. The proposed
foundation substantially outperforms the state-of-the-art across
all experiments, achieving an average median DTW of 30. 5 cm
compared to 95. High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial
semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates
competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Future Directions
This paper addresses the challenge of recovering individual spatial footprints from spatio-temporally entangled mmWave radar point clouds. Overall, the results demonstrate the viability of mmWave radar as a sensing modality for fine-grained crowd analytics and individual behavior understanding in dense, interaction-rich environments. p. 2 RELATED WORK
RF sensing has emerged as a promising modality for crowd
analytics, as it does not depend on lighting conditions, alleviates privacy concerns, and avoids infrastructural overhead
associated with alternative modalities discussed in Sec. Prior
RF-based approaches to crowd sensing span a hierarchy of
inference granularity, from coarse statistics to fine-grained
individual footprint recovery. The second category, on the other hand,
aims to recover individual footprints, but existing systems are
mainly limited to small number of occupants (five or less)
and spatially-sparse scenarios where pedestrian interactions
are minimal and their paths remain largely well separated,
typically following simple and prescribed routes [41]–[45]. More importantly, the existing work does not consider challenging cases where sustained merging or occlusion results in
prolonged observability loss. Over the past decade, WiFi signals have been widely
leveraged for sensing tasks beyond communication [21], including imaging [22]–[24], person identification [25]–[27],
and activity recognition [28]–[30]. While WiFi-based solutions
have been explored for crowd analytics, the inherently low
spatial resolution of commodity WiFi hardware confines such
approaches to occupancy estimation [31], or motion inference
of simple scenarios [32], insufficient for the individual-level
footprint recovery that dense crowd settings demand.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 1 ---
Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors
Abstract—In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. p. 0 This is a highly challenging problem, as
pedestrians can merge or block one another often and for
prolonged periods in crowded areas, leading to strongly entangled
observations. Our approach jointly leverages learned crowd
semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. Our approach jointly leverages learned crowd
semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of
the crowd from radar point clouds, while at the microscopic
level we develop a physics-informed model of radar observability
loss caused by sustained merging and occlusion. At the macroscopic level, we learn spatial usage patterns of
the crowd from radar point clouds, while at the microscopic
level we develop a physics-informed model of radar observability
loss caused by sustained merging and occlusion. Building on
this macro-micro structure, we introduce a semantic-guided
multi-hypothesis reasoning foundation that evaluates competing
hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints. Building on
this macro-micro structure, we introduce a semantic-guided
multi-hypothesis reasoning foundation that evaluates competing
hypotheses via principled reasoning over many-to-one and oneto-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18
real-world experiments using an off-the-shelf mmWave radar,
involving crowds of up to (and including) 10 individuals across
six categories of diverse pedestrian behaviors, and four environments. We evaluate the framework through 18
real-world experiments using an off-the-shelf mmWave radar,
involving crowds of up to (and including) 10 individuals across
six categories of diverse pedestrian behaviors, and four environments. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss,
demonstrating strong structural consistency with the groundtruth. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss,
demonstrating strong structural consistency with the groundtruth. The recovered footprints further enable accurate zonal
analysis of pedestrian space usage. The recovered footprints further enable accurate zonal
analysis of pedestrian space usage. Finally, we present an ablation
study and further compare against state-of-the-art.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 2 ---
Fig. 1. High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C). p. 1 High-level overview of the proposed methodology for untangling individual footprints in dense crowds. The framework first learns collective spatial
semantics (IV-A) and develops a principled model of radar observability loss due to merging and occlusion (IV-B), then systematically generates and evaluates
competing hypotheses to resolve observation ambiguity and recover individual spatial footprints and behaviors (IV-C).Where the paper says it
--- Page 3 ---
We next summarize our key contributions. a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally entangled mmWave radar point clouds in dense crowds, even when such entanglements persist over long periods of time. p. 1 We next summarize our key contributions. Statement of Contributions:
a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally
entangled mmWave radar point clouds in dense crowds, even
when such entanglements persist over long periods of time. Statement of Contributions:
a) We introduce a novel framework that enables the recovery of individual spatial behaviors from spatio-temporally
entangled mmWave radar point clouds in dense crowds, even
when such entanglements persist over long periods of time. By jointly leveraging learned crowd semantics, observability
loss modeling, and principled reasoning over radar observability loss, our framework systematically untangles overlapping
radar observations to recover individual spatial footprints and
behaviors. By jointly leveraging learned crowd semantics, observability
loss modeling, and principled reasoning over radar observability loss, our framework systematically untangles overlapping
radar observations to recover individual spatial footprints and
behaviors. b) The proposed framework adopts a macro-to-micro modeling
paradigm that bridges crowd semantics and radar observability At the macroscopic level, it learns spatial usage
patterns of the crowd from radar point clouds, while at the
microscopic level it develops a physics-informed model of
radar observability loss caused by merging and occlusion,
guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we
propose a principled semantic-guided multi-hypothesis reasoning framework. At the macroscopic level, it learns spatial usage
patterns of the crowd from radar point clouds, while at the
microscopic level it develops a physics-informed model of
radar observability loss caused by merging and occlusion,
guiding downstream reasoning over individual states. c) Building on this macro-to-micro modeling structure, we
propose a principled semantic-guided multi-hypothesis reasoning framework. c) Building on this macro-to-micro modeling structure, we
propose a principled semantic-guided multi-hypothesis reasoning framework. The method explicitly models many-toone (merging) and one-to-none (occlusion) mappings between
predicted states and observations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 4 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. p. 3 The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. The IF signals from all TX-RX
pairs are reorganized into a 2D virtual planar array, and we
denote the discretized IF signal at virtual array element (p, q)
by s
(p,q)
IF [m, n], where m, n index chirps and ADC samples
respectively. Point cloud extraction proceeds in two stages. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. Point cloud extraction proceeds in two stages. First, an FFT
is applied on the IF signals along the ADC dimension n to
obtain per-chirp range profiles. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. This step effectively suppresses static clutter and avoids CFAR-style thresholding [70]
over entire range–angle spectra, yielding a set of candidate
sensing depths over time. Second, azimuthal angle-of-arrival
(AOA) estimation is performed only at the detected ranges. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments. The resulting cluster centroids
then provide radar observations that serve as inputs for our
proposed framework. Statistical Characterization of Spatio-Temporal Entanglement: We next propose two statistical measures derived
directly from radar observations to quantify the extent of
spatio-temporal entanglement across our experiments.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 5 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.
--- Page 6 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.
--- Page 7 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.
--- Page 8 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.
--- Page 9 ---
We next describe the front-end pipeline that generates point clouds, denoted by P(t), from raw radar data, and present metrics to characterize spatio-temporal entanglement in real point clouds, empirically motivating our proposed framework for untangling dense crowd observations in Sec. IV. Point Cloud Generation proceeds in two stages. First, an FFT is applied on the IF signals along the ADC dimension n to obtain per-chirp range profiles. Second, azimuthal angle-of-arrival (AOA) estimation is performed only at the detected ranges. Statistical Characterization of Spatio-Temporal Entanglement involves two statistical measures derived directly from radar observations to quantify the extent of spatiotemporal entanglement across our experiments. The Observability Factor is defined as the ratio of the number of observations detected at a given time step (i.e., not occluded or merged) to the total crowd size, averaged over time. The Merge Factor is defined as the number of observations at time step t that fall within a human radius [72] of at least one observation at time step t + 1, normalized by the crowd size and averaged over time.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved system can recover individual footprints even during prolonged observability loss by jointly leveraging learned crowd semantics and a physics-informed model of radar observability loss caused by sustained merging and occlusion.
-
Improved system can perform zonal analysis of pedestrian space usage by using the recovered footprints to characterize how individuals dwell in different parts of the space over time, as demonstrated in Experiment 9.
-
Improved system can achieve better tracking accuracy than state-of-the-art methods by employing a multi-hypothesis reasoning procedure that systematically evaluates competing hypotheses via principled reasoning over many-to-one and one-to-none observation mappings.
-
Improved system can quantify the degree of spatio temporal crowd entanglement by proposing novel metrics, specifically the Observability Factor and the Merge Factor, which provide a principled measure of observation ambiguity in dense crowds.
-
Improved system can maintain robustness during severe observability loss by treating occlusion as a hypothesis class where an individual produces no observable return due to blockage, scored using a probability based on the worst-case occlusion likelihood phi star i t.
-
Improved system can resolve ambiguity in entangled observations by integrating learned semantic motion dynamics with radar evidence through a dynamic hypothesis tree that propagates competing explanations forward in time until sufficient evidence emerges.
-
Improved system can utilize a macro-to-micro modeling paradigm where at the macroscopic level, it learns spatial usage patterns from radar point clouds via a continuous spatial flow field v hat x, and at the microscopic level, it reconstructs individual footprints guided by this learned semantics.
Abstract
In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion. Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and one-to-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the ground-truth. The recovered footprints further enable accurate zonal analysis of pedestrian space usage. Finally, we present an ablation study and further compare against state-of-the-art. The proposed foundation substantially outperforms the state-of-the-art across all experiments, achieving an average median DTW of 0.31 m compared to 0.96 m, highlighting the framework's robustness to frequent and prolonged merging and occlusion.
Sources
Related papers
- Open-H-RF: A Large-Scale Ultrasound Raw Channel Dataset Initiative
- Scalable Attention for 5G NR Channel Estimation
- Low-Altitude Wireless Networks: A Comprehensive Survey
- Sensing-Assisted OFDM with Index Modulation: A Waveform for Optical Wireless Integrated Communication and Localisation
- Iteration-Free Candidate Pruning and an Index-Domain VLSI Architecture for Sort-Free MIMO Box Decoding up to 1024-QAM
- A Foundation Model for Large-Scale Wireless Network Planning, Operation and Optimization