Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning".
Dev: Here's the gist of the paper. Geometric Entropy (HG) is introduced as a task-agnostic metric quantifying intrinsic diversity of transit trajectories after normalizing away extrinsic variation <ref:2606.208711,Geometric Entropy (HG) is introduced as a task-agnostic metric quantifying intrinsic diversity of transit trajectories after normalizing away extrinsic variation <ref:2606.20871#pg4>.
Rosa: This metric reveals an inverted-U relationship between success and HG, suggesting that trajectory diversity improves robustness in lowdiversity regimes but degrades performance once diversity induces strategy ambiguity <ref:2606.20871#pg5>.
Dev: Metric Definition and Construction Geometric Entropy (HG) is a lightweight, task-agnostic scalar designed to meet requirements by focusing on free-space transit segments and expressing each segment in a canonical, target-anchored representation <ref:2606.20871#pg6>.
Rosa: The alignment pipeline involves four steps to remove extrinsic variation while preserving intrinsic geometric strategy.
Title and authors: Rosa: So, diving into this paper's summary again, they’re really focusing on how trajectory-shape diversity in demonstrations affects imitation learning performance across different models, tasks, and data scales.
Dev: The paper sets up Geometric Entropy as the central tool to quantify that intrinsic diversity of transit trajectories after we normalize away extrinsic variation.
Taro: This normalization step is crucial because it strips out context like the goal pose or workspace scale while preserving the geometric signal that matters for strategy diversity.
Rosa: They describe this process through four specific steps: arc-length resampling, endpoint anchoring, target frame alignment, and displacement scaling to generate a canonical descriptor called xi.
Dev: Then Geometric Entropy is defined by taking the natural logarithm of the difference between that structural complexity and the natural logarithm of a term that captures the variation magnitude in those aligned trajectory descriptors.
Taro: They also break down a typical manipulation episode into two motion regimes: transit motion, which is free-space movement toward a target, and fine manipulation, which involves constrained contact-rich actions like grasping or insertion.
Rosa: The authors focus specifically on the transit segments because they argue that the geometry of that approach angle and curvature is the primary carrier of demonstrator strategy diversity.
Dev: They identify phase boundaries using task-specific semantic events, like gripper closure marking the transition from approach to grasp, which helps generalize this concept to any multiphase task.
Taro: It’s important for us that they focus on transit because it isolates the learnable geometric variation from the mechanics of physical contact itself.
The paper's summary: Rosa: Moving on to what they propose, the paper suggests using Geometric Entropy as a way to diagnose demonstration datasets before we even start training a policy.
Dev: So, if we compute HG on a pilot dataset and find it’s very low, that’s a signal that the data might be brittle or lack sufficient coverage for the task.
Taro: Or conversely, if the entropy is very high, it suggests we might have mixed in too many incompatible strategies that could cause interference during learning.
Rosa: They suggest an adaptive calibration principle: we should match the demonstration diversity to what we actually expect the learner to master.
Dev: This means for a low mastery regime, like when training from scratch, increasing the geometric diversity can be beneficial because it expands effective coverage and pushes the peak of performance to higher entropy values.
Taro: But once you get into a high mastery regime, like with large datasets or strong priors from pretraining, adding more geometric variation doesn't help; it just becomes interference.
Rosa: This is what drives the shift toward lower optimal entropy as proficiency increases—we want to match the diversity to the learner’s current capability.
Dev: They also mention that for pretrained vision language action models, this effect turns into a monotonic decrease, suggesting strong priors are very effective at filtering out unnecessary geometric noise.
Taro: It’s about using HG as a fast pre-training diagnostic so we can actually guide the data collection process rather than just blindly collecting more stuff.
The paper's improvements: Rosa: Wrapping up this discussion on "Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning," the main conclusion is that optimal geometric entropy, H*, shifts toward lower values as task mastery increases.
Dev: They found that for pretrained vision language action models, the trend becomes effectively monotonic decreasing, meaning more geometric diversity hurts performance after pretraining.
Taro: The key implication for us in real-world applications is to stop maximizing diversity by default and start matching it to the learner's mastery level.
Rosa: We use Geometric Entropy as a simple diagnostic: low entropy points to brittleness, high entropy suggests strategy interference. It’s an actionable signal for data curation.
Dev: So, we look at the task, assess our mastery level with proxies like success rate on low-diversity sets, and then adjust our demonstration diversity around that optimal point using HG.
Taro: If we do this correctly, we can build systems that are robust against both insufficient coverage and strategy ambiguity when they run outside the lab.
Rosa: And if we do it right, we can actually make sense of why some datasets perform better than others by looking at their intrinsic geometric structure through Geometric Entropy.
Conclusion: Rosa: So we're wrapping up on Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning. The main idea here is that the optimal geometric entropy, H*, actually goes down as the learner gets better at a task.
Dev: Right, it's about matching the diversity of the demonstrations to how well the AI already knows how to do something without needing more chaotic input.
Taro: I think it’s a lot about avoiding that middle ground where you have too much noise and not enough structure.
Rosa: Exactly. They show that increasing geometric variation helps when the learning is still very difficult, but once the learner has mastered the basics, more variety just creates confusion for the strategy.
Dev: And for pretrained models, it’s even simpler; they find that adding more geometric diversity actually hurts performance because those strong priors you already learned are being confused by extra shape variations.
Taro: It really highlights that pretraining has a way of setting a very specific geometric baseline, and if you push past that baseline with too much noise, it just becomes interference rather than coverage.
Rosa: So the practical thing is to use Geometric Entropy as a diagnostic before training: low entropy means your data might be brittle, and high entropy means you've mixed in incompatible strategies.
Dev: The paper’s focus on normalizing away extrinsic variation while keeping the intrinsic geometry is really clever because it isolates what actually matters for the strategy.
Taro: It gives us a way to calibrate our demonstrations before we spend all our time collecting more data just to chase a higher entropy number.
Rosa: So, Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning suggests we need to be smarter about how much diversity we feed the machine based on where it is in its learning journey.
Dev: Yeah, it’s a useful tool for anyone working on real-time loops or imitation learning efficiency.
Taro: Next up, we’re looking at how those models actually execute those actions in the real world, which brings us to the paper on Real-Time Execution with Autoregressive Policies.
Qian Luo, Ruizhe Liu, Pei Zhou, Xunzhe Zhou, Yanchao Yang
The University of Hong Kong
cs.RO
Submitted: 2026-06-18
Updated: 2026-11-07
Comments: Accepted to IROS 2026. Project page: https://geometric-entropy.github.io/
Project page: https://geometric-entropy.github.io
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 82/100
The gist: Geometric Entropy (HG) is introduced as a task-agnostic metric quantifying intrinsic diversity of transit trajectories after normalizing away extrinsic variation <ref:2606.208711,Geometric Entropy…
Key concepts
- Geometric Entropy (HG)
- A lightweight metric that measures how complex and diverse a robot's movement path is, focusing on the intrinsic shape of the trajectory after normalizing for external variations. It captures both the number of distinct movement modes and their spread in space.
- Inverted-U Relationship
- This describes an experimental finding where performance initially improves as geometric diversity increases, but then performance drops once diversity becomes too high. This suggests that having a moderate amount of variation is beneficial for learning but excessive variation causes the robot to get confused by conflicting movement strategies.
- Mastery-Entropy Principle
- This principle states that the ideal level of trajectory diversity (H*G) depends on how well the learner knows the task. Beginners benefit from more diversity to cover more ground, while experts perform best with less variation because extra changes interfere with their established, efficient movement patterns.
Terminology
Summary
Geometric Entropy (HG) is introduced as a task-agnostic metric quantifying intrinsic diversity of transit trajectories after normalizing away extrinsic variation <ref:2606. p. 0208711,Geometric Entropy (HG) is introduced as a task-agnostic metric quantifying intrinsic diversity of transit trajectories after normalizing away extrinsic variation <ref:2606. p. 020871#pg4>. This metric reveals an inverted-U relationship between success and HG, suggesting that trajectory diversity improves robustness in lowdiversity regimes but degrades performance once diversity induces strategy ambiguity <ref:2606. p. 020871#pg5>.
Metric Definition and Construction
Geometric Entropy (HG) is a lightweight, task-agnostic scalar designed to meet requirements by focusing on free-space transit segments and expressing each segment in a canonical, target-anchored representation <ref:2606.20871#pg6>. The alignment pipeline involves four steps to remove extrinsic variation while preserving intrinsic geometric strategy. p. 2 These steps include arc-length resampling, endpoint anchoring, target-frame alignment, and displacement scaling to create the canonical descriptor ξ. p. 2 HG summarizes the intrinsic complexity of the resulting shape manifold with a stable spectral statistic of the aligned trajectory descriptors. p. 0 Specifically, it is defined as HG = ln deff + lnXjλj where deff captures structural complexity and the second term captures variation magnitude. p. 2
Experimental Findings Across Models and Tasks
Across multiple IL architectures and both simulation and real-robot contact-rich manipulation tasks, a consistent Inverted-U relationship between success and HG is observed <ref:2606.20871#pg5>. This pattern indicates that increasing geometric diversity improves robustness in lowdiversity regimes but degrades performance once diversity induces strategy ambiguity <ref:2606. p. 020871#pg5>. Furthermore, the optimal entropy shifts toward lower values as task mastery increases via more data, easier tasks, or stronger priors <ref:2606. p. 020871#pg5>. For a pretrained visionlanguage-action model the trend becomes effectively monotonic decreasing <ref:2606. p. 020871#pg5>.
The Mastery–Entropy Principle
The optimal geometric entropy H∗G is governed by the learner’s task mastery, which is approximated by proxies such as success rate on low-diversity sets. Low mastery regimes, such as StackCube at small N or a model trained from scratch, benefit from moderate increases in HG because additional diversity expands effective coverage and the peak lies at higher HG. p. 5 Conversely, high mastery regimes, like PegInsertion at large N or a pre-trained VLA, show monotonic decline because additional geometric variation acts as interference rather than useful coverage. p. 5
Metric Robustness and Comparison
HG is superior to common diversity proxies like mean variance and logDet Covariance because it captures both the number of independent shape modes (effective dimensionality) and their overall spread in aligned shape space. Baseline metrics fail by either ignoring mode organization or aliasing multimodality, whereas HG is less prone to collapsing distinct strategy organizations while converging rapidly. p. 6 The spectral decomposition approach used for HG is computationally cheap and sample-efficient, making it a practical audit signal for dataset curation. p. 0
Practical Implications for Curation
Practitioners should match demonstration diversity to the learner’s mastery rather than maximizing diversity by default. HG enables a simple pre-training diagnostic where low HG suggests the dataset may be brittle, while very high HG suggests incompatible strategies are mixed in and may induce mode interference. p. 6 This provides actionable guidance for calibrating demonstrations toward the learnable regime. p. 0
Conclusion
The study concludes that the optimal geometric entropy H∗G shifts toward lower values as task mastery increases, and can become effectively monotonically decreasing for pretrained VLAs <ref:2606.20871#pg5>. Together, these results challenge the “maximize diversity” heuristic and suggest a calibration principle: match demonstration diversity to the learner’s mastery, using HG as a fast diagnostic for data curation. p. 6
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Geometric Entropy as a Pre-training Diagnostic: The AI system can be audited prior to training by computing HG on a pilot dataset to determine if it is
very low
(suggesting brittleness) orvery high
(suggesting mode interference). -
Adaptive Diversity Calibration: The system can dynamically adjust the required geometric diversity based on its current task mastery, shifting the optimal entropy toward lower values as proficiency increases.
-
Robustness Against Mode Ambiguity: By using HG instead of baselines like mean variance or LogDet Covariance, the system avoids aliasing multimodality and can reliably distinguish between
many tight modes
and asingle consistent corridor.
-
Task-Specific Performance Tuning: For tasks like StackCube, the AI system can leverage the Inverted-U relationship to increase diversity when low mastery is detected but decrease it when high mastery is reached, ensuring robustness against both insufficient coverage and strategy ambiguity.
-
Monotonic Decay for Pretrained Models: When fine-tuning a pretrained Vision-Language-Action model like pi0.5, the system can expect a monotonic decay in performance as HG increases, indicating that additional geometric diversity acts as interference with the strong prior learned during pretraining.
Sources
- RT-1: Robotics Transformer for Real-World Control at Scale
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- OpenVLA: An Open-Source Vision-Language-Action Model
- RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
- CUPID: Curating Data your Robot Loves with Influence Functions
- Curating Demonstrations using Online Experience
- ManiBox: Enhancing Embodied Spatial Generalization via Scalable Simulation Data Generations
- Adversarial Data Collection: Human-Collaborative Perturbations for Efficient and Robust Robotic Imitation Learning
- FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generation
- MOVE: A Simple Motion-Based Data Collection Paradigm for Spatial Generalization in Robotic Manipulation
- Is Diversity All You Need for Scalable Robotic Manipulation?
- ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation