LessMimic: Versatile Humanoid-Object Interaction with Unified Distance Field Representations

arXiv:2602.21723 · cs.RO · Submitted 2026-02-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "LessMimic: Versatile Humanoid-Object Interaction with Unified Distance Field Representations".

Dev: Here's the gist of the paper. Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence<ref:2602.21723#pg2> A unified interaction representation enabling reference-free inference, geometric generalization, and long-horizon skill composition within one policy remains an open challenge The paper introduces LESSMIMIC, a reference-free framework that leverages Distance Field (DF) as a unified representation for generalizable, longhorizon humanoid interaction Core Innovation and Representation The core innovation is the use of the Distance Field (DF) to provide a geometryaware state space in which interaction is described through local DF relationships rather than global coordinates This formulation enables the policy to reason about contact and relative motion invariantly to object shape and scale, capturing the essential structure of an interaction rather than memorizing absolute trajectories The DF assigns to every point in space its distance to the nearest object surface, yielding a continuous, differentiable field whose gradient encodes surface normals everywhere—including during contact Furthermore, first-order DF cues—surface gradients and velocity decompositions into normal and tangential components—capture the directional dynamics of ongoing contact This decomposition projects the global velocity into the local coordinate system defined by the object’s surface, capturing approach intensity and surface traversal Training Pipeline Stages LESSMIMIC employs a three-stage training pipeline to produce a wholebody interaction policy requiring only a root trajectory command at inference 1.

Rosa: Behavior Cloning (Interaction Skill Pre-Training): This stage learns a base policy πbase via behavior cloning on data generated by a mimic policy πmimic, which augments motion tracking with a residual module to compensate for dynamic mismatches The objective is to minimize the Mean Squared Error (MSE) loss between the actions of the base and teacher policies using DAgger

42: 2.

Dev: Discriminative Post-Training: This stage refines πbase using Reinforcement Learning in a procedurally augmented environment where object geometries are randomized across scale, shape, and surface properties The policy is optimized using reinforcement learning with a composite reward of the form rt = rtask + λi rinteract + λs rstyle 3.

Rosa: Visual-Motor Policy Distillation: This stage distills πfull into a vision-based policy πvis via a DAgger-style supervision using egocentric depth features to enable deployment without MoCap infrastructure The student minimizes the MSE loss against the teacher’s actions using Ldistill = Es∼pivis pivis ovis − πfull ofull 2> Generalization and Long-Horizon Capability LESSMIMIC demonstrates significant generalization across object scales and shapes, as the DF representation is invariant to object size and approach direction Across four interaction tasks (PickUp, SitStand, Push, Carry) and object scales from 0.4ˆ to 1.6ˆ, LESSMIMIC attains 80–100% success on PickUp and SitStand where baselines degrade sharply For long-horizon execution, the unified DF representation enables implicit skill transitions without explicit sequencing LESSMIMIC achieves 62.1% success on 5-task trajectories and remains viable across sequences of up to 40 heterogeneous task instances sequence—a regime where all ablated variants collapse to zero Key Contributions and Deployment 1.

Title and authors: Rosa: So we’re looking at this paper today called "LessMimic: Versatile Humanoid-Object Interaction with Unified Distance Field Representations." We’ve been talking about how hard it is to get robots to just interact physically in a really flexible way, and this work tackles that by using something called the Distance Field or DF.

Dev: Right, Rosa. It’s interesting because it tries to solve that long-horizon problem where you need the robot to do several different things in a sequence without needing a whole new plan for every single step.

Taro: I’m curious about how they handle the complexity of switching between those different skills, since that's usually where these systems break down when they try to do more than just one thing.

Rosa: Well, what this paper does is introduce this unified representation using the Distance Field to describe interaction. It means instead of memorizing exact trajectories for every move, the robot learns about local relationships—like how close it is to an object's surface—which should make it work better when things change shape or size.

Dev: That sounds promising because if you can reason about contact using these local distance field relationships, you avoid the problem where the policy gets stuck because the object looks slightly different than it did in training.

Taro: But how does that local relationship actually translate into something useful for controlling a whole body? I mean, we're talking about things like grasping or pushing.

Rosa: The paper says that they use first-order cues from this distance field—things like the surface gradients and how the velocity breaks down into normal and tangential components—to capture the actual dynamics of ongoing contact. This lets the robot understand approach intensity and how it’s moving across a surface, not just where it is in space.

Dev: So they are using these geometric cues to build a representation that is invariant to object shape and scale? That's what I need for generalization, especially when we move outside of our controlled lab environment.

Taro: And the results show this works across a lot of different things, including PickUp, SitStand, Push and Carry tasks on objects ranging from zero point four to one point six times the size it was trained on <ref:2602.21723#pg2>? That’s a big range for generalization.

Title and authors: Rosa: Exactly. They report eighty to one hundred percent success rates on PickUp and SitStand across those scales where other methods tend to drop off sharply, which is pretty strong evidence that the representation is robust to those geometric variations.

Dev: Okay, but the method itself has a few stages in its training pipeline. I want to know how they build this system up from scratch so we can assess the stability and latency issues for real-time use.

Rosa: They have three main stages: first, behavior cloning where they learn a base policy using data generated by another mimic policy, and then there’s discriminative post-training using reinforcement learning in an environment where object geometries are randomized across scale and shape.

Taro: And the third stage involves distilling this whole body interaction policy into a vision-based one, which is important for deployment because we don't always have motion capture rigs available.

Dev: That distillation step sounds like a crucial piece for real-world use, as it lets you move from training data to something that relies only on egocentric depth features without needing those expensive MoCap setups.

Rosa: The paper claims this whole process results in a policy that only needs a root trajectory command at inference time, meaning it doesn't need any reference motions or task-specific rewards when it’s actually running.

Taro: That sounds like they achieved the long-horizon composition you were interested in, right? I mean, can we trust that it can actually string together those four different tasks—push, pick up, carry, and sit and stand—sequentially without completely resetting?

Dev: The paper claims it can execute five task trajectories with a sixty-two point one percent success rate on sequences of five tasks and remain viable up to forty sequentially composed task instances. That’s a long horizon for continuous interaction.

Rosa: And the ablation study shows that removing the interaction prior, which is what they call AIP, significantly reduces robustness to scale variation, confirming that this prior is key for keeping geometry consistent across different object shapes.

Taro: So if you take away the geometric prior and just let it learn from raw data, it fails quickly when the object changes size? That makes sense; it's like trying to learn how to hold a cup by only looking at pictures of cups that are all the same height.

Title and authors: Dev: And they also showed that disabling geometry randomization causes severe overfitting, which confirms that forcing the system to generalize across scales is necessary for it to actually perform well outside of the exact training parameters.

Rosa: The real world performance shows the vision-based version hitting eighty point ten percent success on PickUp with a sixty cubic centimeter box and seventy point ten percent on SitStand with a forty six cubic centimeter one, which is solid for those tasks.

Taro: I noticed they said the vision-based policy has slightly reduced tracking accuracy compared to the MoCap based variant, which is expected since it’s relying only on depth observations.

Dev: It makes sense; that perceptual uncertainty from using only depth observations means the robot might need more feedback or correction than when it has full motion reference data.

Rosa: Overall, this paper presents LessMimic as a reference-free framework that uses Distance Fields to create a shape and scale-agnostic representation for humanoid interaction, trained through a three-stage pipeline that allows for long-horizon skill composition with only a root trajectory command at inference.

Dev: It gives us something scalable because it doesn't rely on perfect demonstrations or constant motion reference inputs, which is vital when we’re trying to build systems that can recover from unexpected physical interactions in the real world.

Taro: It seems like the ability for this single policy to execute and transition between different interaction modes over forty task instances, even with varying geometries, is what makes this really significant for practical embodiment.

Rosa: So, to wrap up on LessMimic: they’ve shown that grounding interaction in local geometry through distance fields provides a shape- and scale-invariant signal that enables a single policy to master diverse skills and handle long sequences of tasks.

Dev: It’s a lot of complexity handled by this unified representation, but the engineering achievement here is making sure that the whole thing can run reliably with only an input trajectory command at runtime.

Taro: And for me, the implication is that we might start thinking less about teaching specific motions and more about teaching these fundamental geometric rules of interaction so the system can adapt.

Rosa: Exactly. We’ll keep an eye on how this framework handles unexpected failures when deployed outside of simulation, because that’s where the real test for any embodied intelligence comes down to.

The paper's summary: Rosa: So, to get back to LessMimic, the main idea is that they’ve created this reference-free system using Distance Fields to handle humanoid interaction across different situations and objects without needing pre-recorded motion data for every single move.

Dev: Right. It’s a unified way of looking at things. They use this Distance Field—that continuous field where every point in space tells you how far it is from the nearest object surface—to capture local geometry instead of memorizing exact trajectories from demonstrations.

Taro: So, what does that actually mean for the robot when it’s trying to pick something up versus just pushing a chair? Does this single representation handle those fundamentally different contact patterns?

Rosa: It should. The paper shows that these local distance field relationships are inherently shape and scale-agnostic. This means the policy learns the underlying geometry of interaction behaviors instead of memorizing absolute trajectories for every object it’s ever seen.

Dev: That's a big deal for generalization, because if it can reason about contact based on local distance cues, it shouldn't need to be retrained every time someone introduces a new object shape or size into the environment.

Taro: I saw they tested this across scales from zero point four times the size up to one point six times larger, and they got eighty point ten percent success on PickUp and SitStand in those extreme conditions where other methods usually just break down completely.

Rosa: Exactly, that’s a strong result because those are the exact scenarios where we expect failure when you try to force a system to generalize purely from specific examples.

Dev: And they didn't just stop there; they showed it can handle long sequences of tasks. They found that this single policy could execute five different interaction skills—Push, PickUp, Carry, and SitStand—sequentially without needing a reset after each one.

Taro: Forty task instances in a sequence is a long way to go for continuous control; that suggests the system isn't just good at one isolated move but actually has some kind of skill composition ability built in.

Rosa: That’s the long-horizon capability they call skill composition, and it relies entirely on that unified distance field representation to implicitly handle the transitions between those different modes.

Dev: The training pipeline is also pretty clever; it uses behavior cloning for a stable start, then discriminative reinforcement learning with an adversarial prior to enforce that geometric consistency across randomized objects, and finally visual distillation to get it running on vision data only.

Taro: So the safety case seems to be built by using that AIP—the Adversarial Interaction Prior—to make sure the policy sticks to physically valid interaction patterns regardless of how different the object looks.

Rosa: It really does look like they’ve managed to move away from needing explicit motion references and task-specific rewards at inference time, which is a huge step toward making these robots more practical for unstructured environments.

Dev: It’s definitely a shift in how we think about embodied intelligence, moving toward systems that rely on deep geometric understanding rather than just following pre-programmed paths or hitting specific reward targets.

Taro: It’s less about teaching the robot *how* to do a move and more about giving it the fundamental geometric rules for interacting with physical objects.

Rosa: And that seems like a really promising direction for getting robots to be genuinely useful in the messy, unpredictable world outside of a clean simulation.

The paper's improvements: Taro: So, we’ve seen how they use Distance Fields for generalization and long sequences in the last part of this discussion, but what are some of the specific improvements they suggest to make this system even more robust?

Rosa: Well, one key improvement they highlight is tying that geometric representation directly into an Adversarial Interaction Prior or AIP. They found that this prior is really crucial because it stops the policy from drifting away from physically plausible contact patterns when it encounters a new object geometry.

Dev: That makes sense for a safety case; if the system can’t even learn what a physically valid push looks like, then any long-horizon plan is just guesswork. So this prior enforces that geometric consistency across those randomized scales and shapes they tested.

Taro: I saw in the ablation study that removing that interaction prior really hurt robustness to scale variation, which confirms it’s not just an extra layer, it’s fundamental to how the robot learns interaction rules.

Rosa: Right. And another big improvement is this three-stage training pipeline itself—behavior cloning, then reinforcement learning guided by that AIP, and finally visual distillation—which makes the whole process much more stable and less reliant on perfect initial demonstrations.

Dev: The distillation step is interesting because it lets them transfer the complex whole body policy down into a vision-based one that only needs to see depth images. That’s a huge win for deployment because it removes the need for motion capture equipment entirely.

Taro: It means we can actually put these systems on robots in real, unstructured environments without having to spend weeks gathering perfectly timed motion data for every possible interaction.

Rosa: Exactly. It moves the focus from collecting perfect demonstrations to learning a fundamental geometric understanding that can be adapted instantly through perception alone.

Dev: The implication for our control loops is that we might not need to explicitly model every single contact force or trajectory in advance; instead, the system can rely on these local distance field cues to navigate and recover dynamically.

Taro: It changes the way we look at failure recovery too; because it’s grounded in continuous geometric feedback from those depth observations, the robot can actually recover from unexpected misbehaves more gracefully.

Rosa: So, in short, they’re not just showing a new representation; they're showing how to build a complete training and inference system that learns physical interaction rules robustly and deploys without needing expensive motion capture hardware.

Conclusion: Rosa: So we’ve covered how LessMimic uses Distance Fields to create a shape and scale-agnostic representation for humanoid interaction, trained through that three-stage pipeline with AIP and visual distillation to get it running on vision data only.

Dev: Right. The whole point is moving away from needing explicit demonstrations or motion references at inference time, which really simplifies the control engineering side by removing those latency challenges associated with tracking complex reference trajectories.

Taro: I just want to circle back to what this means for autonomy in the real world; if a system can handle that kind of long-horizon composition and recover from unexpected physical misbehaves, that opens up a whole new class of robot capabilities beyond just following pre-set tasks.

Rosa: It certainly does, because the ability for this single policy to execute five different interaction skills across forty task instances in lessMimic is a big step toward something more general than just executing one simple command.

Dev: The safety case is pretty strong because the geometric grounding means the system isn't guessing its way through contact; it’s using local distance field relationships to maintain stability and predictability during those dynamic skill transitions.

Taro: I think that ability to implicitly transition between interaction modes under a unified representation is where the real autonomy gains happen, moving past brittle, task-specific programming toward something more fluid and continuous.

Rosa: And finally, this work on LessMimic shows that we can build these complex whole body interaction policies using only perception data—specifically egocentric depth features—which makes deployment much more feasible for field robotics.

Dev: It’s a solid piece of engineering because it solves the problem of how to make a single policy robust enough to handle extreme geometric variations and long sequences without needing constant, heavy external reference inputs.

Taro: So it proves that grounding interaction in local geometry, rather than memorizing absolute trajectories, gives us a scalable path toward robots that can genuinely adapt and recover from failures in unstructured environments.

Rosa: That’s the core message of LessMimic: using Distance Fields to enable generalizable, long-horizon humanoid interaction with only a root trajectory command at inference time.

Dev: It’s certainly an interesting paper on how to bridge the gap between learning complex skills and building reliable, real-time control systems.

Taro: We’ll keep an eye on how this framework handles unexpected failures when deployed outside of simulation; that's where we need to see it really shine for true embodied intelligence.

Yutang Lin, Jieming Cui, Yixuan Li, Baoxiong Jia, Yixin Zhu, Siyuan Huang

Institute for AI, Peking University · Beijing Institute for General Artificial Intelligence (BIGAI) · School of Psychological and Cognitive Sciences, Peking University · School of Computer Science and Technology, Beijing Institute of Technology · State Key Lab of General AI · Beijing Key Laboratory of Behavior and Mental Health, Peking University · Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence

cs.RO

Submitted: 2026-02-25

Updated: 2026-11-07

Project page: https://lessmimic.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 98/100

The gist: Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence<ref:2602.21723#pg2> A unified interaction representation…

Key concepts

Distance Field (DF)
A continuous field assigned to every point in space that measures the distance to the nearest object surface. Its gradient encodes surface normals everywhere. This representation is shape- and scale-agnostic, meaning it describes local geometry regardless of the object's absolute size or orientation, providing rich geometric context for interaction.
Reference-Free Inference
The system operates at inference time requiring only a target root trajectory command and the current DF observation. It does not need motion references, task-specific rewards, or separate planners to execute actions. This capability allows the robot to act autonomously based purely on its understanding of the current physical environment geometry.
Long-Horizon Skill Composition
A single policy can execute a complex sequence of different interaction skills—like pushing, picking up, and carrying—sequentially without needing resets or explicit task sequencing. This is achieved because the unified DF representation implicitly captures the necessary transitions between these distinct physical interactions.
Visual-Motor Policy Distillation
This stage transfers a complex policy trained on motion capture data into a simpler vision-based policy that only uses egocentric depth features. This allows the robot to be deployed in real-world scenarios without needing expensive Motion Capture infrastructure, relying instead on its own perception to guide interaction.

Terminology

Summary

Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence<ref:2602. p. 0

Where the paper says it

A single DF-conditioned policy supports (a) online failure recovery through continuous geometric feedback, (b) generalization to unseen object shapes and scales without retraining, and (c) longhorizon composition of heterogeneous interaction skills within a single policy. Abstract—Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence.

21723#pg2> A unified interaction representation enabling reference-free inference, geometric generalization, and long-horizon skill composition within one policy remains an open challenge The paper introduces LESSMIMIC, a reference-free framework that leverages Distance Field (DF) as a unified representation for generalizable, longhorizon humanoid interaction

Core Innovation and Representation

The core innovation is the use of the Distance Field (DF) to provide a geometryaware state space in which interaction is described through local DF relationships rather than global coordinates This formulation enables the policy to reason about contact and relative motion invariantly to object shape and scale, capturing the essential structure of an interaction rather than memorizing absolute trajectories The DF assigns to every point in space its distance to the nearest object surface, yielding a continuous, differentiable field whose gradient encodes surface normals everywhere—including during contact Furthermore, first-order DF cues—surface gradients and velocity decompositions into normal and tangential components—capture the directional dynamics of ongoing contact This decomposition projects the global velocity into the local coordinate system defined by the object’s surface, capturing approach intensity and surface traversal

Training Pipeline Stages

LESSMIMIC employs a three-stage training pipeline to produce a wholebody interaction policy requiring only a root trajectory command at inference

  1. Behavior Cloning (Interaction Skill Pre-Training): This stage learns a base policy πbase via behavior cloning on data generated by a mimic policy πmimic, which augments motion tracking with a residual module to compensate for dynamic mismatches The objective is to minimize the Mean Squared Error (MSE) loss between the actions of the base and teacher policies using DAgger [42]

  2. Discriminative Post-Training: This stage refines πbase using Reinforcement Learning in a procedurally augmented environment where object geometries are randomized across scale, shape, and surface properties The policy is optimized using reinforcement learning with a composite reward of the form rt = rtask + λi rinteract + λs rstyle

  3. Visual-Motor Policy Distillation: This stage distills πfull into a vision-based policy πvis via a DAgger-style supervision using egocentric depth features to enable deployment without MoCap infrastructure The student minimizes the MSE loss against the teacher’s actions using Ldistill = Es∼pivis pivis ovis − πfull ofull 2>

Generalization and Long-Horizon Capability

LESSMIMIC demonstrates significant generalization across object scales and shapes, as the DF representation is invariant to object size and approach direction Across four interaction tasks (PickUp, SitStand, Push, Carry) and object scales from 0.4ˆ to 1.6ˆ, LESSMIMIC attains 80–100% success on PickUp and SitStand where baselines degrade sharply For long-horizon execution, the unified DF representation enables implicit skill transitions without explicit sequencing LESSMIMIC achieves 62. p. 1

Where the paper says it

6ˆ, LESSMIMIC attains 80–100% success on PickUp and SitStand where both reference-based and reference-free baselines degrade sharply. For long-horizon execution, the unified DF representation enables implicit skill transitions without explicit sequencing: LESSMIMIC achieves 62.

1% success on 5-task trajectories and remains viable across sequences of up to 40 heterogeneous task instances sequence—a regime where all ablated variants collapse to zero

Key Contributions and Deployment

The paper's contributions are threefold:

  1. A DF-based interaction representation that encodes local geometric relationships as lightweight, differentiable, and shape-agnostic cues. p. 1

    Where the paper says it

    1% success on 5-task trajectories and remains viable across sequences of up to 40 heterogeneous task instances sequence—a regime where all ablated variants collapse to zero. In summary, our contributions are threefold: ‚ A DF-based interaction representation that encodes local geometric relationships as lightweight, differentiable, and shape-agnostic cues, enabling a single policy to generalize across diverse object geometries without retraining.

  2. A three-stage training pipeline—behavior cloning, AIPguided RL, and visual distillation—that produces a wholebody interaction policy requiring only a root trajectory command at inference. p. 2

    Where the paper says it

    ‚ A three-stage training pipeline—behavior cloning, AIPguided RL, and visual distillation—that produces a wholebody interaction policy requiring only a root trajectory command at inference, with no motion references. ‚ Demonstration that a single LESSMIMIC policy can execute, transition between, and recover from heterogeneous interaction skills over horizons of up to 40 consecutive task instances, extending the practical scope of long-horizon humanoid interaction.

  3. Demonstration that a single LESSMIMIC policy can execute, transition between, and recover from heterogeneous interaction skills over horizons of up to 40 consecutive task instances. p. 2

    Where the paper says it

    ‚ A three-stage training pipeline—behavior cloning, AIPguided RL, and visual distillation—that produces a wholebody interaction policy requiring only a root trajectory command at inference, with no motion references. ‚ Demonstration that a single LESSMIMIC policy can execute, transition between, and recover from heterogeneous interaction skills over horizons of up to 40 consecutive task instances, extending the practical scope of long-horizon humanoid interaction.

Real-World Performance

The vision-based variant of LESSMIMIC achieves 8/10 success on PickUp (22 cm3) and 7/10 on SitStand (46 cm), with slightly reduced tracking accuracy The vision-based policy observes ovis “ roprop, ct, Sts, where St denotes a history of egocentric depth frames This approach allows the robot to recover from failures through continuous geometric feedback

Ablation Study Insights

The ablation study confirms that removing the interaction prior (Ours - AIP) significantly reduces robustness to scale variation, confirming that AIP is the primary mechanism enforcing geometry-consistent interaction across object geometries Removing synthetic physicalization (Ours - Syn), which replaces teacher-generated physically valid trajectories with raw retargeted MoCap data, most severely affects contact-rich tasks such as Carry while leaving others relatively intact Disabling geometry randomization (Ours - Rand) causes severe overfitting with near-zero success outside training scales, confirming its necessity for scale-invariant generalization

Evaluation Metrics

Success Rate is defined by geometric and contact-based criteria tailored to each task, such as the box being lifted above 0.3 m from the ground for PickUp Contact Rate for Push is evaluated as the proportion of time steps during which the humanoid’s end-effectors maintain active contact with the target object Root Tracking Accuracy measures how consistently the humanoid follows the commanded root trajectory over the course of task execution The vision-based policy is evaluated on PickUp (60 cm3) and SitStand (46 cm), with both MoCap-based and vision-based variants under identical conditions using repeated executions The vision-based variant achieves 8/10 success on PickUp (60 cm3) and 7/10 on SitStand (46 cm), with slightly reduced tracking accuracy The vision-based policy is not evaluated on SitStand due to limited egocentric observability of back-side contacts

Conclusion

We present LESSMIMIC, a reference-free framework leveraging DFs as a unified representation for generalizable, longhorizon humanoid interaction The key insight is that local DF geometry—surface distances, gradients, and velocity decompositions—provides a shape- and scale-invariant signal, enabling a single policy to master diverse skills A threestage pipeline of behavior cloning, AIP-guided RL, and visualmotor distillation grounds this into robust, transferable wholebody control In simulation, LESSMIMIC achieves 80–100% success on PickUp and SitStand across extreme object scales (0.4ˆ to 1.6ˆ) Notably, it executes 5-task trajectories with 62. p. 7

Where the paper says it

In simulation, LESSMIMIC achieves 80–100% success on PickUp and SitStand across extreme object scales (0. Notably, it executes 5-task trajectories with 62.

1% success and remains viable up to 40 sequential tasks—a horizon where all baselines collapse Real-world deployment confirms these capabilities transfer reliably across varying object geometries and seat heights The vision-based policy achieves 63. p. 7
Where the paper says it

1% success and remains viable up to 40 sequential tasks—a horizon where all baselines collapse. Real-world deployment confirms these capabilities transfer reliably across varying object geometries and seat heights.

7–99.7% success at scales where all baseline methods exhibit near-zero performance This demonstrates the policy’s ability to implicitly transition between distinct interaction modes under a unified DF-based representation The vision-based policy achieves 8/10 and 7/10 on the two PickUp conditions respectively, with slightly reduced tracking accuracy This moderate performance reduction is attributable to perceptual uncertainty introduced by the depth observations The paper's arXiv ID is 2602.21723 The paper was written by Yutang Lin, Jieming Cui, Yixuan Li, Baoxiong Jia, Yixin Zhu, Siyuan Huang

--- Page 1 ---

LESSMIMIC: Long-Horizon Humanoid Interaction with Unified Distance Field Representations Yutang Lin ˚,1,2,3,5,6,7 Jieming Cui ˚,1,2,3,5,6,7 Yixuan Li 4,2,5 Baoxiong Jia,2,5 Yixin Zhu,3,1,5,6,7 Siyuan Huang ˚Equal contribution Corresponding Author Project Website: https://lessmimic.github.io Institute for AI, Peking University 2Beijing Institute for General Artificial Intelligence (BIGAI)3School of Psychological and Cognitive Sciences, Peking University4School of Computer Science and Technology, Beijing Institute of Technology5State Key Lab of General AI 6Beijing Key Laboratory of Behavior and Mental Health, Peking University7Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence30 cm 60 cm (a) Shape and size generalization. p. 0

Where the paper says it

LESSMIMIC: Long-Horizon Humanoid Interaction with Unified Distance Field Representations Yutang Lin ˚,1,2,3,5,6,7 Jieming Cui ˚,1,2,3,5,6,7 Yixuan Li 4,2,5 Baoxiong Jia ,2,5 Yixin Zhu ,3,1,5,6,7 Siyuan Huang ,2,5 ˚Equal contribution Corresponding Author Project Website: https://lessmimic. io 1 Institute for AI, Peking University 2Beijing Institute for General Artificial Intelligence (BIGAI) 3School of Psychological and Cognitive Sciences, Peking University 4School of Computer Science and Technology, Beijing Institute of Technology 5State Key Lab of General AI 6Beijing Key Laboratory of Behavior and Mental Health, Peking University 7Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence 30 cm 60 cm (a) Shape and size generalization.

The same policy lifts objects ranging from a 23 cm box to a 60 cm-diameter cylinder. p. 0
Where the paper says it

io 1 Institute for AI, Peking University 2Beijing Institute for General Artificial Intelligence (BIGAI) 3School of Psychological and Cognitive Sciences, Peking University 4School of Computer Science and Technology, Beijing Institute of Technology 5State Key Lab of General AI 6Beijing Key Laboratory of Behavior and Mental Health, Peking University 7Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence 30 cm 60 cm (a) Shape and size generalization. The same policy lifts objects ranging from a 23 cm box to a 60 cm-diameter cylinder.

dropped on the ground picked up again failure recovery (b) Failure recovery. p. 0
Where the paper says it

The same policy lifts objects ranging from a 23 cm box to a 60 cm-diameter cylinder. dropped on the ground picked up again failure recovery (b) Failure recovery.

After perturbation, the humanoid re-initiates pickup from the new object location. p. 0
Where the paper says it

dropped on the ground picked up again failure recovery (b) Failure recovery. After perturbation, the humanoid re-initiates pickup from the new object location.

(i) push (ii) pick up (iii) carry (iv) sit and stand (c) Long-horizon skill composition. p. 0
Where the paper says it

After perturbation, the humanoid re-initiates pickup from the new object location. (i) push (ii) pick up (iii) carry (iv) sit and stand (c) Long-horizon skill composition.

A single policy executes push, pick-up, carry, and sit-stand sequentially without resets Abstract—Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence. p. 0
Where the paper says it

A single DF-conditioned policy supports (a) online failure recovery through continuous geometric feedback, (b) generalization to unseen object shapes and scales without retraining, and (c) longhorizon composition of heterogeneous interaction skills within a single policy. Abstract—Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence.

>

--- Page 2 ---

using Adversarial Interaction Priors (AIP) under Reinforcement Learning (RL). Through DAgger-style distillation that aligns DF latents with egocentric depth features, LESSMIMIC further transfers seamlessly to vision-only deployment without motion capture (MoCap) infrastructure A single LESSMIMIC policy achieves 80–100% success across object scales from 0. p. 1

Where the paper says it

Through DAgger-style distillation that aligns DF latents with egocentric depth features, LESSMIMIC further transfers seamlessly to vision-only deployment without motion capture (MoCap) infrastructure. A single LESSMIMIC policy achieves 80–100% success across object scales from 0.

4ˆ to 1.6ˆ on PickUp and SitStand where baselines degrade sharply, attains 62. p. 1
Where the paper says it

A single LESSMIMIC policy achieves 80–100% success across object scales from 0. 6ˆ on PickUp and SitStand where baselines degrade sharply, attains 62.

1% success on 5 task instances trajectories, and remains viable up to 40 sequentially composed tasks By grounding interaction in local geometry rather than demonstrations, LESSMIMIC offers a scalable path toward humanoid robots that generalize, compose skills, and recover from failures in unstructured environments

--- Page 3 ---

I. INTRODUCTION Consider a humanoid tasked with tidying a room: push a chair aside, pick up a box, carry it across the room, and sit down to rest Each sub-task involves distinct contact patterns, object geometries, and body configurations—yet a skilled human executes the entire sequence fluidly, without consulting a motion script The core obstacle is representational We argue that the Distance Field (DF) [35] is precisely this representation A DF assigns to every point in space its distance to the nearest object surface, yielding a continuous, differentiable field whose gradient encodes surface normals everywhere—including during contact This is not merely a convenient geometric abstraction; it has concrete consequences Crucially, the local DF structure near a hand grasping an object is largely invariant to object size and approach direction, making the representation inherently shape- and scale-agnostic Beyond static geometry, first-order DF cues—surface gradients and velocity decompositions into normal and tangential components—capture the directional dynamics of ongoing contact, providing precisely the interaction signal needed for contact-rich whole-body coordination

--- Page 4 ---

TABLE I: Comparison of humanoid-object interaction methods. We compare representative reference-based and reference-free methods across three axes: whether (i) a unified observation space is shared across tasks, (ii) inference requires no motion references, and (iii) a single policy supports autonomous long-horizon skill composition Ours Ref-free for learning. p. 1

Where the paper says it

We compare representative reference-based and reference-free methods across three axes: whether (i) a unified observation space is shared across tasks, (ii) inference requires no motion references, and (iii) a single policy supports autonomous long-horizon skill composition. Method Type Task-unified observation No motion at inference Long-horizon skill composition HDMI [51] Ref-based ✗ ✗ ✗ ResMimic [62] Ref-based ✗ ✗ ✗ CLONE [26] Ref-based ✗ ✗ ✓ Op3-Soccer [10] Ref-free ✗ ✗ ✗ PhysHSI [50] Ref-free ✗ ✓ ✗ Ours Ref-free ✓ ✓ ✓ for learning.

Where point clouds and voxels discretize space and lose gradient information, and where implicit neural representations are too slow for high-frequency control, the DF provides dense, directional geometric cues at negligible query cost Crucially, the local DF structure near a hand grasping an object is largely invariant to object size and approach direction, making the representation inherently shape- and scale-agnostic Beyond static geometry, first-order DF cues—surface gradients and velocity decompositions into normal and tangential components—capture the directional dynamics of ongoing contact, providing precisely the interaction signal needed for contact-rich whole-body coordination

--- Page 5 ---

Guided by this insight, we introduce LESSMIMIC, a reference-free framework that places the DF at the center of humanoid interaction LESSMIMIC extracts a velocitydecomposed, history-dependent interaction feature from the DF—capturing approach intensity, surface traversal, and geometric evolution—and encodes it into a compact latent via a VAE A single whole-body policy is then trained on this latent using a three-stage pipeline: behavior cloning from a teacher for stable initialization, RL fine-tuning with our proposed AIP for geometric generalization across randomized object geometries, and visual distillation for deployment without MoCap At inference, the policy requires only a target root trajectory and the current DF observation—no motion references, no task-specific rewards, no separate planners

--- Page 6 ---

Fig. 1: Generalizable long-horizon humanoid interaction via LESSMIMIC. p. 0

Where the paper says it

A single policy executes push, pick-up, carry, and sit-stand sequentially without resets. 1: Generalizable long-horizon humanoid interaction via LESSMIMIC.

A single DF-conditioned policy supports (a) online failure recovery through continuous geometric feedback, (b) generalization to unseen object shapes and scales without retraining, and (c) longhorizon composition of heterogeneous interaction skills within a single policy

--- Page 7 ---

TABLE II: Comparison of baselines, ablations, and LESSMIMIC under object-scale variation in simulation. Ours - AIP 5. p. 5

Where the paper says it

4q – – – 81. 9q Ablation study Ours - AIP 0.

2 p˘0.2q 0.0 p˘0.0q 23.3 p˘5.0q 57.3 p˘1.2q 64.0 p˘2.9q 98.7 p˘0.5q 98-3-Ours (Mocap) 63-7p˘5-0,10,14,16,2>

--- Page 8 ---

TABLE III: Long-horizon skill composition with increasing task length in simulation. Our method N=5 N=10 N=15 N=25 N=40 The full model achieves 62. p. 6

Where the paper says it

As shown in Tab. III, the full model achieves 62.

1% success on 5-task trajectories and maintains 2. p. 6
Where the paper says it

III, the full model achieves 62. 1% success on 5-task trajectories and maintains 2.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified Interaction Representation: The system can reason about its relation to an object during interaction through local DF relationships rather than memorizing absolute trajectories, enabling generalization across diverse object geometries without retraining.

  2. Shape and Scale Generalization: The policy can generalize to unseen object shapes and scales because the DF representation is shape- and scale-agnostic, allowing the robot to learn the underlying geometry of interaction behaviors rather than memorizing absolute trajectories.

  3. Long-Horizon Skill Composition: A single policy can execute complex sequences of heterogeneous interaction skills, achieving 62.1% success on 5-task trajectories and remains viable across sequences of up to 40 heterogeneous task instances sequence—a regime where all ablated variants collapse to zero.

  4. Reference-Free Inference: The system operates at inference time requiring only a target root trajectory command and the current DF observation—no motion references, no task-specific rewards, no separate planners.

  5. Visual Deployment Transfer: The system can be deployed without Motion Capture infrastructure by distilling the full policy into a vision-based policy that operates solely on egocentric depth observations, enabling it to recover geometric cues from onboard perception alone.

Abstract

Humanoid robots capable of versatile whole-body interaction with everyday objects represent a central goal of embodied intelligence. Existing approaches often rely on motion-reference inputs or task-specific rewards, coupling policies to particular motion scripts, object geometries, and contact timings. This limits geometric generalization and flexible interaction control, where simple commands specify motion intent while object geometry determines how contact should be executed. We introduce LessMimic, a framework for versatile motion-free-at-inference humanoid-object interaction. At inference, a single whole-body policy is steered by root commands and an interaction-type flag instead of motion-reference inputs, and is conditioned on a compact interaction representation built from short histories of distance-field-derived surface distances and local surface directions. This geometry-conditioned interface enables command-steered control while adapting contact behavior to local object geometry. Through visual distillation, LessMimic further enables egocentric-depth-based deployment for MoCap-free sensing. Across object scales from 0.4x to 1.6x, LessMimic maintains robust performance over four interaction tasks, including PickUp, SitStand, Push, and Carry, where motion-conditioned baselines degrade sharply away from the training scale. Beyond single-task evaluation, LessMimic further supports sequential composition of heterogeneous interaction skills, attaining 62.1% success on five-task sequences and retaining 23.5% success at 15 task instances. By grounding interaction control in local geometry rather than motion references, LessMimic provides a path toward geometry-generalizable, command-steerable humanoid-object control with heterogeneous skill composition.

Sources

Related papers