Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models

arXiv:2512.23077 · cs.RO · Submitted 2025-12-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models".

Dev: Here's the gist of the paper. Discovering effective reward functions remains a fundamental challenge in motor control of highdimensional musculoskeletal systems <ref:2512.23077#pg2>.

Rosa: MoVLR is introduced as a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control <ref:2512.23077#pg2>.

Dev: Framework Overview MoVLR is a framework for automatic reward learning in high-dimensional musculoskeletal systems that integrates descriptive and dynamical feedback <ref:2512.23077#pg2>.

Rosa: It extracts musculoskeletal dynamics through policy optimization and rollout, rendering the resulting behaviors as movement videos <ref:2512.23077#pg2>.

Title and authors: Rosa: So, to get into the specifics of Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models, we have to look at who wrote this. The authors are Saraswati Soedarmadji Chen Xuying Yunyue Wei Chen Zhang Yisong Yue and Yanan Sui.

Dev: That’s a pretty big team spanning Tsinghua University and Caltech, which suggests they’re tackling a problem that requires both deep control knowledge and strong AI capabilities.

Taro: I wonder how their different backgrounds—control engineering versus autonomy research versus vision modeling—actually helped them structure this specific workflow.

Rosa: They seem to have clearly divided the roles; you’ve got the control guys handling the policy optimization, and then the VLM side taking on that perceptual evaluation role.

Dev: The paper itself sets up a finite horizon Markov decision process framework for this reward learning problem, meaning they are treating it like a standard control problem where we're just trying to find that optimal single-step reward r star.

Taro: That’s interesting because the paper says the objective is to find one step reward r star that maximizes the global reward function R, which implies they're aiming for a single, effective learning signal.

Rosa: And they frame it as needing two main things: first, an embodied understanding of human movement to extract that implicit biomechanical knowledge from motion descriptions.

Dev: And second, an effective reward design part where the multimodal feedback—the video and the VLM assessment—is used to create interpretable and executable reward terms for that policy optimization.

Taro: It’s like they’re building a system that learns physics by watching itself move through a loop of action and perception.

Rosa: So, this paper is about taking those complex human movements, which are often implicit, and making them explicit so an AI can learn to replicate them effectively.

Dev: The authors clearly argue that existing methods often rely on episodic statistics or simple success signals, which isn't enough for the temporal dynamics involved in musculoskeletal control.

The paper's summary: Rosa: Moving on to what the paper actually summarizes, they introduce MoVLR as a framework specifically designed to bridge that gap between a high-level goal and the low-level movement control policy.

Dev: MoVLR works by taking a control policy, running it for some time to generate movement videos zeta (i), and then feeding those videos into a vision language model.

Taro: The VLM doesn't just give a number; it evaluates the motion against what the current best reward definition r star is supposed to look like, producing structured biomechanical feedback.

Rosa: That feedback is then used by a language model to refine that reward formulation in the next iteration, effectively doing a local search over the reward space.

Dev: This whole loop means they are iteratively distilling those implicit dynamical regularities—the underlying physics of movement—into explicit reward terms for policy optimization.

Taro: It sounds like they are essentially using language as a tool to reason about physical dynamics in a way that is grounded in what the vision model observes.

Rosa: The paper shows that this iterative refinement process leads to reward terms that are biomechanically consistent, which is the key finding they want to highlight.

Dev: They show that by using this multimodal feedback mechanism, you get interpretable and dynamically consistent reward representations for policy optimization.

The paper's improvements: Rosa: The paper highlights several improvements over previous approaches. One major one is the shift from relying on just episodic statistics to using continuous, structured feedback from the VLM.

Dev: They also emphasize that they are focusing on high-dimensional musculoskeletal systems, which means they’re dealing with redundancy and complexity that traditional torque-driven systems don't handle well.

Taro: And another point is how they handle the failure modes; you see them tested in settings like locomotion environments spanning flat, rough, and sloped terrain.

Rosa: The evaluation shows that MoVLR consistently achieves higher task performance in those locomotion environments compared to reward functions engineered by humans.

Dev: In manipulation tasks, they report achieving the lowest average position and orientation errors when compared against all other baselines they tested.

Taro: What’s interesting is how the evolution of those weighted reward terms across the refinement stages reveals how that feedback-driven process actually reorganizes the internal optimization landscape toward more natural behavior.

Conclusion: Rosa: So, to wrap up on Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models, this framework provides a systematic way to discover reward functions without relying solely on manual engineering.

Dev: It’s about using vision language models as a perceptual bridge so that linguistic goals get grounded in physical dynamics through iterative refinement.

Taro: This approach offers a scalable path toward control for complex behaviors and morphologies because it generalizes across different movement tasks and system designs.

Rosa: The paper suggests that the iterative refinement mirrors the structure of human motor learning, moving from coarse stability constraints to fine-grained joint coordination.

Dev: It’s a principled way to get biologically plausible and interpretable rewards that guide musculoskeletal agents toward stable, natural motion.

Taro: I just think the VLM acting as that perceptual bridge really grounds the linguistic goal in physical dynamics, which is crucial for getting that consistency across different morphologies.

Rosa: That’s right. It offers a scalable path toward control for complex behaviors and morphologies by learning those implicit musculoskeletal dynamics automatically.

Tsinghua University · California Institute of Technology

cs.RO

Submitted: 2025-12-28

Updated: 2026-10-29

Comments: Accepted by L4DC 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Discovering effective reward functions remains a fundamental challenge in motor control of highdimensional musculoskeletal systems <ref:2512.23077#pg2>.

Key concepts

Vision-Language Models (VLMs)
VLMs are AI models that can understand both images (like movement videos) and text descriptions simultaneously. In this framework, the VLM observes the physical motion generated by the control policy and translates it into structured, language-based feedback that helps guide reward learning.
Iterative Reward Refinement
This is a learning loop where an initial reward function is tested against movement data. The VLM provides feedback on how well the movement matches desired dynamics, and a language model uses this feedback to synthesize better, more specific reward terms for the next attempt. This process refines rewards until they accurately reflect natural biomechanics.
Musculoskeletal Dynamics
This refers to the complex physical interactions and movements of a body's muscles, bones, and joints. The framework aims to extract these hidden dynamics from observed motion videos so that the AI can learn control policies that result in stable, biologically plausible movements across different environments.
Finite Horizon MDP
This is a mathematical model used to define the control problem as a sequence of decisions over time, where each decision affects future states. The reward function is defined as a combination of terms derived from the learned dynamics, allowing the system to optimize its actions toward long-term goals.

Terminology

Summary

Discovering effective reward functions remains a fundamental challenge in motor control of highdimensional musculoskeletal systems <ref:2512. p. 0

Where the paper says it

EDU Yanan Sui1 YSUI@TSINGHUA. CN 1Tsinghua University, 2California Institute of Technology Abstract Discovering effective reward functions remains a fundamental challenge in motor control of highdimensional musculoskeletal systems.

23077#pg2>. MoVLR is introduced as a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control <ref:2512. p. 0
Where the paper says it

While humans can describe movement goals explicitly such as “walking forward with an upright posture,” the underlying control strategies that realize these goals are largely implicit, making it difficult to directly design rewards from high-level goals and natural language descriptions. We introduce Motion from Vision-Language Representation (MoVLR), a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control.

23077#pg2>.

Framework Overview

MoVLR is a framework for automatic reward learning in high-dimensional musculoskeletal systems that integrates descriptive and dynamical feedback <ref:2512.23077#pg2>. It extracts musculoskeletal dynamics through policy optimization and rollout, rendering the resulting behaviors as movement videos <ref:2512. p. 1

Where the paper says it

In this paper, we introduce Motion from Vision Language Representation (MoVLR), a framework for automatic reward learning in high-dimensional musculoskeletal systems that integrates descriptive and dynamical feedback. As shown in Figure 1, MoVLR extracts musculoskeletal dynamics through policy optimization and rollout, rendering the resulting behaviors as movement videos.

23077#pg2>. A vision-language model evaluates these motions to produce structured biomechanical feedback, which is then used by a language model to refine the reward formulation in subsequent iterations <ref:2512. p. 1
Where the paper says it

As shown in Figure 1, MoVLR extracts musculoskeletal dynamics through policy optimization and rollout, rendering the resulting behaviors as movement videos. A vision-language model evaluates these motions to produce structured biomechanical feedback, which is then used by a language model to refine the reward formulation in subsequent iterations.

23077#pg2>. This iterative process progressively distills implicit dynamical regularities into explicit reward terms <ref:2512. p. 1
Where the paper says it

A vision-language model evaluates these motions to produce structured biomechanical feedback, which is then used by a language model to refine the reward formulation in subsequent iterations. Through this iterative process, implicit dynamical regularities are progressively distilled into explicit reward terms.

23077#pg2>.

Control and Reward Learning

The paper formalizes the control problem as a finite horizon Markov decision process (MDP) where the reward is a linear combination of reward terms <ref:2512.23077#pg5>. The objective of reward learning is to find a single-step reward r∗ that maximizes the global reward function R <ref:2512. p. 3

Where the paper says it

Reward learning for musculoskeletal control While the above control problem provides single-step reward definition, the control performances are usually evaluated over full horizon. The objective of reward learning is to find single step reward r ∗ that maximize the global reward function R: r ∗ = argmaxrR(ζ), ζ = (s0, a0, · · · , sT −1, aT −1), ut = π ∗ r (st), st+1 = f(st,ut), (5) where π ∗ r is the optimal policy derived by maximizing r.

23077#pg5>. Effective reward learning requires two key components: (1) Embodied understanding of human movement which extracts implicit biomechanical knowledge from motion descriptions, and (2) Effective reward design which integrates multimodal feedback to produce interpretable and executable reward terms for policy optimization <ref:2512. p. 3
Where the paper says it

” The single-step reward r consists of multiple reward terms and parameters as code pieces which need to be compatible with the policy optimization framework. Effective reward learning requires two key components: (1) Embodied understanding of human movement which extracts implicit biomechanical knowledge from motion descriptions, and (2) Effective reward design which integrates multimodal feedback to produce interpretable and executable reward terms for policy optimization.

23077#pg5>.

Multimodal Feedback Mechanism

The VLM acts as a semantic observer that produces interpretable, language-based evaluations rather than scalar scores <ref:2512.23077#pg5>. The VLM compares the rendered control dynamics ζ(i) against the dynamics generated under the current best reward definition <ref:2512. p. 5

Where the paper says it

Motion-language representation We use the VLM as a semantic observer that produces interpretable, language-based evaluations rather than scalar scores. The VLM compares the rendered control dynamics ζ (i) against the dynamics generated under the current best reward definition.

23077#pg5>. This feedback serves as reflective input to the LLM, guiding subsequent iterations of reward synthesis <ref:2512. p. 5
Where the paper says it

The VLM also produces structured textual feedback F of r ∗ which qualitatively evaluates the motion relative to R. This feedback serves as reflective input to the LLM, guiding subsequent iterations of reward synthesis.

23077#pg5>. The LLM performs a local search over the current best reward function r∗ to synthesize new reward terms by reasoning over both linguistic and structural context <ref:2512. p. 5
Where the paper says it

, joint angles, actuator lengths, and contact forces) that can parameterize executable reward terms. Given the contextual information and VLM-based evaluations F, the LLM performs a local search over the current best reward function r ∗ to synthesize new reward terms.

23077#pg5>.

Experimental Validation

MoVLR was evaluated across diverse musculoskeletal systems and tasks, including locomotion environments spanning flat, rough, and sloped terrain <ref:2512.23077#pg9>. Experimental results show that MoVLR consistently achieves higher task performance in locomotion environments compared to human-engineered reward functions <ref:2512. p. 9

Where the paper says it

Through this method, we demonstrate that VLMs can successfully translate high-level motion descriptions into detailed control objectives, improving musculoskeletal performance across diverse environments and tasks. Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines.

23077#pg9>. In manipulation tasks, MoVLR achieved the lowest average position and orientation errors compared to all baselines <ref:2512. p. 7
Where the paper says it

These results indicate that multimodal reward refinement produces robust and transferable control objectives across biomechanical structures. As shown in Figure 4 (b), MoVLR achieves the lowest average position and orientation errors compared to all baselines in the manipulation tasks, indicating more precise and stable object interactions.

23077#pg9>. The evolution of weighted reward terms across refinement stages reveals how the feedback-driven process reorganizes the internal optimization landscape toward biomechanically consistent behavior <ref:2512. p. 7
Where the paper says it

Evolution of weighted reward terms across refinement stages. The progression of residual reward weights across refinement stages reveals how the feedback-driven process reorganizes the internal optimization landscape toward biomechanically consistent behavior.

23077#pg10>.

Generalization and Modularity

Ablation studies show that MoVLR generalizes across movement tasks, environments and system morphologies, producing interpretable reward terms that represent implicit musculoskeletal dynamics <ref:2512.23077#pg10>. The framework demonstrates robustness to actuator failure in the injured-body setting <ref:2512. p. 8

Where the paper says it

In the injured-body setting, transfer performance slightly improves (5. 8 m), indicating robustness to actuator failure.

23077#pg10>. Furthermore, the separation of vision–language feedback and language-based code generation components suggests that current vision–language models do not yet possess the compositional or programmatic reasoning required to perform both tasks simultaneously <ref:2512. p. 9
Where the paper says it

The unified variant shows substantially degraded performance, often producing invalid or incomplete reward code and failing to improve control behavior over iterations. These observations suggest that current vision–language models do not yet possess the compositional or programmatic reasoning required to perform both tasks simultaneously, highlighting the importance of maintaining distinct feedback and synthesis stages.

23077#pg11>. The unified variant shows substantially degraded performance when a single multimodal model performs both feedback interpretation and code synthesis <ref:2512. p. 9
Where the paper says it

To assess whether this modularity is necessary, we implement a unified configuration in which a single multimodal model performs both feedback interpretation and code synthesis. The unified variant shows substantially degraded performance, often producing invalid or incomplete reward code and failing to improve control behavior over iterations.

23077#pg11>.

Conclusion

In this work, MoVLR is introduced as an automatic workflow that leverages vision-language models to bridge explicit language descriptions with implicit motor control required for high-dimensional musculoskeletal systems <ref:2512.23077#pg2>. By integrating multimodal feedback into the learning loop, MoVLR finds biomechanically grounded reward functions that are iteratively refined to guide the musculoskeletal agent toward stable, natural motion <ref:2512. p. 9

Where the paper says it

Conclusion and Discussion In this work, we introduce MoVLR , an automatic workflow that leverages vision-language models to bridge explicit language descriptions with implicit motor control required for high-dimensional musculoskeletal systems. By integrating multimodal feedback into the learning loop, MoVLR finds biomechanically grounded reward functions that are iteratively refined to guide the musculoskeletal agent toward stable, natural motion.

23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref: p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

2512.
p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

p. 9
Where the paper says it

Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies.

23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512. p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

p. 9
Where the paper says it

Experimental results across locomotion and manipulation tasks show that MoVLR consistently outperforms both human-designed and language-based baselines. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512. p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

p. 9
Where the paper says it

Beyond performance gains, MoVLR highlights a fundamental connection between explicit language intent and implicit reward emergence. Acting as a perceptual bridge, the VLM grounds linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations.

23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:2512.23077#pg11>. The iterative refinement of rewards mirrors the hierarchical structure of human motor learning, progressing from coarse stability constraints to fine-grained joint coordination <ref:2512.23077#pg10>. The VLM acts as a perceptual bridge, grounding linguistic goals in physical dynamics, producing interpretable and dynamically consistent reward representations <ref:2512.23077#pg11>. This work offers a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors and morphologies <ref:

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Embodied Learning of Reward for Musculoskeletal Control: The system can discover explicit reward terms that represent implicit musculoskeletal dynamics by iteratively exploring reward space through iterative interaction between control optimization and VLM feedback.

  2. Bridging Perception and Dynamics: The improved AI can translate abstract motion descriptions into structured guidance for embodied learning, grounding temporal dynamics in semantically meaningful descriptors to bridge perceptual feedback with domain-informed motion representations.

  3. Multi-modal Reward Refinement: The system can refine reward functions by incorporating both video observations of policy-executed trajectories and VLM assessments, enabling it to produce reward terms that capture nonlinear couplings between balance, posture, and coordination.

  4. Scalable Reward Discovery: MoVLR provides a framework for automatic reward learning in high-dimensional musculoskeletal systems that generalizes across movement tasks, environments and system morphologies, producing interpretable reward terms.

Abstract

Discovering effective reward functions remains a fundamental challenge in motor control of high-dimensional musculoskeletal systems. While humans can describe movement goals explicitly such as "walking forward with an upright posture," the underlying control strategies that realize these goals are largely implicit, making it difficult to directly design rewards from high-level goals and natural language descriptions. We introduce Motion from Vision-Language Representation (MoVLR), a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control. Rather than relying on handcrafted rewards, MoVLR iteratively explores the reward space through iterative interaction between control optimization and VLM feedback, aligning control policies with physically coordinated behaviors. Our approach transforms language and vision-based assessments into structured guidance for embodied learning, enabling the discovery and refinement of reward functions for high-dimensional musculoskeletal locomotion and manipulation. These results suggest that VLMs can effectively ground abstract motion descriptions in the implicit principles governing physiological motor control.

Sources

Related papers