MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation

arXiv:2603.27185 · cs.CV · Submitted 2026-03-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation".

Tom: The gist The paper presents a reinforcement fine-tuning framework, MotionRFT, which integrates a unified heterogeneous-representation reward model with an efficient,

Jane: First, who's behind it and why it matters.

Paper summary: Tom: We’ve just covered the core idea of MotionRFT, which is this unified reinforcement fine-tuning framework. Basically, it moves beyond simply targeting one motion representation or optimizing just text alignment.

Jane: It’s about creating a system that evaluates motion across multiple dimensions—realism, how well it matches the text, and human preference—all at the same time using a single reward mechanism called MotionReward pg2.

Lu: The paper claims this model overcomes three big hurdles: first, the discrepancy between different motion representations making rewards hard to compare; second, having only single objectives that don't capture realism and alignment together; and third, the lack of large scale human preference data.

Meng: And to fix those hurdles, they introduce a unified semantic representation mapped into a shared embedding space anchored by the text itself pg2. This gives them one solid ground for multi-dimensional learning.

Tom: That shared semantic space then lets them use LoRA parameters to transfer that understanding into other reward domains where specific rewards like preference and authenticity can be learned efficiently pg2.

Jane: And they added Self-refinement Preference Learning, which dynamically creates preference pairs from existing data using hard negative mining without needing any new human annotations pg9.

Lu: So, it’s a comprehensive system for evaluation that tries to capture semantics through reconstruction losses and consistency losses while also incorporating distribution regularization and InfoNCE loss pg4.

Meng: From an engineering standpoint, the unified reward model needs to be robust enough to handle all those different signal types without collapsing or giving weird results when the representations are mixed together.

Tom: That robustness is key, because if MotionReward gets confused by mixing joint and rotation data, the whole system falls apart. It has to work seamlessly across these different inputs pg2.

Jane: And that’s where EasyTune steps in; it addresses the practical problem of how to actually fine-tune this complex system without storing a massive computational graph for every step pg6.

Lu: EasyTune performs reinforcement fine-tuning at each denoising step instead of optimizing the whole trajectory, which means it decouples the gradients from the full reverse chain pg7.

Meng: So, for us engineers, that decoupling is a huge win because it allows for dense and fine-grained optimization without needing to store all those intermediate states in memory pg4.

Tom: It makes sense; it turns a huge computational problem into many smaller, manageable ones, which is much easier to handle during training. So the next thing we’re looking at is how they actually schedule these steps.

Conclusion: Jane: We’ve spent this time discussing the MotionRFT framework, which combines MotionReward and EasyTune. To wrap things up, it seems like this work is about unifying evaluation for text-to-motion generation.

Tom: The authors are Xiaofeng Tan, Wanjiang Weng, Hongsong Wang, Fang Zhao, Xin Geng and Liang Wang pg1. They’ve successfully built a framework that handles heterogeneous motion inputs and multi-dimensional evaluation goals.

Lu: They’ve shown that mapping everything into a shared space anchored by text is a powerful way to make the system learn semantic consistency across those different representations across those formats pg2.

Meng: The real practical implication seems to be that we can get much better motion generation results, especially when we look at efficiency gains like reducing memory usage significantly over prior baselines.

Jane: So while they are working on things like self-refinement preference learning and step-wise optimization, the framework aims to make the entire process more robust for generating high quality motion.

Tom: It’s a lot of work to build this system, but they showed that by combining a unified reward model with an efficient fine-tuning method that is step-aware, you can get significant improvements in fidelity and efficiency.

Southeast University · National Science Foundation of China

cs.CV

Submitted: 2026-03-28

Updated: 2026-10-09

Comments: Revised version

License: http://creativecommons.org/publicdomain/zero/1.0/

Importance score: 90/100

The gist: The gist The paper presents a reinforcement fine-tuning framework, MotionRFT, which integrates a unified heterogeneous-representation reward model with an efficient, step-aware fine-tuning method to

Key concepts

MotionReward: This is a reward model designed to evaluate generated motions using multiple dimensions. It maps different types of motions (like joint or rotation data) into one shared semantic space anchored by the text prompt. This allows the model to learn rewards for things like text alignment, realism, and human preference simultaneously.
EasyTune: This is an efficient fine-tuning method that solves a major problem in previous methods: recursive gradient dependence. Instead of calculating gradients across the entire motion trajectory at once, EasyTune performs reinforcement learning optimization at each individual denoising step. This makes the training process faster and more memory-efficient by optimizing incrementally.
Self-refinement Preference Learning (SPL): SPL is a technique used to improve the semantic reward model without needing new human annotations. It dynamically creates preference pairs by comparing a motion against its own past samples and retrieving 'hard negatives'—motions that scored highly but were incorrect. This process refines the reward signals to better capture nuanced human preferences.
Unified Semantic Representation: This is a shared embedding space where all different motion representations (e.g., joint coordinates, rotation data) are projected into a single feature space. By aligning these diverse inputs with the text prompt in this unified space, the model can learn consistent rewards that apply across all motion formats.

Terminology

Fine-Tuning — 44× in this paper · explained in 5 episodes
Diffusion Model — 31× in this paper · explained in 10 episodes
Diffusion Models — 16× in this paper · explained in 9 episodes
Consistency — 12× in this paper · explained in 3 episodes
Semantic Alignment — 12× in this paper · explained in 4 episodes
Accuracy — 11× in this paper · explained in 1 episode
Generalization — 10× in this paper · explained in 13 episodes
Self-refinement Preference Learning — 9× in this paper
Embeddings — 8× in this paper · explained in 2 episodes
Reinforcement Learning — 7× in this paper · explained in 4 episodes
Conference on Computer Vision and Pattern Recognition — 6× in this paper
Generative Model — 6× in this paper · explained in 2 episodes
Fidelity — 5× in this paper · explained in 2 episodes
Representation Learning — 5× in this paper · explained in 3 episodes
Coherence — 4× in this paper · explained in 2 episodes
Computational overhead — 4× in this paper · explained in 2 episodes
Core — 4× in this paper · explained in 2 episodes
Decoder — 4× in this paper · explained in 2 episodes
Generative Models — 4× in this paper · explained in 3 episodes
International Conference on Computer Vision — 4× in this paper
LoRA Adapters — 4× in this paper · explained in 3 episodes
Regularization — 4× in this paper · explained in 3 episodes
Ablation — 3× in this paper · explained in 1 episode
Contrastive Learning — 3× in this paper · explained in 7 episodes
Flow Matching — 3× in this paper · explained in 15 episodes
Ablation Study — 2× in this paper · explained in 3 episodes
Adaptation — 2× in this paper · explained in 1 episode
Editor — 2× in this paper · explained in 2 episodes
Flow Matching Model — 2× in this paper · explained in 3 episodes
GRPO — 2× in this paper · explained in 6 episodes
Generator — 2× in this paper · explained in 2 episodes
Latent Space — 2× in this paper · explained in 7 episodes
Loss Function — 2× in this paper · explained in 2 episodes
Aggregation — 1× in this paper
Attention Mechanism — 1× in this paper · explained in 3 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Continual Learning — 1× in this paper · explained in 3 episodes
Cross-modal Alignment — 1× in this paper · explained in 2 episodes
Decoupling — 1× in this paper · explained in 2 episodes
F1 Score — 1× in this paper · explained in 3 episodes
Foundation Model — 1× in this paper · explained in 3 episodes
Frechet Inception Distance — 1× in this paper
Generative Modeling — 1× in this paper · explained in 3 episodes
Hallucination — 1× in this paper · explained in 4 episodes
LoRA Adaptation — 1× in this paper · explained in 2 episodes
New Laboratory of Pattern Recognition — 1× in this paper
Objective Function — 1× in this paper · explained in 2 episodes
Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
Reward Hacking — 1× in this paper · explained in 11 episodes
Robustness — 1× in this paper · explained in 5 episodes
Self-Supervised Learning — 1× in this paper · explained in 8 episodes
Semantic Understanding — 1× in this paper · explained in 2 episodes
Transformer — 1× in this paper · explained in 2 episodes

Summary

The gist The paper presents a reinforcement fine-tuning framework, MotionRFT, which integrates a unified heterogeneous-representation reward model with an efficient, step-aware fine-tuning method to enhance text-to-motion generation across various motion representations and objectives.

Unified Reward Model

MotionReward is designed to overcome three challenges in multi-dimensional evaluation: (1) the discrepancy across heterogeneous motion representations that makes reward signals hard to compare and generalize, (2) the limitation of single objective rewards that fail to jointly capture motion realism, motion to text alignment, and human perceived preference, and (3) the scarcity of large-scale, high-quality preference annotations. To address the representation discrepancy, it establishes a unified semantic representation where heterogeneous motion representations are mapped into a shared embedding space via lightweight linear projections and aligned with the textual anchor; this unified semantic representation serves as the natural text-motion alignment reward and provides a unified representation for heterogeneous motions pg2. Building upon this, it adopts a lightweight LoRA parameter to transfer the semantic representation into other reward domain, where specific rewards, such as human preference and motion authenticity, can be captured and learned in an efficient way; furthermore, Self-refinement Preference Learning (SPL) is proposed to refine the pre-trained reward model without additional data by dynamically constructing preference pairs using pre-training samples as positives and failed retrieval results as negatives pg2.

Efficient Fine-Tuning Method

EasyTune is introduced to address the key bottleneck in existing differentiable reward-based methods, which is the recursive gradient dependence across denoising steps, meaning that the reward of the final motion Rϕ(xθ) depends on all intermediate states and the gradient ∇θxθt must backpropagate through the entire reverse chain, which requires storing the full computational graph pg6. EasyTune performs reinforcement fine-tuning at each denoising step to decouple gradients from the full trajectory, enabling dense and fine-grained optimization and expensive memory saving pg4. The training objective is defined as LEasyTune(θ) = -Ec∼DT,xθt ∼πθ(·c),t∼U(0,T) h Rϕ(xθt, t, c)i pg7. This step-wise optimization requires a noise-aware reward model that can evaluate noised motions at each step; the unified reward aggregates four terms: Rϕ(xt, t, c) = w1 Rsϕ(xt, t, c) + w2 Rpϕ(xt, t) + w3 Raϕ(xt, t) + w4 Rmϕ (xt, t), where Rsϕ, Rp ϕ, and Ra ϕ are the semantic, preference, and authenticity reward heads from MotionReward pg7.

Curriculum Timestep Scheduling

To address the vanishing coefficient Qt−1s=1∂πθ∂xθs renders prior methods ineffective at high-noise steps, EasyTune adopts a coarse-to-fine curriculum that gradually shifts optimization from high-noise to low-noise steps pg8. Gradients are computed only within a sliding window of k consecutive steps, W(p) = s(p),..., s(p)+k−1, where the start index s is scheduled by the training progress p ∈ [0, 1] pg8. For p ≤ ρ, the window W(p) linearly sweeps from high-noise steps to low-noise steps, while once p > ρ, the window is fixed at low-noise steps for fine-grained refinement pg8.

Effectiveness and Generalization

Extensive experiments validate the effectiveness of this framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT pg1. The framework demonstrates strong cross-model and cross-representation generalization, achieving a 70.7% improvement in FID on joint-based ACMDM and a 23.3% FID improvement on rotation-based HY Motion pg1. MotionReward achieves the best text-motion retrieval accuracy across all three representations, improving over ReAlign [29] by 1.5% on kinematic features and outperforming the corresponding baselines by over 8% on joint and rotation representations pg9. EasyTune achieves state-of-the-art performance with an FID of 0.132, a 70.7% improvement over MLD [8], while reducing peak memory usage to 22.10 GB pg9.

Conclusion

MotionRFT successfully establishes a unified semantic space across heterogeneous motion representations and trains multi-dimensional rewards for semantic alignment, human preference, and motion authenticity pg1. EasyTune effectively addresses the recursive gradient dependence in differentiable-reward optimization by performing step-aware per-step updates, enabling dense and memory-efficient finetuning pg9. Extensive experiments validate strong cross-model and cross-representation generalization, achieving FID 0.132 with 22.10 GB peak memory, saving up to 15.22 GB over prior baselines, reducing FID by 22.9% on joint-based ACMDM, and improving rotation-based HY Motion by 12.6% in R-Precision and 23.3% in FID >MotionRFT is a unified reinforcement fine-tuning framework for text-to-motion generation, comprising MotionReward and EasyTune pg1. The paper's work is an extension of their published conference version pg9.

How it works

The framework comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient and effective fine-tuning method, EasyTune pg2. MotionReward learns a unified semantic space by projecting heterogeneous motions into a shared feature space with lightweight adapters and aligning them with text embeddings pg4. This involves mapping each representation o ∈ O = k, j, r into a shared feature space ho = ϕo(xo) ∈ R T × d, where d=256 pg4. The optimization objectives for MotionReward include semantic reconstruction loss Lrec, distribution regularization loss Lkl, semantic consistency loss Llat, and InfoNCE loss Lcl pg4.

EasyTune is a step-aware differentiable reward-based fine-tuning framework that solves the recursive gradient issue by performing reinforcement fine-tuning at each denoising step pg7. It maximizes the reward value at each step, defined as LEasyTune(θ) = -Ec∼DT,xθt ∼πθ(·c),t∼U(0,T) h Rϕ(xθt, t, c)i pg7. The training objective is defined as LEasyTune(θ) = -Ec∼DT,xθt ∼πθ(·c),t∼U(0,T) h Rϕ(xθt, t, c)i pg7.

Self-refinement for Semantic Enhancement

Self-refinement Preference Learning (SPL) is proposed to refine the semantic reward by dynamically constructing preference pairs from existing data via hard negative mining without additional human annotations pg9. Given a motion x and a text description c, the semantic reward is computed as Rϕ(x, c) = cosEM(x), ET(c) pg4. Hard Negative Mining involves retrieving the top-k motions from the training set based on reward scores; if the ground-truth motion xgt is not among the top-k, the highest-scoring incorrect motion serves as a hard negative pg9. Preference-based Refinement minimizes LSPL(ϕ) = DKL(Q∥ P), where P is the probability distribution derived from reward scores, encouraging higher scores for matched motions and lower scores for hard negatives pg4.

Ablation Study and Results

EasyTune achieves the best overall performance across key metrics, attaining the lowest FID of 0.132, improving by 70.

Improvements for AI systems

textbf Unified Heterogeneous Representation Reward Model (MotionReward): The framework maps heterogeneous motions into a shared semantic space anchored by text, enabling it to evaluate motion quality across kinematic-, rotation-, and joint-based representations in semantics, preference, and authenticity aspects.
textbf Self-Refinement Preference Learning (SPL): Enhanced Semantic Alignment: SPL dynamically constructs preference pairs by using pre-training samples as positives and failed retrieval results as negatives, which refines the reward model to enhance semantics without additional annotations.
textbf EasyTune: Fine-Grained, Memory-Efficient Optimization: By performing reinforcement fine-tuning at each denoising step, EasyTune decouples gradients from the full trajectory, achieving dense and fine-grained optimization and expensive memory saving by maintaining a constant O(1) memory footprint instead of O(T).
textbf Multi-Dimensional Reward Aggregation: Comprehensive Quality Assessment: The unified reward aggregates four terms—semantic alignment Rs, human preference Rp, authenticity Ra, and motion-to-motion similarity Rmϕ—allowing the model to simultaneously optimize for motion realism, text-to-motion alignment, and human perceived preference.
textbf Curriculum Timestep Scheduling: Optimized Optimization Flow: The framework implements a coarse-to-fine curriculum that gradually shifts optimization from high-noise to low-noise steps, ensuring that early denoising steps mainly determine global structure while late steps refine local semantics.

Abstract

Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pre-training remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multi-dimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.1% R-Precision gain and 21.7% FID improvement on rotation-based HY-Motion. Our project page with code is publicly available.