HumanScore: Benchmarking Human Motions in Generated Videos

arXiv:2604.20157 · cs.CV · Submitted 2026-04-22 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "HumanScore: Benchmarking Human Motions in Generated Videos".

Jane: Here's the gist of the paper. HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos This benchmark extracts structured human motion from generated videos and scores it using a suite of anatomically- and biomechanically grounded metrics Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e.g., temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria.

Tom: Evaluation Framework HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency The evaluation framework follows a three-tier hierarchy: the base tier enforces anatomical correctness (valid human topology with stable body proportions), the middle tier enforces kinematic correctness (geometric feasibility of poses), and the top tier enforces kinetic correctness (temporal coherence and physically plausible dynamics).

Jane: The metrics are designed to follow this hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions, hyperextension or interpenetration, non-human spikes or jitter, joint range of motion, self-collision, and kinematic extremes.

Tom: Metric Design Details The metrics are calculated using a generic framework where per-frame calculations produce three outputs: bt – a binary indicator that flags whether a frame is normal or abnormal, mt – a relative metric score that measures how “bad” a frame is, in the range (0, 1), and Lmax – the length of the longest consecutive abnormal (according to bt) frame segment.

Paper summary: Tom: To break down the paper, they define six specific metrics that cover kinematic plausibility, temporal stability, and biomechanical consistency. They use a three-tier hierarchy where you check anatomical correctness first, then geometric feasibility for poses, and finally kinetic correctness for how the motion actually flows over time.

Jane: That hierarchy makes sense because it starts with the basics—does the body even have a stable shape to begin with?

Lu: Yes, the base tier focuses on anatomical correctness, meaning checking if you have valid human topology with stable body proportions across all frames.

Meng: And then they move up to kinematic correctness, which is checking if the poses are geometrically feasible—like can a limb actually reach that position without breaking physics?

Tom: Then there's the top tier, kinetic correctness, which is about temporal coherence and physically plausible dynamics over time. That’s where you check for things like motion drift or sudden jolts in the action.

Lalam: The metrics are calculated frame by frame using a generic system that outputs three values: bt, mt, and Lmax.

Jane: So bt is a binary indicator flagging if a frame is normal or abnormal, mt is a relative metric score from zero to one measuring how bad that frame is, and Lmax tracks the longest streak of abnormal frames.

Tom: And then they combine these into one final video-level metric using a weighted sum: D equals alpha r plus beta s plus gamma p, and the final score is one hundred minus D, with those weights being zero point five for r, zero point three for s, and zero point two for p <ref:2604.20157#pg2>.

Lu: Specifically they use metrics like Extra Limbs to detect anomalous limbs using HADM to find abnormal limbs, Bone Length by fitting OpenSim skeletons to see the relative length deviation from the median length over time.

Meng: And they also check self-collision using a bounding volume hierarchy and fast triangle-triangle intersection tests, which is pretty intense computation for frame-level checking.

Jane: They also have checks for joint range of motion against anatomical limits, kinematic extremes by looking at joint angular and body segment linear velocities against biomechanics standards, and motion smoothness by comparing angular acceleration and jerk to per degree of freedom limits.

Conclusion: Tom: So, to wrap up the "HumanScore: Benchmarking Human Motions in Generated Videos" paper, the core message is that these six metrics provide a principled, quantitative evaluation of motion realism from a biomechanical perspective.

Jane: They show that when you apply this benchmark to state-of-the-art video generators—like Sora two point two and Veo three point one fast—you find consistent weaknesses in motion fidelity despite their visual appeal.

Lu: The paper's contributions are providing the first systematic evaluation framework centered on human motion in generated videos, proposing quantitative indicators grounded in anatomy and biomechanics with transparent baselines, and applying it to show where the gaps between visual appearance and physical realism are.

Meng: What this means for deployment is that even if a model looks great visually, you still have to check these physical plausibility scores before you trust it for anything important like sports commentary or training simulations.

Lalam: The authors found that the correlation between their HumanScore metric values and human preference win ratio shows strong agreement across all six evaluation dimensions, with Spearman correlation coefficients close to one point zero.

Tom: That strong agreement is significant because it means the numbers they calculate actually track what people perceive as good motion very well.

Jane: So, in simple terms, this paper gives us a reliable tool to distinguish real human movement from synthetic movement in AI videos. It shifts the focus from just looking good to being physically believable.

Stanford University · Peking University

cs.CV

Submitted: 2026-04-22

Updated: 2026-11-03

Project page: https://cs.stanford.edu/~xtiange

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 95/100

The gist: HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore…

Key concepts

Evaluation Framework
A three-tier system for judging motion quality: base tier checks anatomical correctness (correct body structure), middle tier checks kinematic correctness (feasible poses), and the top tier checks kinetic correctness (smooth, physically plausible movement).
Metric Design Details
Calculations use per-frame outputs like bt (normal/abnormal flag) and mt (severity score) to aggregate into video metrics. The final score is a weighted sum of three components—r, s, and p—which are mapped to a 100-point scale representing quality.
Kinematic Extremes
This metric evaluates how natural the motion is by checking joint angular and body-segment linear velocities against established biomechanical standards. High scores indicate more natural and physically plausible movement patterns.
Human Preference Alignment
The HumanScore metrics strongly correlate with human preference ratings, showing high Spearman correlation coefficients. This validates the framework as a reliable way to measure how well generated human motion aligns with what humans perceive as realistic.

Terminology

Summary

HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos This benchmark extracts structured human motion from generated videos and scores it using a suite of anatomically- and biomechanically grounded metrics Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e. p. 0

Where the paper says it

Through carefully designed prompts, we elicit a diverse set of movements at varying intensities and evaluate videos generated by thirteen state-of-the-art models. Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e.

g., temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria. p. 0
Where the paper says it

Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e. , temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria.

Evaluation Framework

HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency The evaluation framework follows a three-tier hierarchy: the base tier enforces anatomical correctness (valid human topology with stable body proportions), the middle tier enforces kinematic correctness (geometric feasibility of poses), and the top tier enforces kinetic correctness (temporal coherence and physically plausible dynamics). The metrics are designed to follow this hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions, hyperextension or interpenetration, non-human spikes or jitter, joint range of motion, self-collision, and kinematic extremes. p. 6

Where the paper says it

The metric designs in HumanScore follow the principle of such hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions (the base tier); hyperextension or interpenetration (the middle tier); and non-human spikes or jitter (the top tier). Kinetic consistency Kinematic correctness Anatomy correctness (v) Kinematic Extremes (vi) Motion Smoothness (iv )Self-collision(iii) Joint Range of Motion (i) Extra Limbs (ii) Bone Length Biomechanical Hierarchy Evaluation Dimensions F = ma Fig.

Metric Design Details

The metrics are calculated using a generic framework where per-frame calculations produce three outputs: bt – a binary indicator that flags whether a frame is normal or abnormal, mt – a relative metric score that measures how “bad” a frame is, in the range (0, 1), and Lmax – the length of the longest consecutive abnormal (according to bt) frame segment. The final video-level metric is given by a weighted sum of these three aggregated scores: D = alpha r + beta s + gamma p, Score = 100(1 - D), where alpha=0. p. 8

Where the paper says it

Similar to the Kinematic Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum. For all the metrics computed above, we normalize their scores to a scale from 0 to 100.

5, beta=0.3, and gamma=0.2 are hyper-parameters. p. 6
Where the paper says it

Specifically, we use HADM [69], a recent model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates, we add temporal-consistency checks and tune its parameters to better capture dynamic ghosting.

p. 6
Where the paper says it

Specifically, we use HADM [69], a recent model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates, we add temporal-consistency checks and tune its parameters to better capture dynamic ghosting.

p. 6
Where the paper says it

Specifically, we use HADM [69], a recent model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates, we add temporal-consistency checks and tune its parameters to better capture dynamic ghosting.

Specific metrics include Extra Limbs using HADM [69] to detect anomalous limbs, Bone Length using fitted OpenSim skeletons to compute the relative l1 deviation from its median length over time, Joint Range of Motion checking violations against predefined anatomical limits, Self-collision using a Bounding Volume Hierarchy and fast triangle–triangle intersection tests, Kinematic Extremes evaluating joint angular and body-segment linear velocities against biomechanics standards, and Motion Smoothness comparing angular acceleration and jerk against per-DoF limits. p. 8
Where the paper says it

Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits. Similar to the Kinematic Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum.

Experimental Results

The experiments benchmarked thirteen video generation models, including four open-source systems—Wan 2.2 [68], CogVideoX-5B [74], HunyuanVideo 1. p. 8

Where the paper says it

1 Evaluate Video Generation Models We benchmark thirteen video generation models, including four open-source systems—Wan 2. 2 [68], CogVideoX-5B [74], HunyuanVideo 1.

5 [72], and Kandinsky 5. p. 8
Where the paper says it

2 [68], CogVideoX-5B [74], HunyuanVideo 1. 5 [72], and Kandinsky 5.

0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. p. 8
Where the paper says it

5 [72], and Kandinsky 5. 0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3.

1 fast [38], KlingAI 2. p. 8
Where the paper says it

0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI 2.

p. 8
Where the paper says it

0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI 2.

5 Turbo Pro [2], Seedance 1. p. 8
Where the paper says it

1 fast [38], KlingAI 2. 5 Turbo Pro [2], Seedance 1.

p. 8
Where the paper says it

1 fast [38], KlingAI 2. 5 Turbo Pro [2], Seedance 1.

0 Pro fast [16], Hailuo 02 [45], Pika v2. p. 8
Where the paper says it

5 Turbo Pro [2], Seedance 1. 0 Pro fast [16], Hailuo 02 [45], Pika v2.

2 [49]. The best score in each dimension is highlighted in cell colors on the HumanScore Leaderboard. p. 9
Where the paper says it

Higher scores indicate better performance. The best score in each dimension is highlighted in cell colors.

Seedance 1.0 Pro fast and HunyuanVideo 1. p. 9
Where the paper says it

According to the leaderboard reported in Table 1, Seedance 1. 0 Pro fast and HunyuanVideo 1.

5 co-lead the leaderboard overall (91. p. 9
Where the paper says it

0 Pro fast and HunyuanVideo 1. 5 co-lead the leaderboard overall (91.

1), followed closely by KlingAI 2. p. 9
Where the paper says it

5 co-lead the leaderboard overall (91. 1), followed by KlingAI 2.

5 Turbo Pro (90. p. 8
Where the paper says it

1 fast [38], KlingAI 2. 5 Turbo Pro [2], Seedance 1.

p. 8
Where the paper says it

1 fast [38], KlingAI 2. 5 Turbo Pro [2], Seedance 1.

8). The gap between real and generated videos indicates that current generators still struggle with biomechanics-constrained motion despite strong visual realism. p. 9
Where the paper says it

We also include real videos as an upper-bound reference; as expected, they score highest overall (94. The gap between real and generated videos indicates that current generators still struggle with biomechanics-constrained motion despite strong visual realism.

Validation and Conclusions

The correlation between HumanScore metric values and human preference win ratio shows strong agreement across all evaluation dimensions, with Spearman correlation coefficients close to 1.0. Furthermore, the average score on real-world videos was 94. p. 1

Where the paper says it

1 Introduction Recent advances in generative modeling have led to rapid progress in visual content creation. When training on large-scale real-world data, scaled models are now able to generate compelling scenes with realistic appearance, lighting, and camera motion, and they score highly on prevailing video benchmarks such as VBench [23] and T2V-CompBench [61].

3, which is higher than the results from all generated videos, validating the effectiveness of our proposed metrics in distinguishing real motion from synthetic. p. 1
Where the paper says it

Yet, despite these advances, the realism of human motion remains a distinguishing signal between real and generated content. This convergence—pose estimators achieving stable performance and generative models saturating at standard visual metrics—makes it timely and necessary to push for

The findings highlight that while leading models produce visually compelling videos, a substantial gap remains in achieving biomechanically accurate human motion. p. 1
Where the paper says it

Yet, despite these advances, the realism of human motion remains a distinguishing signal between real and generated content. This convergence—pose estimators achieving stable performance and generative models saturating at standard visual metrics—makes it timely and necessary to push for

HumanScore provides a timely framework for examining the physical/biological plausibility of generated human motion. p. 0
Where the paper says it

In this paper, we present HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos. HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency, enabling fine-grained diagnosis beyond visual realism alone.

Limitations and Future Directions

A limitation is that HumanScore relies on external models to compute its metrics, which can introduce inaccuracies, particularly in components such as human pose estimation. Moreover, even state-of-the-art fitting methods remain fundamentally imperfect and cannot perfectly recover all human motions from monocular video. p. 7

Where the paper says it

Detecting interpenetration requires accurate 3D modeling of human geometry from videos. We therefore rely on the state-of-the-art monocular fitting method PromptHMR [70] to obtain the best possible 3D SMPL-X meshes.

Future work expects this benchmark to serve as a useful indicator for evaluating generated human motion from both open-source and proprietary video generation models. p. 3
Where the paper says it

The refined prompts are then passed to both proprietary and open-source state-of-the-art video generation models. Human verification is incorporated at all stages for quality check.

The modular design of HumanScore means it will naturally benefit from future improvements in the underlying vision models. p. 1
Where the paper says it

When training on large-scale real-world data, scaled models are now able to generate compelling scenes with realistic appearance, lighting, and camera motion, and they score highly on prevailing video benchmarks such as VBench [23] and T2V-CompBench [61]. Yet, despite these visual improvements, people can still distinguish generated content from real ones based on a key signal: human motion.

The paper concludes by motivating the incorporation of biomechanical and physical constraints into video generation models. p. 2
Where the paper says it

2 Human Motion Evaluation Methods Motion-focused evaluation in videos often emphasizes perceptual alignment or distributional similarity rather than explicit biomechanical constraints. As also seen in Video-Bench [21], such metrics include action consistency, motion quality, and temporal consistency; perception-driven motion metrics (PMM) from VMBench [34, 35]; recognizer-based Action-Score [28, 32, 33, 71]; and distributional

Full Prompt Set

In this benchmark, we include 51 unique motion types, each paired with gentle and intense prompt variants, for a total of 102 prompts. The motions are balanced across three difficulty levels: 17 easy, 17 medium, and 17 hard. p. 5

Where the paper says it

This quality control step uses both objective measures (using the metrics proposed in this work) and manual subjective inspection. The final result is a curated set of 51 distinct motion types, balanced across three difficulty levels—simple, medium, and hard—with 17 motion types per level.

Table 3 lists the full list of prompts from the 51-motion benchmark. p. 3
Where the paper says it

3 HumanScore Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components: (i) Curation of the list of motions that ask video generators to generate and to benchmark; (ii) Carefully designed prompts given motion types, for reliable

The prompts are designed to mitigate model-specific biases and ensure consistent conditioning across generators. p. 3
Where the paper says it

The pipeline begins with curating a representative set of human motions from a large pool of common actions. For each motion, we carefully design prompts to mitigate model-specific biases and ensure consistent conditioning across generators.

The final prompts entail five components: Scene, Motion, Intensity, Description, and Camera. p. 5
Where the paper says it

’ Finally, to (iv) ensure a single, focused subject, we explicitly enforce ‘A single person’. Combining these standardized elements with attributes for the motion itself, our final prompts entail five components: Scene, Motion, Intensity, Description, and Camera.

Video Generator Descriptions

The evaluation includes models such as Sora-2 [46], Veo 3.1 fast [38], KlingAI 2.5 Turbo Pro [2], and Seedance 1. p. 8

Where the paper says it

1 fast [38], KlingAI 2. 5 Turbo Pro [2], Seedance 1.

0 Pro fast [16]. p. 8
Where the paper says it

0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI 2.

These models are grouped into open-source systems and proprietary systems. p. 8
Where the paper says it

4 Experiments and Main Results 4. 1 Evaluate Video Generation Models We benchmark thirteen video generation models, including four open-source systems—Wan 2.

For example, Sora-2 is OpenAI’s second-generation, proprietary video-andaudio generation model designed to produce high-resolution videos with more accurate physics. p. 0
Where the paper says it

HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye. Recent advances in model architectures, compute, and data scale have driven rapid progress in video generation, producing increasingly realistic content.

Veo 3.1 Fast is Google’s latest video generator integrated into the Gemini ecosystem. p. 2
Where the paper says it

HumanScore 3 principled, quantitative evaluation of motion realism from a biomechanical perspective. See Figure 1 and challenge yourself to distinguish between real videos and those generated by the latest video generators.

Detailed Metric Calculations

The calculation for Extra Limbs involves a frame-level calculation where b t is a binary indicator that shows whether the frame has a strong enough detection of anomalous limbs, and mt is the per-frame severity score in the range [0, 1]. The video-level aggregation seeks to obtain a single numeric value as a unified score that represents the entire video from the three aspects: frequency, severity, and persistence. p. 3

Where the paper says it

3 HumanScore Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components: (i) Curation of the list of motions that ask video generators to generate and to benchmark; (ii) Carefully designed prompts given motion types, for reliable

For Bone Length, the bone-level average error Eb is computed across all bones b. p. 6
Where the paper says it

We run HADM on each frame and aggregate the predicted limb-detection confidence scores across the video to compute the average metric. (II) Bone length.

The final score for this metric is given by a weighted sum of the three aggregated quantities D = alpha r + beta s + gamma p, Score = 100(1 - D), where alpha=0. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

5, beta=0.3, and gamma=0.2 are hyper-parameters. The calculation for Motion Smoothness involves angular acceleration and jerk compared against per-DoF limits defined by biomechanics standards. p. 8
Where the paper says it

Jerk is accumulated as local energy over a short temporal window. Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits.

The final score is again a weighted sum of (r, s, p) and is linearly mapped to the range [0, 100], where higher values indicate smoother and more physically plausible motion. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

The calculation for Self-collision involves a weighted combination of three aspects: r severe, s, and p. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

The final metric value is obtained from a weighted combination of these three aspects D = alpha r severe + beta s + gamma p + delta, Score = 100(1 - clip(D, 0, 1)), where α > δ. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

The calculation for Kinematic Extremes involves joint angular and body-segment linear velocities examined against per-DoF and per-segment limits. p. 8
Where the paper says it

Body-segment linear velocities are obtained by Forward Kinematics (FK) on pre-defined segment Center-ofMass. We compare joint angular velocities and body-segment linear velocities against per-DoF and per-segment limits referenced from common biomechanics standards [62].

p. 8
Where the paper says it

Body-segment linear velocities are obtained by Forward Kinematics (FK) on pre-defined segment Center-ofMass. We compare joint angular velocities and body-segment linear velocities against per-DoF and per-segment limits referenced from common biomechanics standards [62].

The final score is again a weighted sum of (r, s, p) and is linearly mapped to the range [0, 100], where higher values indicate more natural motion. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

The calculation for Joint Range of Motion involves calculating the averages of bt and mt over all frames. p. 6
Where the paper says it

The metric designs in HumanScore follow the principle of such hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions (the base tier); hyperextension or interpenetration (the middle tier); and non-human spikes or jitter (the top tier). Kinetic consistency Kinematic correctness Anatomy correctness (v) Kinematic Extremes (vi) Motion Smoothness (iv )Self-collision(iii) Joint Range of Motion (i) Extra Limbs (ii) Bone Length Biomechanical Hierarchy Evaluation Dimensions F = ma Fig.

The final score is given by a weighted sum of the three aggregated quantities D = alpha r + beta s + gamma p, Score = 100(1 - D), where α=0. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

5, β=0.3, and γ=0.2 are hyper-parameters. The calculation for Motion Smoothness involves computing per-frame relative violation ratios for both acceleration and jerk. p. 8
Where the paper says it

Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits. Similar to the Kinematic Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum.

The final score is then computed across the entire sequence, where T is the total number of frames and Lmax is the length of the longest consecutive segment with bt=1. p. 7
Where the paper says it

Importantly, we modify the original fitting step by removing the global scale/bone-length rigidity constraint of the template skeleton, allowing bone lengths to vary freely across frames. This yields a per-frame sequence of fitted bone lengths, from which we compute, for each bone, the relative ℓ1 deviation from its median length over time.

The final score is again a weighted sum of (r, s, p) and linearly mapped to the range [0, 100], where higher values indicate smoother and more physically plausible motion. p. 7
Where the paper says it

When computing the metric, we first obtain the number of colliding faces at each frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the ratio of mild and severe collisions.

The calculation for Self-collision involves a weighted combination of three aspects: r severe, s, and p. The final metric value is obtained from a weighted combination of these three aspects D = alpha r severe + beta s + gamma p + delta, Score = 100(1 - clip(D, 0, 1)), where α > δ. The calculation for Kinematic Extremes involves joint angular and body-segment linear velocities examined against per-DoF and per-segment limits. The final score is again a weighted sum of (r, s, p) and is linearly mapped to the range [0, 100], where higher values indicate more natural motion.

Discussion and Findings

Top-performing models come from both the proprietary and open-source camps, with Seedance 1.0 Pro Fast [16] and HunyuanVideo 1. p. 9

Where the paper says it

According to the leaderboard reported in Table 1, Seedance 1. 0 Pro fast and HunyuanVideo 1.

5 [72] co-leading the overall leaderboard, followed closely by KlingAI 2. p. 9
Where the paper says it

5 co-lead the leaderboard overall (91. 1), followed by KlingAI 2.

5 Turbo Pro (90.8). These complementary profiles lead to different qualitative failure modes across models. p. 2
Where the paper says it

The benchmark extracts structured human motion from generated videos and scores it using a suite of anatomically- and biomechanicallygrounded metrics. By design, our evaluation is complementary to appearancecentric metrics and isolates failure modes that traditional measures overlook.

Models that generate highly dynamic motions often sacrifice anatomical correctness or kinematic smoothness, while more conservative models produce smoother but under-actuated motions. p. 1
Where the paper says it

1 Introduction Recent advances in generative modeling have led to rapid progress in visual content creation. When training on large-scale real-world data, scaled models are now able to generate compelling scenes with realistic appearance, lighting, and camera motion, and they score highly on prevailing video benchmarks such as VBench [23] and T2V-CompBench [61].

Model Robustness

The metrics are robust to the choice of pose estimation method, as observed across both alternative pipelines. The rankings remain consistent across different tolerance scales, demonstrating the robustness of our metrics. The ranking dynamics are also consistent across different hyperparameters. p. 3

Where the paper says it

The pipeline begins with curating a representative set of human motions from a large pool of common actions. For each motion, we carefully design prompts to mitigate model-specific biases and ensure consistent conditioning across generators.

Human Preference Alignment

HumanScore exhibits strong agreement with human preference overall, with Spearman correlation coefficients close to 1.0 across all evaluation dimensions. p. 6

Where the paper says it

This hierarchy reflects a strict dependency: samples that fail at a lower tier will inherently lead to errors in higher tiers. The metric designs in HumanScore follow the principle of such hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions (the base tier); hyperextension or interpenetration (the middle tier); and non-human spikes or jitter (the top tier).

This supports HumanScore as a reliable proxy for human assessment. p. 3
Where the paper says it

3 HumanScore Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components: (i) Curation of the list of motions that ask video generators to generate and to benchmark; (ii) Carefully designed prompts given motion types, for reliable

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Biomechanical Constraint Integration in Generation Models The improved AI system can be trained to avoid anatomically implausible poses and motion drift by incorporating HumanScore metrics directly into the loss function during training. This would allow models to generate videos that satisfy the three-tier hierarchy of biomechanical plausibility, ensuring that generated motions are not only visually realistic but also kinematically and kinetically correct.

  2. Prompt Engineering Guided by Human Preference Data The system can be updated to use human preference studies to refine prompts, moving beyond simple semantic alignment. This allows the generation model to produce motions that align with what humans perceive as natural, as HumanScore exhibits strong agreement with human preference overall.

  3. Multi-dimensional Motion Quality Ranking The AI system can be upgraded to generate a comprehensive HumanScore across all six dimensions (Anatomy correctness, Kinematic correctness, Kinetic correctness) rather than just pixel-level realism. This enables researchers to diagnose specific failure modes, such as temporal jitter or joint range of motion violations, providing a more fine-grained diagnosis of motion fidelity.

  4. Robust Pose Estimation for Metric Computation The system can be designed to use the state-of-the-art pose estimation methods like PromptHMR for metric calculation, ensuring robustness across different evaluation pipelines. This prevents inaccuracies, particularly in components such as human pose estimation from undermining the overall quality assessment of generated motion.

  5. Difficulty-Stratified Training Adaptation The generation model can be trained to perform better on complex motions by incorporating the difficulty stratification used in HumanScore curation. This ensures that models demonstrate improved performance across all motion complexities, as we observe a clear performance drop as motion difficulty increases.

Abstract

Recent advances in model architectures, compute, and data scale have driven rapid progress in video generation, producing increasingly realistic content. Yet, no prior method systematically measures how faithfully these systems render human bodies and motion dynamics. In this paper, we present HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos. HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency, enabling fine-grained diagnosis beyond visual realism alone. Through carefully designed prompts, we elicit a diverse set of movements at varying intensities and evaluate videos generated by thirteen state-of-the-art models. Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e.g., temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria.

Sources

Related papers