HumanScore: Benchmarking Human Motions in Generated Videos
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "HumanScore: Benchmarking Human Motions in Generated Videos".
Jane: Here's the gist of the paper. HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos This benchmark extracts structured human motion from generated videos and scores it using a suite of anatomically- and biomechanically grounded metrics Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e.g., temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria.
Tom: Evaluation Framework HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency The evaluation framework follows a three-tier hierarchy: the base tier enforces anatomical correctness (valid human topology with stable body proportions), the middle tier enforces kinematic correctness (geometric feasibility of poses), and the top tier enforces kinetic correctness (temporal coherence and physically plausible dynamics).
Jane: The metrics are designed to follow this hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions, hyperextension or interpenetration, non-human spikes or jitter, joint range of motion, self-collision, and kinematic extremes.
Tom: Metric Design Details The metrics are calculated using a generic framework where per-frame calculations produce three outputs: bt – a binary indicator that flags whether a frame is normal or abnormal, mt – a relative metric score that measures how “bad” a frame is, in the range (0, 1), and Lmax – the length of the longest consecutive abnormal (according to bt) frame segment.
Paper summary: Tom: To break down the paper, they define six specific metrics that cover kinematic plausibility, temporal stability, and biomechanical consistency. They use a three-tier hierarchy where you check anatomical correctness first, then geometric feasibility for poses, and finally kinetic correctness for how the motion actually flows over time.
Jane: That hierarchy makes sense because it starts with the basics—does the body even have a stable shape to begin with?
Lu: Yes, the base tier focuses on anatomical correctness, meaning checking if you have valid human topology with stable body proportions across all frames.
Meng: And then they move up to kinematic correctness, which is checking if the poses are geometrically feasible—like can a limb actually reach that position without breaking physics?
Tom: Then there's the top tier, kinetic correctness, which is about temporal coherence and physically plausible dynamics over time. That’s where you check for things like motion drift or sudden jolts in the action.
Lalam: The metrics are calculated frame by frame using a generic system that outputs three values: bt, mt, and Lmax.
Jane: So bt is a binary indicator flagging if a frame is normal or abnormal, mt is a relative metric score from zero to one measuring how bad that frame is, and Lmax tracks the longest streak of abnormal frames.
Tom: And then they combine these into one final video-level metric using a weighted sum: D equals alpha r plus beta s plus gamma p, and the final score is one hundred minus D, with those weights being zero point five for r, zero point three for s, and zero point two for p <ref:2604.20157#pg2>.
Lu: Specifically they use metrics like Extra Limbs to detect anomalous limbs using HADM to find abnormal limbs, Bone Length by fitting OpenSim skeletons to see the relative length deviation from the median length over time.
Meng: And they also check self-collision using a bounding volume hierarchy and fast triangle-triangle intersection tests, which is pretty intense computation for frame-level checking.
Jane: They also have checks for joint range of motion against anatomical limits, kinematic extremes by looking at joint angular and body segment linear velocities against biomechanics standards, and motion smoothness by comparing angular acceleration and jerk to per degree of freedom limits.
Conclusion: Tom: So, to wrap up the "HumanScore: Benchmarking Human Motions in Generated Videos" paper, the core message is that these six metrics provide a principled, quantitative evaluation of motion realism from a biomechanical perspective.
Jane: They show that when you apply this benchmark to state-of-the-art video generators—like Sora two point two and Veo three point one fast—you find consistent weaknesses in motion fidelity despite their visual appeal.
Lu: The paper's contributions are providing the first systematic evaluation framework centered on human motion in generated videos, proposing quantitative indicators grounded in anatomy and biomechanics with transparent baselines, and applying it to show where the gaps between visual appearance and physical realism are.
Meng: What this means for deployment is that even if a model looks great visually, you still have to check these physical plausibility scores before you trust it for anything important like sports commentary or training simulations.
Lalam: The authors found that the correlation between their HumanScore metric values and human preference win ratio shows strong agreement across all six evaluation dimensions, with Spearman correlation coefficients close to one point zero.
Tom: That strong agreement is significant because it means the numbers they calculate actually track what people perceive as good motion very well.
Jane: So, in simple terms, this paper gives us a reliable tool to distinguish real human movement from synthetic movement in AI videos. It shifts the focus from just looking good to being physically believable.
Stanford University · Peking University
cs.CV
Submitted: 2026-04-22
Updated: 2026-11-03
Project page: https://cs.stanford.edu/~xtiange
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 95/100
The gist: HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore…
Key concepts
- Evaluation Framework
- A three-tier system for judging motion quality: base tier checks anatomical correctness (correct body structure), middle tier checks kinematic correctness (feasible poses), and the top tier checks kinetic correctness (smooth, physically plausible movement).
- Metric Design Details
- Calculations use per-frame outputs like bt (normal/abnormal flag) and mt (severity score) to aggregate into video metrics. The final score is a weighted sum of three components—r, s, and p—which are mapped to a 100-point scale representing quality.
- Kinematic Extremes
- This metric evaluates how natural the motion is by checking joint angular and body-segment linear velocities against established biomechanical standards. High scores indicate more natural and physically plausible movement patterns.
- Human Preference Alignment
- The HumanScore metrics strongly correlate with human preference ratings, showing high Spearman correlation coefficients. This validates the framework as a reliable way to measure how well generated human motion aligns with what humans perceive as realistic.
Terminology
Summary
HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye The paper presents HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos This benchmark extracts structured human motion from generated videos and scores it using a suite of anatomically- and biomechanically grounded metrics Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e. p. 0 Through carefully designed prompts,
we elicit a diverse set of movements at varying intensities and evaluate
videos generated by thirteen state-of-the-art models. Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e. Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e. , temporal jitter,
anatomically implausible poses, and motion drift), and produces robust
model rankings from quantitative and physically meaningful criteria.Where the paper says it
Where the paper says it
Evaluation Framework
HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency The evaluation framework follows a three-tier hierarchy: the base tier enforces anatomical correctness (valid human topology with stable body proportions), the middle tier enforces kinematic correctness (geometric feasibility of poses), and the top tier enforces kinetic correctness (temporal coherence and physically plausible dynamics). The metrics are designed to follow this hierarchy at six evaluation dimensions: spurious limbs or inconsistent proportions, hyperextension or interpenetration, non-human spikes or jitter, joint range of motion, self-collision, and kinematic extremes. p. 6 The metric designs in
HumanScore follow the principle of such hierarchy at six evaluation dimensions:
spurious limbs or inconsistent proportions (the base tier); hyperextension or
interpenetration (the middle tier); and non-human spikes or jitter (the top tier). Kinetic
consistency
Kinematic
correctness
Anatomy
correctness
(v) Kinematic Extremes (vi) Motion Smoothness
(iv )Self-collision(iii) Joint Range of Motion
(i) Extra Limbs (ii) Bone Length
Biomechanical Hierarchy Evaluation Dimensions
F = ma
Fig.Where the paper says it
Metric Design Details
The metrics are calculated using a generic framework where per-frame calculations produce three outputs: bt – a binary indicator that flags whether a frame is normal or abnormal, mt – a relative metric score that measures how “bad” a frame is, in the range (0, 1), and Lmax – the length of the longest consecutive abnormal (according to bt) frame segment. The final video-level metric is given by a weighted sum of these three aggregated scores: D = alpha r + beta s + gamma p, Score = 100(1 - D), where alpha=0. p. 8 Similar to the Kinematic
Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum. For all the metrics computed above, we normalize their scores to a scale from 0
to 100. Specifically, we use HADM [69], a recent
model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates,
we add temporal-consistency checks and tune its parameters to better capture
dynamic ghosting. Specifically, we use HADM [69], a recent
model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates,
we add temporal-consistency checks and tune its parameters to better capture
dynamic ghosting. Specifically, we use HADM [69], a recent
model trained on human images to detect anomalous limbs. Because the original model is better at detecting clear extra limbs than motion-blur duplicates,
we add temporal-consistency checks and tune its parameters to better capture
dynamic ghosting. Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits. Similar to the Kinematic
Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Experimental Results
The experiments benchmarked thirteen video generation models, including four open-source systems—Wan 2.2 [68], CogVideoX-5B [74], HunyuanVideo 1. p. 8 1 Evaluate Video Generation Models
We benchmark thirteen video generation models, including four open-source systems—Wan 2. 2 [68], CogVideoX-5B [74], HunyuanVideo 1. 2 [68], CogVideoX-5B [74], HunyuanVideo 1. 5 [72], and Kandinsky
5. 5 [72], and Kandinsky
5. 0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI
2. 0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI
2. 1 fast [38], KlingAI
2. 5 Turbo Pro [2], Seedance 1. 1 fast [38], KlingAI
2. 5 Turbo Pro [2], Seedance 1. 5 Turbo Pro [2], Seedance 1. 0 Pro fast [16], Hailuo 02 [45], Pika v2. Higher scores indicate better performance. The best score in each dimension is highlighted in cell colors. According to the leaderboard reported in Table 1, Seedance 1. 0 Pro fast and
HunyuanVideo 1. 0 Pro fast and
HunyuanVideo 1. 5 co-lead the leaderboard overall (91. 5 co-lead the leaderboard overall (91. 1), followed by KlingAI
2. 1 fast [38], KlingAI
2. 5 Turbo Pro [2], Seedance 1. 1 fast [38], KlingAI
2. 5 Turbo Pro [2], Seedance 1. We also include real videos
as an upper-bound reference; as expected, they score highest overall (94. The
gap between real and generated videos indicates that current generators still
struggle with biomechanics-constrained motion despite strong visual realism.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Validation and Conclusions
The correlation between HumanScore metric values and human preference win ratio shows strong agreement across all evaluation dimensions, with Spearman correlation coefficients close to 1.0. Furthermore, the average score on real-world videos was 94. p. 1 1 Introduction
Recent advances in generative modeling have led to rapid progress in visual
content creation. When training on large-scale real-world data, scaled models
are now able to generate compelling scenes with realistic appearance, lighting,
and camera motion, and they score highly on prevailing video benchmarks such
as VBench [23] and T2V-CompBench [61]. Yet, despite these advances, the realism of human motion
remains a distinguishing signal between real and generated content. This convergence—pose estimators achieving stable performance and generative models
saturating at standard visual metrics—makes it timely and necessary to push for Yet, despite these advances, the realism of human motion
remains a distinguishing signal between real and generated content. This convergence—pose estimators achieving stable performance and generative models
saturating at standard visual metrics—makes it timely and necessary to push for In
this paper, we present HumanScore, a systematic framework to evaluate
the quality of human motions in AI-generated videos. HumanScore
defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency, enabling fine-grained diagnosis beyond visual realism alone.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Limitations and Future Directions
A limitation is that HumanScore relies on external models to compute its metrics, which can introduce inaccuracies, particularly in components such as human pose estimation. Moreover, even state-of-the-art fitting methods remain fundamentally imperfect and cannot perfectly recover all human motions from monocular video. p. 7 Detecting interpenetration requires accurate 3D modeling of human geometry
from videos. We therefore rely on the state-of-the-art monocular fitting method
PromptHMR [70] to obtain the best possible 3D SMPL-X meshes. The refined prompts are then passed to both proprietary and open-source state-of-the-art video generation models. Human verification is
incorporated at all stages for quality check. When training on large-scale real-world data, scaled models
are now able to generate compelling scenes with realistic appearance, lighting,
and camera motion, and they score highly on prevailing video benchmarks such
as VBench [23] and T2V-CompBench [61]. Yet, despite these visual improvements, people can still distinguish generated content from real ones based on
a key signal: human motion. 2 Human Motion Evaluation Methods
Motion-focused evaluation in videos often emphasizes perceptual alignment or
distributional similarity rather than explicit biomechanical constraints. As also
seen in Video-Bench [21], such metrics include action consistency, motion quality,
and temporal consistency; perception-driven motion metrics (PMM) from VMBench [34, 35]; recognizer-based Action-Score [28, 32, 33, 71]; and distributionalWhere the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Full Prompt Set
In this benchmark, we include 51 unique motion types, each paired with gentle and intense prompt variants, for a total of 102 prompts. The motions are balanced across three difficulty levels: 17 easy, 17 medium, and 17 hard. p. 5 This quality control step uses both objective measures (using the metrics
proposed in this work) and manual subjective inspection. The final result is
a curated set of 51 distinct motion types, balanced across three difficulty levels—simple, medium, and hard—with 17 motion types per level. 3 HumanScore
Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components:
(i) Curation of the list of motions that ask video generators to generate and
to benchmark; (ii) Carefully designed prompts given motion types, for reliable The pipeline begins with curating a representative set of human motions from a large pool of common actions. For each motion,
we carefully design prompts to mitigate model-specific biases and ensure consistent
conditioning across generators. ’ Finally, to
(iv) ensure a single, focused subject, we explicitly enforce ‘A single person’. Combining these standardized elements with attributes for the motion itself,
our final prompts entail five components: Scene, Motion, Intensity, Description, and Camera.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Video Generator Descriptions
The evaluation includes models such as Sora-2 [46], Veo 3.1 fast [38], KlingAI 2.5 Turbo Pro [2], and Seedance 1. p. 8 1 fast [38], KlingAI
2. 5 Turbo Pro [2], Seedance 1. 0 pro [5]—and nine proprietary systems: Sora-2 [46], Veo 3. 1 fast [38], KlingAI
2. 4 Experiments and Main Results
4. 1 Evaluate Video Generation Models
We benchmark thirteen video generation models, including four open-source systems—Wan 2. HumanScore is designed to automatically detect subtle biomechanical violations produced by AI video generators, even when they are imperceptible to the human eye. Recent advances in model architectures, compute, and data
scale have driven rapid progress in video generation, producing increasingly realistic content. HumanScore 3
principled, quantitative evaluation of motion realism from a biomechanical perspective. See Figure 1 and challenge yourself to distinguish between real videos
and those generated by the latest video generators.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Detailed Metric Calculations
The calculation for Extra Limbs involves a frame-level calculation where b t is a binary indicator that shows whether the frame has a strong enough detection of anomalous limbs, and mt is the per-frame severity score in the range [0, 1]. The video-level aggregation seeks to obtain a single numeric value as a unified score that represents the entire video from the three aspects: frequency, severity, and persistence. p. 3 3 HumanScore
Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components:
(i) Curation of the list of motions that ask video generators to generate and
to benchmark; (ii) Carefully designed prompts given motion types, for reliable We run HADM on each frame and aggregate the predicted
limb-detection confidence scores across the video to compute the average metric. (II) Bone length. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. Jerk is accumulated as local energy over a short
temporal window. Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. Body-segment linear velocities
are obtained by Forward Kinematics (FK) on pre-defined segment Center-ofMass. We compare joint angular velocities and body-segment linear velocities
against per-DoF and per-segment limits referenced from common biomechanics
standards [62]. Body-segment linear velocities
are obtained by Forward Kinematics (FK) on pre-defined segment Center-ofMass. We compare joint angular velocities and body-segment linear velocities
against per-DoF and per-segment limits referenced from common biomechanics
standards [62]. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. The metric designs in
HumanScore follow the principle of such hierarchy at six evaluation dimensions:
spurious limbs or inconsistent proportions (the base tier); hyperextension or
interpenetration (the middle tier); and non-human spikes or jitter (the top tier). Kinetic
consistency
Kinematic
correctness
Anatomy
correctness
(v) Kinematic Extremes (vi) Motion Smoothness
(iv )Self-collision(iii) Joint Range of Motion
(i) Extra Limbs (ii) Bone Length
Biomechanical Hierarchy Evaluation Dimensions
F = ma
Fig. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions. Following biomechanics standards [18], we compare both angular acceleration and jerk against per-DoF limits. Similar to the Kinematic
Extremes metric, the violation proportions relative to these limits are first normalized and then aggregated for each joint using a weighted sum. Importantly, we modify the original fitting step by removing the global
scale/bone-length rigidity constraint of the template skeleton, allowing bone
lengths to vary freely across frames. This yields a per-frame sequence of fitted
bone lengths, from which we compute, for each bone, the relative ℓ1 deviation
from its median length over time. When computing the metric, we first obtain the number of colliding faces at each
frame and determine thresholds to cater for mild self-collision and severe selfcollision cases separately. The final score is obtained by a weighted sum of the
ratio of mild and severe collisions.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Discussion and Findings
Top-performing models come from both the proprietary and open-source camps, with Seedance 1.0 Pro Fast [16] and HunyuanVideo 1. p. 9 According to the leaderboard reported in Table 1, Seedance 1. 0 Pro fast and
HunyuanVideo 1. 5 co-lead the leaderboard overall (91. 1), followed by KlingAI
2. The benchmark extracts structured human motion from
generated videos and scores it using a suite of anatomically- and biomechanicallygrounded metrics. By design, our evaluation is complementary to appearancecentric metrics and isolates failure modes that traditional measures overlook. 1 Introduction
Recent advances in generative modeling have led to rapid progress in visual
content creation. When training on large-scale real-world data, scaled models
are now able to generate compelling scenes with realistic appearance, lighting,
and camera motion, and they score highly on prevailing video benchmarks such
as VBench [23] and T2V-CompBench [61].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Model Robustness
The metrics are robust to the choice of pose estimation method, as observed across both alternative pipelines. The rankings remain consistent across different tolerance scales, demonstrating the robustness of our metrics. The ranking dynamics are also consistent across different hyperparameters. p. 3 The pipeline begins with curating a representative set of human motions from a large pool of common actions. For each motion,
we carefully design prompts to mitigate model-specific biases and ensure consistent
conditioning across generators.Where the paper says it
Human Preference Alignment
HumanScore exhibits strong agreement with human preference overall, with Spearman correlation coefficients close to 1.0 across all evaluation dimensions. p. 6 This hierarchy reflects a strict dependency: samples that fail
at a lower tier will inherently lead to errors in higher tiers. The metric designs in
HumanScore follow the principle of such hierarchy at six evaluation dimensions:
spurious limbs or inconsistent proportions (the base tier); hyperextension or
interpenetration (the middle tier); and non-human spikes or jitter (the top tier). 3 HumanScore
Evaluating single-person motions systematically entails a sequence of components. As outlined in Figure 3, HumanScore contains three key components:
(i) Curation of the list of motions that ask video generators to generate and
to benchmark; (ii) Carefully designed prompts given motion types, for reliableWhere the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Biomechanical Constraint Integration in Generation Models The improved AI system can be trained to avoid
anatomically implausible poses
andmotion drift
by incorporating HumanScore metrics directly into the loss function during training. This would allow models to generate videos that satisfy thethree-tier hierarchy
of biomechanical plausibility, ensuring that generated motions are not only visually realistic but also kinematically and kinetically correct. -
Prompt Engineering Guided by Human Preference Data The system can be updated to use human preference studies to refine prompts, moving beyond simple semantic alignment. This allows the generation model to produce motions that align with what humans perceive as natural, as
HumanScore exhibits strong agreement with human preference overall.
-
Multi-dimensional Motion Quality Ranking The AI system can be upgraded to generate a comprehensive HumanScore across all six dimensions (Anatomy correctness, Kinematic correctness, Kinetic correctness) rather than just pixel-level realism. This enables researchers to diagnose specific failure modes, such as
temporal jitter
orjoint range of motion
violations, providing a more fine-grained diagnosis of motion fidelity. -
Robust Pose Estimation for Metric Computation The system can be designed to use the state-of-the-art pose estimation methods like PromptHMR for metric calculation, ensuring robustness across different evaluation pipelines. This prevents
inaccuracies, particularly in components such as human pose estimation
from undermining the overall quality assessment of generated motion. -
Difficulty-Stratified Training Adaptation The generation model can be trained to perform better on complex motions by incorporating the difficulty stratification used in HumanScore curation. This ensures that models demonstrate improved performance across all motion complexities, as
we observe a clear performance drop as motion difficulty increases.
Abstract
Recent advances in model architectures, compute, and data scale have driven rapid progress in video generation, producing increasingly realistic content. Yet, no prior method systematically measures how faithfully these systems render human bodies and motion dynamics. In this paper, we present HumanScore, a systematic framework to evaluate the quality of human motions in AI-generated videos. HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency, enabling fine-grained diagnosis beyond visual realism alone. Through carefully designed prompts, we elicit a diverse set of movements at varying intensities and evaluate videos generated by thirteen state-of-the-art models. Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity, identifies recurrent failure modes (e.g., temporal jitter, anatomically implausible poses, and motion drift), and produces robust model rankings from quantitative and physically meaningful criteria.
Sources
- Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation
- A Short Note on the Kinetics-700 Human Action Dataset
- WorldScore: A Unified Evaluation Benchmark for World Generation
- Seedance 1.0: Exploring the Boundaries of Video Generation Models
- VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models
- STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models
- GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
- The Quest for Generalizable Motion Generation: Data, Model, and Evaluation
- Towards Understanding Camera Motions in Any Video
- Evaluating Text-to-Visual Generation with Image-to-Text Generation
- Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos
- Evaluating the Quality of a Synthesized Motion with the Fr'echet Motion Distance
- Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld Smartphone
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- Towards Accurate Generative Models of Video: A New Metric & Challenges
- Wan: Open and Advanced Large-Scale Video Generative Models
- Detecting Human Artifacts from Text-to-Image Models
- HunyuanVideo 1.5 Technical Report
- CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs