IMPose: Closed-Loop Correction Propagation for Interactive Multi-Person Pose Annotation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "IMPose: Closed-Loop Correction Propagation for Interactive Multi-Person Pose Annotation".
Jane: Here's the gist of the paper. High-quality dynamic human pose annotation equips AI with precise motion kinematics to enable human behavior mastery, yet remains labor-intensive and time-consuming The core contribution of IMPose lies in its unified architecture, which jointly enables temporal correction propagation and multi-person association This framework effectively preserves both pose trajectory continuity and identity consistency under challenging conditions such as occlusion and motion blur Pruning redundant reasoning tokens IMPose is introduced as an interactive tool for multi-person dynamic pose annotation that features a dual-level tracking mechanism propagating one-frame multi-person pose corrections from annotators across entire videos The keypoint-level ensures corrections temporal propagation via sequential modeling, while the instance-level employs keypoint-aware embedding with relative positional encoding to maintain multi-person cross-frame consistency To further improve robustness, IMPose maintains historical pose and instance cues in a trajectory bank which enhances long-range temporal association and stabilizes annotation in challenging cases such as occlusion and motion blur By converting sparse human corrections into dense and coherent pose trajectories, our framework significantly reduces repeated manual refinement across frames How the training signal is built The framework utilizes a multi-level tracking mechanism combining keypoint-level and instance-level tracking to disseminate user-provided pose corrections across entire video sequences The keypoint-level tracking facilitates the temporal propagation of corrections by performing sequential modeling and joint inference over multiple frames, thereby ensuring continuous trajectory of key points The instance-level tracking module incorporates a keypoint-aware embedding to alleviate the impact of occlusion and motion blur This module leverages relative positional encoding to model kinematic trajectories and fuse low-level visual cues with high-level postural semantics The instance-aware embedding couples identity embeddings with both box features and pose-aware geometric representations, enabling more reliable association in crowded scenes Where the gains come from The primary advantage of IMPose is its ability to jointly model temporal correction propagation and multiperson association, transforming sparse manual corrections into dense, temporally coherent, and identity-consistent pose trajectories Experiments demonstrate that IMPose achieves dynamic pose annotation with only 27 corrections for a video from the 3DPW dataset
12: , comprising an average of 1,050 frames with two individuals Furthermore, to address the sparse annotation of the original PoseTrack21 dataset, IMPose expands the dataset annotation by adding 187,920 new persons and 3,548,968 additional keypoints with just 10 annotators in only 10 hours The combination of Keypoint-aware Embedding (KP-AE) and Keypoint-level Temporal Correction Propagation (KTCP) achieves a synergistic improvement, increasing mAP by 29.37 and VNoC@95 by 7.51 Robustness analysis IMPose demonstrates superior corrected keypoint propagation robustness compared to CoTracker
11: under motion blur, maintaining predefined OKS thresholds for a higher average number of frames across both blurred and clear videos Specifically, under motion blur at OKS@75, IMPose successfully propagates the corrected keypoints for an average of 34.02 frames, significantly outperforming CoTracker’s 21.89 frames Moreover, IMPose consistently achieves higher recovery rates across all occlusion settings and OKS thresholds compared to CoTracker
11: under prolonged occlusion Quantitative performance IMPose consistently outperforms other methods across all interaction levels in the Video Number of Clicks (VNoC) metric, achieving the best performance–interaction trade-off across different click budgets At VNoC@95, IMPose reduces manual effort by 3.11 and 4.14 corrections on the 3DPW
12: and PoseTrack21
13: datasets compared to AlphaPose
7: Notably, with only one user correction, IMPose improves mAP from 38.87 to 67.99, yielding a gain of 29.12 points over X-AnyLabeling
1: Time cost under real manual corrections IMPose achieves the lowest annotation time on both datasets, demonstrating its clear advantage in annotation efficiency In Table IV, IMPose achieves speedups of approximately 4 and 3 times compared to AlphaPose and Click-Pose, respectively This reduction in time cost confirms the practical value of IMPose for scaling high-quality pose annotation to longer videos and more crowded real-world scenes This paper presents IMPose, an interactive framework for video-based multi-person pose annotation, built upon a novel dual-level tracking mechanism The core contribution of IMPose lies in its unified architecture, which jointly enables temporal correction propagation and multi-person association By integrating keypoint-level tracking for temporal correction propagation with instance-level tracking based on keypointaware embedding, IMPose effectively preserves both pose trajectory continuity and identity consistency under challenging conditions such as occlusion and motion blur Extensive experiments demonstrate that IMPose achieves highly efficient multi-person pose annotation with minimal manual effort Acknowledgments This work was supported in part by National Key R&D Program of China (2023YFC3082100), National Natural Science Foundation of China (62501416), Science Fund for Distinguished Young Scholars of Tianjin (No.
Paper summary: Tom: So, wrapping up this look at IMPose: Closed-Loop Correction Propagation for Interactive Multi-Person Pose Annotation. The title itself really captures the essence of what they're doing here—it's about a closed loop where corrections propagate back and forth.
Jane: They are essentially proposing an interactive framework that uses dual-level tracking to solve the intertwined problems of temporal correction propagation and multi-person association simultaneously.
Lu: The core contribution, as the authors state, is this unified architecture that preserves both pose trajectory continuity and identity consistency even when facing things like occlusion or motion blur.
Meng: For me, the implication is that if we can get models to reliably propagate these corrections across frames so effectively, it lowers the barrier for acquiring massive amounts of high-quality dynamic human pose data without requiring endless manual rework.
Lalam: It moves the focus from tedious frame-by-frame refinement to building a system that establishes a coherent whole trajectory for each person.
Tom: The authors show strong quantitative evidence, like achieving twenty seven corrections on the three DPW dataset with an average of one thousand fifty frames, which is a significant reduction in manual effort compared to previous work.
Jane: And they also expanded their dataset annotation substantially—adding over three million keypoints using just ten annotators in ten hours—which speaks to the practical utility for scaling up data collection.
Lu: It suggests that future work should focus on testing this framework in more diverse, out-of-distribution real-world scenarios where the initial corrections might be less accurate.
Meng: And from an engineering deployment view, we still need to nail down the computational cost of maintaining that trajectory bank and ensuring low latency when processing long sequences in a production environment.
Lalam: But what’s certain is that IMPose provides a way to achieve highly efficient multi-person pose annotation with minimal manual effort based on the mechanism they've described here.
Conclusion: Tom: So, IMPose is basically an interactive tool for annotating dynamic human poses in videos where people are moving around together.
Jane: It sounds like they're trying to fix that big headache of getting both the correct pose for each person and making sure they stay consistent across frames, especially when things get blurry or messy.
Lu: The core idea is this unified architecture that handles two main things at once: propagating corrections over time and keeping track of different people in a scene.
Meng: So it’s not just one method for tracking, but a system where you can correct the pose of one person, and that correction ripples through the whole video sequence.
Lalam: Exactly, it uses keypoint-level tracking to send those corrections forward sequentially, while also using instance-level tracking with embeddings to keep track of who is who.
Tom: The big win they're showing is how this combination turns those sparse manual corrections into these dense, coherent pose trajectories automatically.
Jane: They mention they only needed twenty seven corrections for one video from the three DPW dataset, which is a lot less than you'd expect for that kind of detail.
Lu: And on the PoseTrack21 dataset, they expanded it by adding almost four million keypoints using just ten annotators in ten hours. That’s a huge lift for data collection.
Meng: From a deployment standpoint, that efficiency is impressive because it means we don't have to pay annotators to fix every single frame manually anymore.
Lalam: It really shows how building this closed loop system helps stabilize the data collection process for complex scenes with multiple people.
Tom: So, it’s about making the annotation process much faster and more reliable for handling those crowded real-world videos we deal with in production.
Jane: It feels like they're moving toward a system where you just guide the AI with a few starting points and it figures out the rest of the motion.
Lalam: And that’s what makes it different; it’s not just about tracking; it’s about using those corrections to build a complete, consistent story of movement for every person in the video.
Tom: So, this paper lays out how to make high-quality multi-person pose annotation much more efficient and robust by tying temporal correction propagation directly into multi-person association.
Jane: It definitely moves the goalposts for how we think about getting accurate motion data from video.
Lu: The next thing we need to look at is how this framework performs when we take it out of the controlled environment and into messy, real-world footage with really poor lighting or heavy motion blur.
Haoyang Ge, Jian Ma, Ziwen Wang, Qihe Wang, Jianqi Fan, Hongzhi Yu, Xingyu Chen, Kun Li
Tianjin University · Zhongguancun Academy
cs.CV, cs.HC
Submitted: 2026-06-03
Updated: 2026-10-31
Code: https://github.com/CVHub520/X-AnyLabeling
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 83/100
The gist: High-quality dynamic human pose annotation equips AI with precise motion kinematics to enable human behavior mastery…
Key concepts
- Keypoint-level Tracking
- This mechanism focuses on tracking individual body joints (keypoints) sequentially across frames. It ensures that when an annotator corrects a pose in one frame, those corrections are smoothly propagated to the next frame by modeling the movement over time. This is crucial for maintaining a continuous and accurate trajectory of each person's body parts throughout the video.
- Instance-level Tracking with Keypoint-aware Embedding
- This part tracks entire people (instances) using embeddings that are aware of their keypoints. By incorporating relative positional encoding, it helps maintain the identity of each person even when they are partially hidden by occlusion or when motion blur makes individual keypoints hard to see. This allows the system to reliably associate corrections with the correct person across different frames.
- Temporal Correction Propagation
- This is the core function where user-provided pose adjustments are spread throughout a video sequence. Instead of just fixing one frame, IMPose uses sequential modeling to ensure that a correction made at time T is accurately applied to the pose estimation at time T+1, T+2, and so on. This creates dense and temporally coherent motion trajectories.
- Trajectory Bank
- This is a memory system that stores historical pose and instance cues. By keeping this bank, IMPose can look back at past information to better associate current corrections with previous states. This enhances long-range temporal association, making the annotation process more stable and robust when dealing with difficult visual conditions like prolonged occlusion or blur.
Terminology
Summary
High-quality dynamic human pose annotation equips AI with precise motion kinematics to enable human behavior mastery, yet remains labor-intensive and time-consuming The core contribution of IMPose lies in its unified architecture, which jointly enables temporal correction propagation and multi-person association This framework effectively preserves both pose trajectory continuity and identity consistency under challenging conditions such as occlusion and motion blur
Pruning redundant reasoning tokens
IMPose is introduced as an interactive tool for multi-person dynamic pose annotation that features a dual-level tracking mechanism propagating one-frame multi-person pose corrections from annotators across entire videos The keypoint-level ensures corrections temporal propagation via sequential modeling, while the instance-level employs keypoint-aware embedding with relative positional encoding to maintain multi-person cross-frame consistency To further improve robustness, IMPose maintains historical pose and instance cues in a trajectory bank which enhances long-range temporal association and stabilizes annotation in challenging cases such as occlusion and motion blur By converting sparse human corrections into dense and coherent pose trajectories, our framework significantly reduces repeated manual refinement across frames
How the training signal is built
The framework utilizes a multi-level tracking mechanism combining keypoint-level and instance-level tracking to disseminate user-provided pose corrections across entire video sequences The keypoint-level tracking facilitates the temporal propagation of corrections by performing sequential modeling and joint inference over multiple frames, thereby ensuring continuous trajectory of key points The instance-level tracking module incorporates a keypoint-aware embedding to alleviate the impact of occlusion and motion blur This module leverages relative positional encoding to model kinematic trajectories and fuse low-level visual cues with high-level postural semantics The instance-aware embedding couples identity embeddings with both box features and pose-aware geometric representations, enabling more reliable association in crowded scenes
Where the gains come from
The primary advantage of IMPose is its ability to jointly model temporal correction propagation and multiperson association, transforming sparse manual corrections into dense, temporally coherent, and identity-consistent pose trajectories Experiments demonstrate that IMPose achieves dynamic pose annotation with only 27 corrections for a video from the 3DPW dataset [12], comprising an average of 1,050 frames with two individuals Furthermore, to address the sparse annotation of the original PoseTrack21 dataset, IMPose expands the dataset annotation by adding 187,920 new persons and 3,548,968 additional keypoints with just 10 annotators in only 10 hours The combination of Keypoint-aware Embedding (KP-AE) and Keypoint-level Temporal Correction Propagation (KTCP) achieves a synergistic improvement, increasing mAP by 29.37 and VNoC@95 by 7. p. 551
Robustness analysis
IMPose demonstrates superior corrected keypoint propagation robustness compared to CoTracker [11] under motion blur, maintaining predefined OKS thresholds for a higher average number of frames across both blurred and clear videos Specifically, under motion blur at OKS@75, IMPose successfully propagates the corrected keypoints for an average of 34.02 frames, significantly outperforming CoTracker’s 21. p. 989 frames Moreover, IMPose consistently achieves higher recovery rates across all occlusion settings and OKS thresholds compared to CoTracker [11] under prolonged occlusion
Quantitative performance
IMPose consistently outperforms other methods across all interaction levels in the Video Number of Clicks (VNoC) metric, achieving the best performance–interaction trade-off across different click budgets At VNoC@95, IMPose reduces manual effort by 3.11 and 4.14 corrections on the 3DPW [12] and PoseTrack21 [13] datasets compared to AlphaPose [7] Notably, with only one user correction, IMPose improves mAP from 38. p. 787 to 67.99, yielding a gain of 29. p. 712 points over X-AnyLabeling [1]
Time cost under real manual corrections
IMPose achieves the lowest annotation time on both datasets, demonstrating its clear advantage in annotation efficiency In Table IV, IMPose achieves speedups of approximately 4 and 3 times compared to AlphaPose and Click-Pose, respectively This reduction in time cost confirms the practical value of IMPose for scaling high-quality pose annotation to longer videos and more crowded real-world scenes
Conclusion
This paper presents IMPose, an interactive framework for video-based multi-person pose annotation, built upon a novel dual-level tracking mechanism The core contribution of IMPose lies in its unified architecture, which jointly enables temporal correction propagation and multi-person association By integrating keypoint-level tracking for temporal correction propagation with instance-level tracking based on keypointaware embedding, IMPose effectively preserves both pose trajectory continuity and identity consistency under challenging conditions such as occlusion and motion blur Extensive experiments demonstrate that IMPose achieves highly efficient multi-person pose annotation with minimal manual effort
Acknowledgments
This work was supported in part by National Key R&D Program of China (2023YFC3082100), National Natural Science Foundation of China (62501416), Science Fund for Distinguished Young Scholars of Tianjin (No.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Multi-level tracking mechanism for dynamic correction propagation allows for
keypoint-level tracking facilitates the temporal propagation of corrections by performing sequential modeling and joint inference over multiple frames, thereby ensuring continuous trajectory of key points.
This enables corrected poses to besmoothly transferred to neighboring frames
while maintainingpose trajectory continuity.
-
Instance-level tracking with a keypoint aware embedding maintains
correct multi-person keypoint assignment under challenging conditions like occlusion and motion blur
by encoding pose geometry through features like relative position features and Fourier features. This preventstemporal inconsistencies and identity swaps which otherwise arise when visual information is degraded.
-
Joint modeling of temporal correction propagation and multi-person association allows the system to
jointly model spatio-temporal multi-person pose correction,
convertingsparse manual corrections into dense, temporally coherent, and identityconsistent pose trajectories across video frames.
This significantly reduces repeated manual refinement. -
The framework can achieve high precision annotation with high efficiency by demonstrating that it requires only
27 clicks per 1,050 frame video on 3DPW
or3 clicks per tracklet per 84-frame on PoseTrack21.
-
The system can be used to re-annotate datasets, as demonstrated by achieving
over twice the number of keypoints with 100 person-hours versus the original 16,000 person-hours
for PoseTrack21.
Sources
- CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos
- SAM 2: Segment Anything in Images and Videos
- BoT-SORT: Robust Associations Multi-Pedestrian Tracking
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
- Bridging the Gap Between End-to-end and Non-End-to-end Multi-Object Tracking
Related papers
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs
- From Formulas to Protocols: A Compositional View of Binary Target Segmentation Metrics
- Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment