From Keypoints to Predictive Distributions: Post-Hoc Uncertainty for YOLO-Pose Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "From Keypoints to Predictive Distributions".
Jane: Here's the gist of the paper. YOLO-Pose models offer efficient keypoint localization but lack quantification of associated spatial uncertainty.
Tom: The authors introduce a lightweight, post-hoc probabilistic extension that augments a trained YOLO-Pose model by incorporating calibrated bivariate predictive distributions over keypoint locations, centered around the model's original predictions.
Jane: This is achieved by training auxiliary probabilistic heads to predict an input-dependent 2 times 2 dispersion matrix for each keypoint, followed by Gaussian or Student-t calibration to ensure distributional fidelity.
Tom: The proposed evaluation protocol combines distributional calibration diagnostics with average keypoint precision (AKP).
Paper summary: Tom: So, looking at the paper "From Keypoints to Predictive Distributions: Post-Hoc Uncertainty for YOLO-Pose Models," the main thrust is taking a standard keypoint detector and giving it a calibrated measure of its uncertainty.
Jane: It moves beyond just finding where things are to telling you how confident the model is in those locations, which is vital for safety.
Lu: The authors propose this lightweight post-hoc probabilistic extension that adds heads to predict a two by two dispersion matrix for each keypoint, followed by calibration methods like Gaussian or Student-t.
Meng: It’s about making the model smarter about its own limitations instead of just trusting its raw output, which is something I need to think about when deploying models at scale.
Lalam: They demonstrate this on vision-based aircraft landing using calibrated covariances that feed into an extended kalman filter for sensor fusion, giving a runway-relative position estimate and its covariance <ref:2607.26921#pg3>.
Tom: The implication is that you can prune unreliable keypoints before your main task, which sounds like it could reduce downstream processing errors significantly <ref:2607.26921#pg3>.
Jane: It’s about giving the system a way to handle situations where the model isn't fully confident, instead of just outputting a guess.
Lu: The title itself is good because it moves from just keypoints to predictive distributions, which is a richer representation of what the model actually knows.
Tom: Overall, this work shows that adding uncertainty quantification post-hoc can be effective for vision tasks where precision and reliability are non-negotiable <ref:2607.26921#pg1>.
Conclusion: Tom: So we've talked about how YOLO-Pose models give us keypoints, and now this paper is adding something extra: uncertainty quantification.
Jane: Right, so what's the big picture here? This paper takes a model that just says "this pixel is an eye" and it figures out how sure it really is about that eye.
Lu: They introduce a lightweight way to do this post-hoc, they train some auxiliary heads to predict these two by two dispersion matrices for each keypoint.
Meng: A dispersion matrix? So instead of just one number saying the error is big or small, you're getting a whole table of how the errors are spread out in every direction.
Lalam: They then calibrate those predictions using Gaussian or Student-t methods to make sure the predicted uncertainty actually matches what's happening in the real data.
Tom: And they test this by ranking keypoints based on this new uncertainty estimate, which helps you decide which ones to trust and which ones to ignore.
Jane: That sounds really useful for deployment because if you know exactly where the model is unsure, you can build safety checks around it.
Lu: They show that Student-t calibration is actually pretty good at capturing the real errors seen in their models, especially those with heavy tails, which is a common issue.
Meng: But what about making this run fast? If we add these heads and calibrations on top of the existing YOLO-Pose model, how much extra computation are we talking about at scale?
Lalam: The authors confirm that the added complexity is minimal because they use importance-weighted negative log-likelihood to train those heads.
Tom: So it's a small overhead for a much more reliable output, and they even show it helps with pruning unreliable keypoints before the next step.
Jane: It seems like this moves us closer to systems that can reliably communicate their confidence level, which is crucial for complex decision-making tasks.
Airbus Central Research & Technology · Karlsruhe Institute of Technology · Mistral AI Acubed
cs.CV
Submitted: 2026-07-29
Updated: 2026-11-19
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 100/100
The gist: YOLO-Pose models offer efficient keypoint localization but lack quantification of associated spatial uncertainty.
Key concepts
- YOLO-Pose Models
- These are models used for efficient keypoint localization in images. While good at finding where points are, they do not provide a measure of how certain the model is about those locations. The paper extends these models by adding uncertainty predictions to quantify this lack of certainty.
- Dispersion Matrix
- This is a $2 imes 2$ matrix predicted for each keypoint that describes the spatial uncertainty around that keypoint's location. Training auxiliary heads to predict this matrix helps determine how much the model trusts its own keypoint predictions.
- Student-t Calibration
- This is a statistical method used to refine the predicted dispersion matrices. It is chosen because it better captures heavy-tailed residual structures often found in detectors trained with OKS, providing a more accurate representation of the true uncertainty distribution.
Terminology
Summary
YOLO-Pose models offer efficient keypoint localization but lack quantification of associated spatial uncertainty. p. 0 From Keypoints to Predictive Distributions:
Post-Hoc Uncertainty for YOLO-Pose Models
Alexej Klushyn1† Juan Rivero Sesma2 Florian Seligmann3
Richard Kurle4 Kinh Tieu5 Jayant Sen Gupta1
1Airbus Central Research & Technology 2Airbus Commercial Aircraft
3Karlsruhe Institute of Technology 4Mistral AI 5Acubed
Abstract
YOLO-Pose models provide efficient keypoint localization, but do not quantify the associated spatial uncertainty. We introduce a lightweight post-hoc probabilistic extension that
augments a trained YOLO-Pose model with calibrated bivariate predictive distributions
over keypoint locations, centered at the model’s original predictions. We introduce a lightweight post-hoc probabilistic extension that
augments a trained YOLO-Pose model with calibrated bivariate predictive distributions
over keypoint locations, centered at the model’s original predictions. Concretely, we train
additional probabilistic heads with an importance-weighted negative log-likelihood to predict an input-dependent 2 × 2 dispersion matrix for each keypoint, followed by Gaussian
calibration for broad downstream compatibility or Student-t calibration for distributional
fidelity.Where the paper says it
Where the paper says it
The proposed evaluation protocol combines distributional calibration diagnostics with average keypoint precision (AKP). p. 0 Concretely, we train
additional probabilistic heads with an importance-weighted negative log-likelihood to predict an input-dependent 2 × 2 dispersion matrix for each keypoint, followed by Gaussian
calibration for broad downstream compatibility or Student-t calibration for distributional
fidelity. Complementing this, we propose an evaluation protocol that combines a suite of
distributional calibration diagnostics with average keypoint precision (AKP), a keypointlevel extension of the COCO AP protocol for assessing reliability rankings. Complementing this, we propose an evaluation protocol that combines a suite of
distributional calibration diagnostics with average keypoint precision (AKP), a keypointlevel extension of the COCO AP protocol for assessing reliability rankings. Experiments
on COCO show that the learned uncertainty estimates enable effective keypoint-level reliability ranking, Student-t calibration best captures the empirical residual distribution, and
uncertainty-based pruning removes unreliable keypoints. Both calibration schemes are applied to the same frozen YOLO-Pose model and trained dispersion model. Using calibrated uncertainty estimates as a keypoint-level reliability signal, we further propose uncertaintybased keypoint pruning to remove high-uncertainty keypoints before downstream processing.Where the paper says it
Where the paper says it
Where the paper says it
A primary application demonstration involves vision-based aircraft landing, where calibrated covariances support uncertainty-aware aircraft position estimation and subsequent sensor fusion. p. 0 Experiments
on COCO show that the learned uncertainty estimates enable effective keypoint-level reliability ranking, Student-t calibration best captures the empirical residual distribution, and
uncertainty-based pruning removes unreliable keypoints. A central application-level demonstration is vision-based aircraft landing, where calibrated covariances for runway keypoints
support uncertainty-aware aircraft position estimation and downstream sensor fusion. 1, we introduce an importance-weighted negative log-likelihood (NLL)
for training the probabilistic heads; specifically, we justify using a Gaussian likelihood to learn the conditional
dispersion structure. 2, we propose Gaussian calibration for broad downstream compatibility with
4 3 that both wret and wscale improve
calibration, with wret having the greater impact on overall calibration, while wscale reduces the calibration
gap between small and large instances, leading to more balanced calibration across object scales. Importance-Weighted Training Objective.Where the paper says it
Where the paper says it
Where the paper says it
In the context of vision-based aircraft landing, the system utilizes a Vision-Based Landing (VBL) modality that is independent of ILS ground transmitters and GNSS, offering robustness against radio navigation infrastructure limitations. p. 0 Experiments
on COCO show that the learned uncertainty estimates enable effective keypoint-level reliability ranking, Student-t calibration best captures the empirical residual distribution, and
uncertainty-based pruning removes unreliable keypoints. A central application-level demonstration is vision-based aircraft landing, where calibrated covariances for runway keypoints
support uncertainty-aware aircraft position estimation and downstream sensor fusion. Experiments
on COCO show that the learned uncertainty estimates enable effective keypoint-level reliability ranking, Student-t calibration best captures the empirical residual distribution, and
uncertainty-based pruning removes unreliable keypoints. A central application-level demonstration is vision-based aircraft landing, where calibrated covariances for runway keypoints
support uncertainty-aware aircraft position estimation and downstream sensor fusion. (2021) use normalizing flows to model flexible residual densities for improving localization accuracy. By contrast, we target calibrated predictive distributions within prescribed parametric families, yielding
explicit covariance or scale matrices that are directly compatible with downstream weighting, uncertainty
propagation, and sensor fusion. Finally, we demonstrate the relevance of
calibrated keypoint uncertainty estimates in vision-based aircraft landing, a real-world safety-critical application, using a large-scale internal aircraft-landing dataset. 1 visualizes the predicted image-plane
covariances for runway keypoints; these covariances support uncertainty-aware Perspective-n-Point and provide the measurement covariances required for downstream sensor fusion. Finally, we demonstrate the relevance of
calibrated keypoint uncertainty estimates in vision-based aircraft landing, a real-world safety-critical application, using a large-scale internal aircraft-landing dataset. 1 visualizes the predicted image-plane
covariances for runway keypoints; these covariances support uncertainty-aware Perspective-n-Point and provide the measurement covariances required for downstream sensor fusion.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Post-hoc probabilistic extension for YOLO-Pose models allows for calibrated predictive distributions over keypoint locations centered at original predictions, which enables uncertainty quantification at the level of individual keypoints in real-time applications.
-
The system can perform uncertainty based keypoint pruning by setting a pruning threshold based on localization confidence scores derived from calibrated heteroscedastic covariances, removing unreliable predictions before downstream processing.
-
Using Student-t calibration instead of Gaussian calibration allows the model to better capture the heavy-tailed structure of keypoint localization residuals, leading to improved distributional fidelity and lower Average Coverage Error (ACE) in COCO benchmarks.
-
The system can be used for vision-based aircraft landing by providing calibrated image-plane covariances that support uncertainty aware Perspective n Point (PnP) estimation and downstream sensor fusion within an Extended Kalman Filter (EKF)-based module.
-
Uncertainty based keypoint pruning, when combined with object scale importance weights, provides more balanced calibration across object scales, substantially improving calibration for smaller objects by mitigating the dominance of large instances in the optimization signal.
Abstract
YOLO-Pose models provide efficient multi-object keypoint localization, but do not quantify the associated spatial uncertainty. We introduce a lightweight post-hoc probabilistic extension that augments a trained YOLO-Pose model with calibrated bivariate predictive distributions over keypoint locations, centered at the model's original predictions. Concretely, we train additional probabilistic heads with an importance-weighted negative log-likelihood to predict an input-dependent 2 times2 dispersion matrix for each keypoint, followed by Gaussian calibration for broad downstream compatibility or Student- t calibration for distributional fidelity. Complementing this, we propose an evaluation protocol that combines a suite of distributional calibration diagnostics with average keypoint precision (AKP), a keypoint-level extension of the COCO AP protocol for assessing reliability rankings. Experiments on COCO with YOLO-NAS-Pose and YOLO26-Pose show that the learned uncertainty estimates enable keypoint-level reliability ranking, Student- t calibration best captures the empirical residual distribution, and uncertainty-based pruning removes unreliable keypoints. In vision-based aircraft landing, calibrated predictive distributions over runway keypoints support uncertainty-aware position estimation and downstream sensor fusion.
Sources
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs