From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier

arXiv:2610.10642 · stat.ML, cs.AI, cs.LG · Submitted 2026-10-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "From Log-Odds to Shapley Values".

Jane: The gist: This paper studies how to construct an explanatory space for a weighted naive Bayes classifier by transitioning from conditional log-likelihoods to log-odds representations,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier." It starts by looking at how you can build an explanatory space for a weighted naive Bayes classifier using what the model already generates.

Jane: Basically, they start with the classical supervised distance based on conditional log-likelihoods, but then they introduce a different way to look at it using log-odds instead.

Tom: And this new formulation, based on log-odds, is more directly related to how the classifier actually makes its decision. They claim that this new representation turns out to induce a distance that exactly matches the L1 distance between vectors of analytical Shapley values.

Jane: That’s a big claim because it gives us a formal way to interpret the geometry of what's happening inside the model, linking it directly to those Shapley values.

Lu: It’s interesting because it moves us from something that’s just about conditional probabilities to something that focuses on how variables contribute individually to the classification contrast.

Meng: So, if we’re talking about this paper, what does this mean for how we think about a classifier? Does it help us understand *why* the classifier is making a certain decision?

Tom: Exactly. The authors show that by using log-odds contributions, you get a distance that reflects the actual decision rule better than the original one. This is useful because it connects local neighborhoods in this new space to how predictive behavior actually looks in practice.

Jane: It’s about making sure that when two observations are close in this new distance, they are also close in terms of how likely the classifier predicts them to be similar.

Lu: The paper says they define a 'grow' local contribution to the one-vs-rest log-odds, which is written as l(j)m(D) equals wm log p(Dm Cj) over p(Dm Cj). This measures how much variable m helps discriminate between class Cj and all the other classes.

Meng: How does that local contribution translate into a distance? Is it just summing those contributions up?

Tom: They define the one-vs-rest supervised discriminant distance as the L1 distance between these local contributions. It’s basically taking the sum of those variable contributions, but using L1 instead of squared distances.

Jane: And that's where it gets powerful because they show that this resulting geometry is exactly equal to the L1 distance between Shapley value vectors. That’s what provides that formal explanatory interpretation we talked about.

Lu: They establish a theorem, theorem one which proves for two individuals D and D prime, the L1 distance between their Shapley value vectors is exactly equal to the L1 distance between their log-odds contribution vectors > Page four <ref:2610.10642#pg3>

Conclusion: Tom: So we're wrapping up this look at the paper "From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier." It boils down to using log-odds instead of just log-likelihoods to define a distance that actually reflects how the model makes its decisions.

Jane: That's right. The authors take what used to be a standard way to measure distance based on raw probabilities and switch it over to something more focused on the classification itself, which they call discriminative contrasts.

Lu: What's really neat is how they tie this new distance directly into Shapley values, which are those analytical contributions of each variable to the final prediction. They show that the L1 distance between these Shapley value vectors is exactly the same as this log-odds based distance.

Meng: So it’s not just a neat mathematical trick for them; they're actually linking a geometric shape in their data space to something we already understand from game theory, which is how much each player contributes to an outcome.

Lalam: If we think about that for our AI systems, it means we can use this distance not just to cluster data points, but to understand *why* the model separates them in the first place. It gives us a way to measure the "influence" of features on a specific prediction.

Tom: Exactly. So what does this mean for how we build and trust these classifiers? It suggests that we can structure our local neighborhoods in a way that is consistent with how those classifiers actually behave when they're making predictions, which is something we need for better debugging and understanding.

Jane: It gives us an explanatory lens over the geometry. Instead of just seeing points as close or far apart, we see them in terms of their actual role in the classification process across different variables.

Lu: The implication for future work seems pretty big. If this works reliably across different setups, it opens up a whole new way to formally interpret complex model spaces without getting bogged down in the messy details of raw probabilities.

Meng: From an engineering standpoint, if we can use this distance to find meaningful local neighborhoods that align with predictive behavior, it could make our k-nearest neighbors classifiers much more robust and interpretable in real-world scenarios.

Lalam: And for us, Lalam, it means being able to map out how the model's decision boundary is shaped by the individual pieces of evidence we feed it. It helps us see the structure of its learned knowledge.

Tom: So we've seen that moving from conditional log-likelihoods to log-odds contributions yields a distance that matches Shapley values, and this new distance seems much better at capturing the local predictive logic for weighted naive Bayes classifiers. Next time, we’re going to look at some of those empirical results.

Vincent Lemaire, Fabrice Cl´erot

Orange Research

stat.ML, cs.AI, cs.LG

Submitted: 2026-10-07

Updated: 2026-10-07

Comments: 15 pages

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist: This paper studies how to construct an explanatory space for a weighted naive Bayes classifier by transitioning from conditional log-likelihoods to log-odds representations, ultimately

Key concepts

Conditional Log-Likelihoods: This is the initial way to measure how close two data points are based on the Naive Bayes model. It uses the probability of observing a data point given its class. The paper argues this approach is less direct for understanding why a specific prediction was made.
Log-Odds Representation: Instead of using raw probabilities, this method focuses on discriminative contrasts, specifically log-odds. This representation measures the contribution of each feature to discriminating between a specific class and all other classes, effectively focusing on the decision rule itself rather than global normalization.
Analytical Shapley Values: Shapley values provide a formal way to fairly attribute the total prediction score of a model to individual features. The paper proves that the L1 distance between these calculated Shapley value vectors is exactly equal to the L1 distance between data points when represented using log-odds contributions, offering an interpretable geometric measure.
Discriminative Distance (d_shap): This is the proposed new supervised distance based on log-odds. It measures how different two observations are based on their local contribution to class discrimination. Experiments show this distance performs better than others in classification accuracy and predictive consistency, suggesting it captures local decision boundaries effectively.

Terminology

Shapley values — 49× in this paper · explained in 2 episodes
AUC — 7× in this paper · explained in 2 episodes
Consistency — 5× in this paper · explained in 3 episodes
Aggregation — 2× in this paper
Clustering — 2× in this paper · explained in 2 episodes
Homogeneity — 2× in this paper · explained in 2 episodes
Accuracy — 1× in this paper · explained in 1 episode
Conformal Prediction — 1× in this paper · explained in 10 episodes
Explainability — 1× in this paper · explained in 4 episodes
Generalization — 1× in this paper · explained in 13 episodes
K-means Clustering — 1× in this paper · explained in 3 episodes
Pairwise Comparisons — 1× in this paper · explained in 2 episodes
Personalization — 1× in this paper · explained in 2 episodes

Summary

The gist: This paper studies how to construct an explanatory space for a weighted naive Bayes classifier by transitioning from conditional log-likelihoods to log-odds representations, ultimately showing that this new representation induces a distance exactly coinciding with the L1 distance between analytical Shapley values.

Context: Supervised distance induced by a classifier

The paper begins by discussing the utility of having an observation representation consistent with how the model generates its prediction scores, which allows for defining a distance based on both raw geometry and role in the decision <ref:2610.10642#pg6> The naive Bayes classifier naturally leads to constructing a supervised representation based on conditional probabilities associated with explanatory variables <ref:2610.10642#pg4> Two observations close in this representation are expected to be close in terms of their predictive behavior as well <ref:2610.10642#pg4>

Proposal for a log-odds-based supervised discriminant distance

The authors propose replacing the representation based on conditional log-likelihoods with one based on discriminative contrasts, specifically log-odds, to eliminate the global normalization term present in the initial distance formulation <ref:2610.10642#pg6> They define a 'grow' local contribution to the one-vs-rest log-odds as l(j)m(D) = wm log p(Dm Cj) / p(Dm Cj) <ref:2610.10642#pg4> This quantity measures the contribution of variable m to discriminating between class Cj and all other classes <ref:2610.10642#pg4> The resulting one-vs-rest supervised discriminant distance is defined as the L1 distance between these local contributions <ref:2610.10642#pg4> This formulation has several advantages, including directly related to the decision rule and eliminating the normalization term p(D) from equation 4 <ref:2610.10642#pg6>

Link to Shapley Values

The paper establishes a formal link between the log-odds-based representation and the analytical Shapley values of the naive Bayes classifier <ref:2610.10642#pg4> Theorem 1 proves that for two individuals D and D′, the L1 distance between their Shapley value vectors is exactly equal to the L1 distance between their log-odds contribution vectors <ref:2610.10642#pg4> This equivalence holds in the binary case and under a chosen reference distribution for the Shapley values <ref:2610.10642#pg4> In the multiclass case, this equivalence is maintained if a detailed vector representation indexed by both the variable and the reference class is preserved, meaning individuals must be compared component by component <ref:2610.10642#pg4>

Experiments

Empirical evaluation using a k-nearest-neighbor (k-NN) classifier on twelve different datasets was conducted to assess the relevance of these distances for structuring local neighborhoods consistent with predictive behavior <ref:2610.10642#pg6> The experiments compare three distances: the initial weighted supervised distance d global, the discriminant distance d odds, and the original weighted supervised distance d SNB <ref:2610.10642#pg4> Results indicate that the discriminant distance d shap achieves the best average performance in both ACC and AUC across all twelve datasets <ref:2610.10642#pg6> Furthermore, for predictive consistency, the distance d shap yields the highest values for LocAgreek, indicating that its neighbors are more frequently assigned the same predicted class than other distances <ref:2610.10642#pg4>

Conclusion

The article proposes a reformulation of the supervised distance based on local contributions to log-odds, which is more closely related to the discriminative nature of the decision rule <ref:2610.10642#pg6> The resulting geometry has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4> Experiments suggest that d shap is a relevant alternative to the supervised distance used in [10] because it tends to preserve the local homogeneity of the classes predicted by the SNB more effectively <ref:2610.10642#pg6> This suggests that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>. The results suggest that this distance is a relevant alternative to the supervised distance used in [10]; this is achieved with an interpretation based on Shapley values <ref:2610.10642#pg4>. The proposed geometry thus has an interpretation that is both supervised and explanatory <ref:2610.10642#pg4>.

Improvements for AI systems

Shapley Value Geometry for Explanatory Distance This improvement allows AI systems to define a supervised distance that exactly coincides with the l1 distance between vectors of analytical Shapley values, providing a formal explanatory interpretation of the model's geometry, moving beyond raw conditional log-likelihoods.
Discriminative Log-Odds Representation Implementing the log-odds-based distance allows AI systems to create a representation where the discriminant distance d odds coincides exactly with the l1 distance between vectors of analytical Shapley values, which is more directly related to the decision and eliminates global normalization terms.
Enhanced Local Neighborhood Homogeneity Using the d shap distance in k-NN frameworks, AI systems can construct local neighborhoods that tends to preserve the stability of the predicted class better, as demonstrated by higher LocAgreek values compared to other distances, leading to more reliable local predictions.
Improved Local Class Separation Margin The ability to calculate a normalized local margin allows AI systems to evaluate the ability of each distance to locally separate the true classes, enabling the selection of distances that maximize this margin for better classification performance in a 1-ppv framework.

Abstract

This paper studies the construction of an explanatory space for a weighted naive Bayes classifier from the supervised representation induced by the model. We start from the classical supervised distance based on conditional log-likelihoods and introduce a discriminative reformulation based on log-odds, which is more directly related to the classification decision. We then show that this representation induces a distance that exactly coincides with the 1 distance between vectors of analytical Shapley values, thereby providing a formal explanatory interpretation of the geometry induced by the model. Finally, we empirically compare several supervised distances derived from these representations using a k-nearest neighbors classifier. This work highlights a close link between supervised distance, local explanation, and predictive behavior, from a primarily methodological perspective.