Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence

arXiv:2610.14809 · cs.LG · Submitted 2026-10-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Groups Don't Move as One".

Jane: The gist The Bregman view makes this compression explicit: BI quantifies the omitted variation,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So, this paper is called "Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence". It sounds pretty dense. Basically, it’s about how we estimate the effect of a whole group of data points on a model.

Jane: That’s the core idea—that when you look at all those individual points in a group, they don't just move together as one big blob. They actually have different internal dynamics, and the standard way we calculate influence functions hides that difference by just summing everything up.

Lu: Exactly. The authors are proposing using a Bregman centroid formulation to make this explicit. They’re taking that aggregate estimate, which they call the single representative theta bS epsilon, and then they use Bregman Information, or BI, to measure all the variation that this single representative omits when looking at the actual data points.

Meng: So what does that mean for deployment? If we rely on standard group influence estimates without this decomposition, we might be missing something important about how different subgroups within our training set are actually behaving.

Lalam: Lalam has the literature on this, and what it says is that standard first-order group influence estimates reduce individual contributions to a sum that can hide substantial differences among members, including opposing effects that cancel out > pg1.

Tom: Right. So the paper suggests that instead of just one big number for the group's effect, we should be looking at how those individual points contribute separately and seeing how they reinforce or oppose each other > pg1. This is a shift from treating the group as a single entity to treating it as a collection of parts.

Jane: And this new view lets us use that structure to get a better estimate by accounting for the variation, which they call Bregman Information, which quantifies the variation it omits > pg5. It’s like getting an extra diagnostic tool for our influence estimates.

Lu: The paper suggests that extending this formulation to multiple clusters reveals distinct contributions whose sum recovers the original estimate > pg2. It’s about breaking down that one big group into pieces where you can see what each piece is actually doing on its own.

Meng: If we can break it down, then maybe we can deploy models where we know which parts of the data are driving specific behaviors, instead of just trusting the aggregate score > pg2.

The paper's summary: Tom: Okay, so summarizing the core mechanism of "Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence". The paper shows that there’s a standard way to estimate group influence that uses a single centroid, which is called theta bS epsilon > pg3.

Jane: That centroid representation is what’s keeping the standard group influence estimate simple. But the paper then introduces component decomposition, which breaks down the total variation into these individual cluster contributions, denoted as sum r Delta theta r = ̂ theta bS epsilon − θ0 > pg3 <ref:2610.14809#pg3>.

Lu: These component contributions are defined by a size-weighted displacement, where Delta theta r is like the difference between that cluster’s representative mu r and the overall centroid theta0 > pg3. It's basically quantifying how much each group part pushes away from the center of everything.

Meng: So, if we have ten clusters, we get ten individual displacement terms. That sounds like a lot of bookkeeping to manage when you’re trying to deploy something quickly > pg2.

Tom: It is a lot of bookkeeping initially, but the next part is where they propose the improvement: they show that instead of just summing these displacements up again, we can jointly optimize the weights for each cluster's contribution rather than just summing them > pg2.

Jane: That joint optimization step is key because it keeps those cluster centroids fixed while letting the relative importance of different clusters change based on what we’re trying to predict > pg2. It’s a way to capture that internal reinforcement or opposition you mentioned earlier.

Lalam: Lalam confirms that this decomposition reveals structure without changing the additive estimate, where component contributions sum to standard GI > pg3. It shows the same result but with more underlying detail than just the single number we get from standard group influence functions > pg3.

The paper's improvements: Tom: Okay, so let’s talk about what they actually improved. They propose an estimator called Joint E2, which is characterized as a Bregman centroid objective > pg3. This objective identifies both the single representative that standard group influence uses and the dispersion it omits > pg3.

Jane: That dispersion is measured by Bregman Information, or BI, which they say quantifies this omitted variation as the minimum average divergence from the response points to a single representative > pg5. It’s a measure of how much structure we’re leaving out when we only use that one centroid > pg5.

Lu: The real gain is that this decomposition allows us to adjust the weights of these cluster contributions, instead of just adding them up, which they call component reweighting > pg2. This is what allows the relative contributions of different clusters to change based on target-specific BI > pg2.

Meng: So this means we can get a better R2 score when testing against target changes under the nonlinear proximal Bregman response function, or PBRF, which they call it > pg2. That’s a concrete win for accuracy in our evaluation metrics.

Tom: And they even look at ways to make things faster. They propose two approximations: one is within-component curvature approximation, where Intra E2 only uses alpha r Delta theta r in the additional quadratic term, omitting cross-direction terms > pg6.

Jane: And another way to speed things up is Euclidean component construction, which involves forming components by clustering raw gradients using Euclidean distances without any curvature weighting > pg6. This sounds like a practical trick for reducing computational cost.

Lalam: Lalam notes that the Joint E2 estimator jointly optimizes the component coefficients by minimizing QE2 S, which incorporates a GGN-based quadratic approximation of the PBRF edit objective > pg4. It leads to a target-change prediction of ∇ theta f(theta0)⊤ (theta(alpha⋆) − theta0 > pg4.

Conclusion: Tom: So, to wrap up "Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence". The main implication is that structure hidden by just summing up influences can be exposed and used to refine our group influence estimates > pg1.

Jane: This means examining within-group structure can support both a finer understanding of group influence and more accurate estimation, especially when dealing with complex, nonlinear loss surfaces like the proximal Bregman response function > pg2.

Lu: The paper proves that this characterization is exact for the linearized response, which gives us a solid theoretical foundation for how we estimate these effects > pg3.

Meng: From an engineering standpoint, it means we can get higher R2 scores than standard group influence when evaluating against target changes under the nonlinear proximal Bregman response function > pg2.

Lalam: Lalam thinks the results show that structure hidden by additive aggregation can be exposed and used to refine group influence estimates > pg1. It’s about getting a better handle on those hidden patterns in groups, which is what matters most for model accuracy > pg1.

Tom: So, ultimately, this is about moving from a single aggregate number to a system that analyzes the internal structure of the data group > pg1. We’ve seen how this decomposition can lead to more accurate influence estimates by looking at what the paper calls Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence > pg1.

Jane: And that’s where we are for this paper. It shows us how to use structure within a group for better understanding and more accurate estimation > pg2.

Lu: We’re ready to move on now, but the next step is seeing how these component structures work in practice > pg3.

Hyunho Choi, Jaeseung Heo, Youngbin Choi, MoonJeong Park, Dongwoo Kim

Department of Computer Science & Engineering, POSTECH · Graduate School of Artificial Intelligence, POSTECH

cs.LG

Submitted: 2026-10-10

Updated: 2026-10-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 78/100

The gist: The gist The Bregman view makes this compression explicit: BI quantifies the omitted variation, while multiple components preserve the original estimate and reveal how different parts of the group

Key concepts

Bregman Information (BI)
BI quantifies the variation omitted when estimating group influence by a single representative. It measures the minimum average divergence from response points to that representative, exposing structural differences within the group that simple summation hides.
Bregman Centroid Representation
The linearized objective underlying group influence is equivalent to finding a Bregman centroid. This centroid acts as a single representative of scaled individual parameter-change estimates, capturing the primary fit while leaving dispersion uncaptured.
Component Decomposition
Groups are partitioned into disjoint components, each represented by a cluster mean. The displacement of these component representatives from the overall group centroid defines their unique contribution to the group's total influence effect.

Terminology

Summary

The gist The Bregman view makes this compression explicit: BI quantifies the omitted variation, while multiple components preserve the original estimate and reveal how different parts of the group reinforce or oppose one another.

How it works

  1. Standard first-order group influence estimates this joint effect by summing individual influence estimates, but this sum hides differences among group members (Page 1).

  2. The linearized objective underlying group influence is equivalent to a Bregman centroid problem, where the aggregate is a single representative of scaled individual parameter-change estimates (Page 1).

  3. Bregman Information (BI) provides a measure of this hidden variation, quantifying the variation it omits (Page 1).

  4. The linearized group PBRF objective is characterized as a Bregman centroid objective, which identifies both the single representative retained by GI and the dispersion it omits (Page 3).

  5. This decomposition reveals structure without changing the additive estimate, where component contributions sum to standard GI (Page 3).

Bregman Centroid Representation

The linearized group PBRF objective is shown to be a Bregman centroid objective, which identifies both the single representative retained by GI and the dispersion it omits (Page 3). The unique minimizer of this objective is the Bregman centroid, denoted as theta bS (epsilon) = theta0 - epsilonG−1cS (Page 3). This formulation shows that standard GI fits a single representative to the response cloud in the geometry induced by G (Page 3).

Bregman Information and Dispersion

Bregman Information (BI) measures this omitted dispersion as the minimum average divergence from the response points to a single representative, corresponding to the minimum value of the first term on the right-hand side of Equation (8) (Page 5). Projected BI measures variation among unscaled influence scores for a chosen target, rather than variation across the full parameter-response space (Page 5).

Component Decomposition and Correction

The group can be partitioned into k nonempty, disjoint components C1,..., Ck, where each Cr indexes a subset of the response cloud with representative µr in R d (Page 3). The size-weighted displacement defines the component contribution ∆θr = Cr/m (µr − θ0) = −ϵG−1 X i∈Cr zi (Page 3).

Improving Group Influence with Component Structure

A proposed estimator jointly optimizes the weights of these cluster contributions, rather than simply summing them, which keeps the centroids fixed while allowing the relative contributions of different clusters to change (Page 2). This estimator achieves higher R2 than standard group influence when evaluated against target changes under the nonlinear proximal Bregman response function (PBRF) (Page 2).

Efficient Approximations

Two approximations are considered to reduce computational costs:

  1. Within-component curvature approximation: Intra E2 uses only αr∆θr in the additional quadratic term, omitting cross-direction terms and evaluating each direction’s curvature only on its own component’s examples (Page 6).

  2. Euclidean component construction: This involves forming components by clustering raw gradients zi using Euclidean distances, without curvature weighting (Page 6).

The proposed Joint E2 estimator jointly optimizes the component coefficients by minimizing QE2 S, which incorporates a GGN-based quadratic approximation of the PBRF edit objective (Page 4). The resulting target-change prediction is ∇θf(θ0)⊤(θ(α⋆) − θ0) (Page 4).

Conclusion

The Bregman characterization is exact for the linearized response, whereas the BI–error association and estimation gains are empirical (Page 10). The results show that structure hidden by additive aggregation can be exposed and used to refine group influence estimates (Page 15). Together, these results show that examining within-group structure can support both a finer understanding of group influence and more accurate estimation (Page 2).

--- Page 1 ---

Preprint. Under review.

GROUPS DON’T MOVE AS ONE: A BREGMAN CLUSTERING VIEW OF LINEARIZED GROUP INFLUENCEHyunho Choi1, Jaeseung Heo2, Youngbin Choi2, MoonJeong Park2, Dongwoo Kim1,2,…

--- Page 30 ---

Preprint. Under review.

Table 10: Dogfish, 30%: component-count sweep on the same 100 groups and 100 queries. Each entry reports the median over fixed test queries. The four variants coincide at k = 1.

--- Page 32 ---

Preprint. Under review.

Table 9: Correction results against actual PBRF reference points. All settings use 100 random groups; metrics are medians over the common retained queries (100 for Dogfish and MNIST, and 97 for ResNet9). All methods within a setting share the checkpoint, groups, queries, and actual PBRF references.

--- Page 33 ---

Preprint. Under review.

Table 12: Dogfish, 10%: component-count sweep on the same 100 groups and 100 queries. Each entry reports the median over fixed test queries. The four variants coincide at k = 1.

--- Page 34 ---

Preprint. Under review.

Table 8: Geometry ablation on the same random 1% groups and fixed queries as the BI experiment. Entries are medians over 100 query-wise Spearman correlations with squared GI error, each computed across 100 groups.

--- Page 35 ---

Preprint. Under review.

Table 6: Median query-wise accuracy and two-sided query-paired p-values for Euclidean E2 (k = 2) versus single-component E2 (k = 1). Holm correction covers all 26 comparisons jointly across settings, variants, and both metrics (Page 35).

--- Page 36 ---

Preprint. Under review.

Table 7: Additional removal ratios, with 100 random groups and 100 retained fixed queries per setting. All methods within a setting share the checkpoint, groups, queries, and actual PBRF references (Page 36).

--- Page 37 ---

Preprint. Under review.

Table 8: Geometry ablation on the same random 1% groups and fixed queries as the BI experiment (Page 37).

--- Page 28 ---

Preprint. Under review.

Table 5: Removal-ratio ablation for projected BI versus squared GI error (Page 28).

--- Page 29 ---

Preprint. Under review.

Table 6: Median query-wise accuracy and two-sided query-paired p-values for Euclidean E2 (k = 2) versus single-component E2 (k = 1). Holm correction covers all 26 comparisons jointly across settings, variants, and both metrics (Page 29).

--- Page 31 ---

Preprint. Under review.

Table 7: Additional removal ratios, with 100 random groups and 100 retained fixed queries per setting (Page 31).

--- Page 32 ---

Preprint. Under review.

Table 8: Geometry ablation on the same random 1% groups and fixed queries as the BI experiment (Page 32).

--- Page 34 ---

Preprint. Under review.

Table 9: Correction results against actual PBRF reference points (Page 34).

--- Page 32 ---

Preprint. Under review.

Table 10: Dogfish, component-count sweep on the same 100 groups and 100 queries (Page 32).

--- Page 33 ---

Preprint. Under review.

Table 11: MNIST, component-count sweep on the same 100 groups and 100 queries (Page 33).

--- Page 27 ---

Preprint. Under review.

Table 14: Median edit-curvature stage time (seconds), MNIST 10%, fixed Euclidean components, 49 matched groups on an RTX3090 (Page 27).

--- Page 16 ---

Preprint. Under review.

Table 1: Ablation of clustering geometry and edit-curvature scope at k = 2 (Page 16).

--- Page 29 ---

Preprint. Under review.

Table 8: Geometry ablation on the same random 1% groups and fixed queries as the BI experiment (Page 29).

--- Page 30 ---

Preprint. Under review.

Table 9: Correction results against actual PBRF reference points (Page 30).

--- Page 31 ---

Preprint. Under review.

Table 10: Dogfish, component-count sweep on the same 100 groups and 10% removal (Page 31).

--- Page 32 ---

Preprint. Under review.

Table 9: Correction results against actual PBRF reference points (Page 32).

--- Page 34 ---

Preprint. Under review.

Table 10: Dogfish, component-count sweep on the same 100 groups and fixed test queries (Page 34).

--- Page 34 ---

Preprint. Under review.

Improvements for AI systems

  1. Bold header: Target-Projected Bregman Information (BI) as an Error Diagnostic. This allows for quantifying what additive group influence leaves out by measuring variation among the unscaled influence scores for the chosen target, providing a direct link between hidden structure and approximation error.

  2. Bold header: Component-Based Influence Reweighting for Enhanced Accuracy. By proposing an estimator that jointly optimizes the weights of these cluster contributions, rather than simply summing them, the system can achieve higher R2 than standard group influence by accounting for distinct cluster-wise influence patterns hidden by the aggregate.

  3. Bold header: Dynamic Group Structure Exploitation via Component Decomposition. The ability to partition a group into components allows for adjusting the component weights based on distinct class-wise influence patterns, enabling the system to capture internal reinforcement or opposition that a single centroid view misses.

  4. Bold header: Efficient Computational Cost Reduction via Euclidean Approximation. Utilizing Euclidean clustering for partition construction can substantially reduce time, as demonstrated by a 99.5% reduction from the two medians over 100 matched groups, making the analysis of group structure more computationally feasible on large datasets.

  5. Bold header: Query-Specific Influence Estimation under Nonlinear Response. The proposed estimator allows for prediction of target changes using target-projected BI, which is associated with approximation error under the nonlinear proximal Bregman response function (PBRF), enabling more accurate evaluation against complex loss surfaces like those in ResNet9.

Abstract

Influence functions provide an efficient alternative to retraining for assessing the effects of removing or reweighting training data. For groups, however, the standard first-order estimate reduces individual contributions to a sum that can conceal substantial differences among members, including opposing effects that cancel. We establish a Bregman centroid formulation of the linearized group influence objective, interpreting this aggregate as a single representative and using Bregman information (BI) to quantify the variation it omits. Extending this formulation to multiple clusters reveals distinct contributions whose sum recovers the original estimate. We further propose an estimator that jointly optimizes the weights of these cluster contributions, rather than simply summing them. Experiments uncover cluster-specific influence patterns and a positive association between target-specific BI and approximation error. The proposed estimator achieves higher R squared than standard group influence in predicting target changes under the nonlinear proximal Bregman response function. Simplified clustering and correction variants reduce computational cost. The centroid view thus explains what additive group influence leaves out and provides a basis for using that information to improve estimation.

Sources

Related papers