Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging

arXiv:2611.13194 · econ.GN, q-fin.EC · Submitted 2026-11-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging".

Warren: Here's the gist of the paper. Using a rich dataset from figure skating competitions, the authors find evidence of a strategic compatriot premium and a targeted penalty imposed on close rivals.

Ruth: This premium is influenced by factors such as randomly drawn start orders and rises when close rivals have judges from their own federations on the panel.

Warren: The penalty for rivals grows with the compatriot's chances of good placement and diminishes once she is out of contention.

Ruth: A counterfactual analysis demonstrates that these biases can significantly shape close competitions.

Paper summary: Ruth: So we're looking at this paper now, "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging." It’s about finding evidence for these strategic biases in judging, specifically a premium for the compatriot and a penalty against close rivals. The authors use a rich dataset from figure skating competitions to look at how these decisions are made when judges have ties.

Warren: That sounds like it starts with separating the motive—is it just personal taste or is there something strategic happening? It claims they found evidence of a strategic compatriot premium and targeted penalties imposed on close rivals. They also point out that this premium is influenced by things like randomly drawn start orders, and it rises when close rivals have judges from their own federations.

Iris: So the core claim is that these biases are not just accidental favoritism but have a strategic element to them, which is interesting because the existing literature on the compatriot premium often struggles to separate intentional bias from non-intentional explanations like shared taste or familiarity.

Ruth: Exactly. The paper argues that even when start orders are purely random, you see the judge's premium for her compatriot drop as the competition goes on, which suggests something else is at play beyond just who started where.

Warren: And they also found a targeted penalty against rivals, which grows based on how likely the compatriot is to get a good placement and then fades once she’s out of contention. It sounds like they're mapping out the whole dynamic of competition outcomes.

Iris: What I find particularly compelling is how they use a counterfactual analysis to show that these biases—the premium, the penalty, and what they call colleague-compatriot premium—jointly influence competition results through the random draw of the judging panel composition. That helps establish a causal link between these factors and the actual scores.

Ruth: So, in simple terms, it means that when you look at close events in figure skating, you have to consider how these three elements—the compatriot premium, the competitor penalty, and the colleague-compatriot premium—all work together based on how the judging panel is set up randomly. It’s a model of competition shaped by these biases.

Warren: And what this means for us is that current rules might not offer much protection against these biases because they only remove a small portion of whatever premium or penalty exists. The authors strongly recommend keeping the federations of leading skaters off judging panels when possible, or seating their judges symmetrically—either all on one segment or none—for all judges on a segment, either for all judges on a segment or none.

Iris: That recommendation moves beyond just describing the bias; it suggests a practical design change to mitigate it. They suggest that building upon this approach by using statistical tools and judging records to monitor judge behavior and find ways to reduce the intentional and strategic components of the compatriot premium is the next step for research.

Ruth: So, while machine judging might handle some of these issues, they are calling for more focused human intervention based on statistical monitoring to curb that strategic component. It shifts the focus from just trusting an algorithm to actively managing how the panel itself is formed or operated.

Warren: Thinking about the implications, if these strategic elements are driving results in close competitions, then understanding this mechanism helps us see where external factors—like panel composition—are introducing non-random noise into performance evaluation. It suggests that relying solely on the judging outcome doesn't tell you the full story of what happened.

Iris: It frames the problem less as a failure of athletes or judges, and more as a structural issue in how subjective expert panels are designed and composed, especially when those panels have internal connections to the competitors.

Ruth: This paper, "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging," points out that the decisions made by expert panels aren't always just about what a judge personally likes; they can be shaped by strategic factors related to nationality and rivalry management. It shows that understanding these mechanisms is key to figuring out how to make judging systems fairer, even when they rely on human judgment.

Conclusion: Ruth: So we're wrapping up this look at "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging." Basically, these authors found evidence for a strategic premium for a compatriot and targeted penalties against rivals.

Warren: It’s about moving past just thinking judges are biased by personal taste and showing that there's a real strategy going into how those panels are put together.

Iris: The paper looked at the data from figure skating competitions and found that these biases aren't just random errors; they have a structure to them based on the panel composition.

Ruth: Exactly. They used this rich dataset to show how things like random start orders and whether rivals share judges’ federations actually influence those scores in ways we didn't expect.

Warren: And the results showed that these three things—the compatriot premium, the competitor penalty, and what they called the colleague-compatriot premium—all work together based on how the judging panel is drawn randomly. It’s a system where all three factors interact to shape competition outcomes.

Iris: The methodology was pretty solid in showing that this joint influence is what drives the variation we're looking at, which helps identify what actually matters in terms of statistical identification.

Ruth: So, what does this mean for the world outside of figure skating? It suggests that even when you think you’re getting a purely objective score from judges, there can be these systemic biases baked into how those panels are selected.

Warren: If these strategic elements are driving results in close events, then current rules might just be missing something because they only remove a small piece of whatever premium or penalty is actually there.

Iris: The authors suggest keeping the federations of leading skaters off judging panels when it’s feasible, or seating judges symmetrically—either all on one segment or none—for all judges on a segment, either for all judges on a segment or none.

Ruth: They want us to use statistical tools and judge marking records to actually monitor judge behavior and find ways to reduce those intentional strategic components of the premium.

Warren: It shifts the focus from just trusting machine judging to actively managing how the panel itself is formed or operated when you're dealing with these kinds of biases.

Iris: The paper shows that even in a subjective system, there’s an underlying mechanism where connections between people and judges can introduce a measurable, strategic effect on performance evaluation.

Ruth: Next up, we’ll look at how this kind of structural bias might show up in other high-stakes judging environments.

Jialin Liu, Shu Shen

University of California, Davis

econ.GN, q-fin.EC

Submitted: 2026-11-09

Updated: 2026-11-09

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Using a rich dataset from figure skating competitions, the authors find evidence of a strategic compatriot premium and a targeted penalty imposed on close rivals.

Key concepts

Compatriot Premium
This is an advantage given to a skater who is from the same country as their close rival. This advantage isn't random; it can be strategic, increasing if rivals have judges from their own federation on the panel.
Targeted Penalty
Judges sometimes impose penalties specifically against a close rival. This penalty increases if the compatriot has a high chance of placing well, suggesting an intentional effort to disadvantage the competitor.
Counterfactual Analysis
This method tests what would have happened under different conditions. The analysis showed that when considering start orders and judging panel composition, these biases—premium and penalties—jointly determine the final competition results.

Terminology

Summary

Using a rich dataset from figure skating competitions, the authors find evidence of a strategic compatriot premium and a targeted penalty imposed on close rivals. p. 0

Where the paper says it

edu ∗ November 11, 2026 Abstract Many decisions rest on expert panels whose members have ties to some candidates, and favoritism is hard to tell from shared taste or better information. Using a rich dataset from figure skating, we find evidence of a strategic compatriot premium and a targeted penalty on the compatriot’s close rivals.

This premium is influenced by factors such as randomly drawn start orders and rises when close rivals have judges from their own federations on the panel. p. 0
Where the paper says it

Using a rich dataset from figure skating, we find evidence of a strategic compatriot premium and a targeted penalty on the compatriot’s close rivals. The premium falls along a randomly drawn start order and, before 2022/23, rises when close rivals have their own judges on the panel.

The penalty for rivals grows with the compatriot's chances of good placement and diminishes once she is out of contention. A counterfactual analysis demonstrates that these biases can significantly shape close competitions. p. 0
Where the paper says it

The penalty grows with the compatriot’s placement chances and fades once she is out of contention. A counterfactual analysis shows that these biases can shape close competitions.

In figure skating, while the compatriot premium is well-documented, the authors find it to be at least in part strategic. p. 2

Where the paper says it

Leveraging the institutional design of figure skating, we find that the premium is, at least in part, strategic. Even when the start order is purely random, a judge’s premium for her compatriot declines as the competition goes on.

Even when start orders are purely random, a judge's premium for her compatriot declines as the competition progresses. Furthermore, the compatriot premium increases when close rivals have judges from their own federations on the panel. p. 2
Where the paper says it

When she marks an early-starting compatriot, a judge cannot tell how big a premium she needs to award to make it matter, whereas for a late-starting compatriot she knows better whether a premium is worthwhile and, if it is, how large it must be to change a placement. The compatriot premium also rises when the compatriot’s close rivals, especially those who start later, have judges of their own federations on the panel.

Judges also mark down compatriots' close rivals in a targeted manner. The competitor penalty is also found to be targeted, increasing with the compatriot's chance of a good placement. The counterfactual analysis confirms that in close events, the compatriot premium, competitor penalty, and colleague-compatriot premium jointly influence competition results through the random draw of the judging panel composition. p. 4
Where the paper says it

With the above mentioned findings of strategic nationalistic scoring, we carry out a counterfactual analysis of the figure skating events at the 2026 Milan Winter Olympic Games. We show that the compatriot premium, the competitor penalty, and the colleague-compatriot premium, through the random draw of judge panels, can jointly shape close competitions.

The authors conclude that current rules do not offer much protection against these biases, as they only remove a small portion of a typical premium or penalty. p. 4

Where the paper says it

The ISU’s trimmed mean offers little protection. We derive how much of one judge’s bias a nine-judge trim removes, and trimming removes only a small part of the compatriot premium or the competitor penalty of a typical size.

They strongly recommend keeping the federations of leading skaters off judging panels when feasible, or seating their judges symmetrically (either all on one segment or none), either for all judges on a segment or none. p. 4
Where the paper says it

We therefore suggest keeping the federations of the leading skaters off the judging panel, when feasible, as the ISU already does for the technical panel. Last but not least, we would like to point out that, in this paper, we refrain from either selecting a set of biased judges with multiple testing procedures or ranking the judges, although we obtain judge-specific compatriot premium estimates as an intermediate step of our regression analysis.

The paper suggests building upon this approach by using statistical tools and judges' marking records to monitor judge behavior and find ways to reduce the intentional and strategic components of the compatriot premium, rather than relying solely on machine judging.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement strategic nationalistic scoring detection by analyzing whether judges adjust their premium based on how well their compatriot has skated or her start position, as this can be explained by unintentional factors, or is it instead in line with nationalistic incentives.

  2. Develop a counterfactual analysis module to simulate competition outcomes under different panel draws, showing that the compatriot premium, the competitor penalty, and the colleague-compatriot premium influence competition results through the random draw of the panel composition.

  3. Create a system to monitor judge behavior by tracking whether judges escalate their nationalistic bias in response to panel composition and colleagues’ reputation, which could be done by measuring coefficients like the slope coefficients beta+3 and beta-in2R-in in model (4).

  4. Build a mechanism to assess the impact of specific judge characteristics, such as federation, on outcomes by using the controls defined in equation (2') to see how the colleague-compatriot premium shrinks substantially when these novel controls are included.

  5. Integrate panel composition rules into decision support systems, advising whether keeping the federations of the leading skaters off the judging panel is feasible or suggesting an alternative like seating judges symmetrically, either all of them on a segment or none of them.

Abstract

Many decisions rest on expert panels whose members have ties to some candidates, and favoritism is hard to tell from shared taste or better information. Using a rich dataset from figure skating, we find evidence of a strategic compatriot premium and a targeted penalty on the compatriot's close rivals. The premium falls along a randomly drawn start order and, before 2022/23, rises when close rivals have their own judges on the panel. The penalty grows with the compatriot's placement chances and fades once she is out of contention. A counterfactual analysis shows that these biases can shape close competitions.

Sources

Related papers