Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging

summary

The gist

Using a rich dataset from figure skating competitions, the authors find evidence of a strategic compatriot premium and a targeted penalty imposed on close rivals.

In short

The study analyzed figure skating judging to find evidence of strategic bias between compatriots and close rivals. It discovered a 'compatriot premium' influenced by start orders and judge federation, alongside targeted penalties against rivals that increase as the compatriot improves. These biases significantly shape competition outcomes.

Key concepts

Compatriot Premium
This is an advantage given to a skater who is from the same country as their close rival. This advantage isn't random; it can be strategic, increasing if rivals have judges from their own federation on the panel.
Targeted Penalty
Judges sometimes impose penalties specifically against a close rival. This penalty increases if the compatriot has a high chance of placing well, suggesting an intentional effort to disadvantage the competitor.
Counterfactual Analysis
This method tests what would have happened under different conditions. The analysis showed that when considering start orders and judging panel composition, these biases—premium and penalties—jointly determine the final competition results.

Terminology used across episodes

This episode discusses

The paper

Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging · Read on arXiv

Jialin Liu, Shu Shen

University of California, Davis

Many decisions rest on expert panels whose members have ties to some candidates, and favoritism is hard to tell from shared taste or better information. Using a rich dataset from figure skating, we find evidence of a strategic compatriot premium and a targeted penalty on the compatriot's close rivals. The premium falls along a randomly drawn start order and, before 2022/23, rises when close rivals have their own judges on the panel. The penalty grows with the compatriot's placement chances and fades once she is out of contention. A counterfactual analysis shows that these biases can shape close competitions.

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging".

Warren: Here's the gist of the paper. Using a rich dataset from figure skating competitions, the authors find evidence of a strategic compatriot premium and a targeted penalty imposed on close rivals.

Ruth: This premium is influenced by factors such as randomly drawn start orders and rises when close rivals have judges from their own federations on the panel.

Warren: The penalty for rivals grows with the compatriot's chances of good placement and diminishes once she is out of contention.

Ruth: A counterfactual analysis demonstrates that these biases can significantly shape close competitions.

Paper summary: Ruth: So we're looking at this paper now, "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging." It’s about finding evidence for these strategic biases in judging, specifically a premium for the compatriot and a penalty against close rivals. The authors use a rich dataset from figure skating competitions to look at how these decisions are made when judges have ties.

Warren: That sounds like it starts with separating the motive—is it just personal taste or is there something strategic happening? It claims they found evidence of a strategic compatriot premium and targeted penalties imposed on close rivals. They also point out that this premium is influenced by things like randomly drawn start orders, and it rises when close rivals have judges from their own federations.

Iris: So the core claim is that these biases are not just accidental favoritism but have a strategic element to them, which is interesting because the existing literature on the compatriot premium often struggles to separate intentional bias from non-intentional explanations like shared taste or familiarity.

Ruth: Exactly. The paper argues that even when start orders are purely random, you see the judge's premium for her compatriot drop as the competition goes on, which suggests something else is at play beyond just who started where.

Warren: And they also found a targeted penalty against rivals, which grows based on how likely the compatriot is to get a good placement and then fades once she’s out of contention. It sounds like they're mapping out the whole dynamic of competition outcomes.

Iris: What I find particularly compelling is how they use a counterfactual analysis to show that these biases—the premium, the penalty, and what they call colleague-compatriot premium—jointly influence competition results through the random draw of the judging panel composition. That helps establish a causal link between these factors and the actual scores.

Ruth: So, in simple terms, it means that when you look at close events in figure skating, you have to consider how these three elements—the compatriot premium, the competitor penalty, and the colleague-compatriot premium—all work together based on how the judging panel is set up randomly. It’s a model of competition shaped by these biases.

Warren: And what this means for us is that current rules might not offer much protection against these biases because they only remove a small portion of whatever premium or penalty exists. The authors strongly recommend keeping the federations of leading skaters off judging panels when possible, or seating their judges symmetrically—either all on one segment or none—for all judges on a segment, either for all judges on a segment or none.

Iris: That recommendation moves beyond just describing the bias; it suggests a practical design change to mitigate it. They suggest that building upon this approach by using statistical tools and judging records to monitor judge behavior and find ways to reduce the intentional and strategic components of the compatriot premium is the next step for research.

Ruth: So, while machine judging might handle some of these issues, they are calling for more focused human intervention based on statistical monitoring to curb that strategic component. It shifts the focus from just trusting an algorithm to actively managing how the panel itself is formed or operated.

Warren: Thinking about the implications, if these strategic elements are driving results in close competitions, then understanding this mechanism helps us see where external factors—like panel composition—are introducing non-random noise into performance evaluation. It suggests that relying solely on the judging outcome doesn't tell you the full story of what happened.

Iris: It frames the problem less as a failure of athletes or judges, and more as a structural issue in how subjective expert panels are designed and composed, especially when those panels have internal connections to the competitors.

Ruth: This paper, "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging," points out that the decisions made by expert panels aren't always just about what a judge personally likes; they can be shaped by strategic factors related to nationality and rivalry management. It shows that understanding these mechanisms is key to figuring out how to make judging systems fairer, even when they rely on human judgment.

Conclusion: Ruth: So we're wrapping up this look at "Taste or Strategy? Nationalistic Bias and Competitor Penalties in Figure Skating Judging." Basically, these authors found evidence for a strategic premium for a compatriot and targeted penalties against rivals.

Warren: It’s about moving past just thinking judges are biased by personal taste and showing that there's a real strategy going into how those panels are put together.

Iris: The paper looked at the data from figure skating competitions and found that these biases aren't just random errors; they have a structure to them based on the panel composition.

Ruth: Exactly. They used this rich dataset to show how things like random start orders and whether rivals share judges’ federations actually influence those scores in ways we didn't expect.

Warren: And the results showed that these three things—the compatriot premium, the competitor penalty, and what they called the colleague-compatriot premium—all work together based on how the judging panel is drawn randomly. It’s a system where all three factors interact to shape competition outcomes.

Iris: The methodology was pretty solid in showing that this joint influence is what drives the variation we're looking at, which helps identify what actually matters in terms of statistical identification.

Ruth: So, what does this mean for the world outside of figure skating? It suggests that even when you think you’re getting a purely objective score from judges, there can be these systemic biases baked into how those panels are selected.

Warren: If these strategic elements are driving results in close events, then current rules might just be missing something because they only remove a small piece of whatever premium or penalty is actually there.

Iris: The authors suggest keeping the federations of leading skaters off judging panels when it’s feasible, or seating judges symmetrically—either all on one segment or none—for all judges on a segment, either for all judges on a segment or none.

Ruth: They want us to use statistical tools and judge marking records to actually monitor judge behavior and find ways to reduce those intentional strategic components of the premium.

Warren: It shifts the focus from just trusting machine judging to actively managing how the panel itself is formed or operated when you're dealing with these kinds of biases.

Iris: The paper shows that even in a subjective system, there’s an underlying mechanism where connections between people and judges can introduce a measurable, strategic effect on performance evaluation.

Ruth: Next up, we’ll look at how this kind of structural bias might show up in other high-stakes judging environments.

More episodes

← Home