Conflict Forecasting via Conformal Prediction for Markov Processes

summary

The gist

Whether or not a country is at war, or experiencing escalating or deescalating levels of conflict, has massive ramifications on a country’s national and foreign policy.

In short

The paper uses conformal prediction to forecast future conflict state-sequences based on temporal data from a discrete-state Markov process. It compares this method against likelihood-based prediction, showing that CP provides better calibrated uncertainty quantification for long forecasts, while likelihood methods can over-cover.

Key concepts

Conformal Prediction (CP)
CP is a method used to create prediction sets that guarantee a certain level of coverage for the true underlying data distribution without needing to perfectly specify the statistical model. It achieves this by requiring only an exchangeability assumption, making it robust against model misspecification in conflict forecasting.
Markov Process
A Markov process models conflict dynamics as a sequence of states transitioning over time, where the next state depends only on the current state and not on the entire history. This sequential nature makes it useful for tracking how a country's conflict status evolves over time.
Prediction Sets
Instead of predicting just one future outcome (point prediction), this method constructs a set of all plausible future sequences. These sets provide a range of possible outcomes, which is crucial for decision-makers to understand the uncertainty in conflict forecasting.

Terminology used across episodes

This episode discusses

The paper

Conflict Forecasting via Conformal Prediction for Markov Processes · Read on arXiv

Department of Statistics, North Carolina State University · Department of Mathematical Sciences, The University of Texas at Dallas · Department of War Studies, Swedish Defense University · Department of Mathematics, University of Oslo

Whether or not a country is at war, or experiencing escalating or deescalating levels of conflict, has massive ramifications on a country's national and foreign policy. Given a country's history of conflict, or lack thereof, future predictions about the war-status of a country are valuable information. In this paper, we present the use of conformal prediction on temporally-dependent data to obtain prediction sets of possible future conflict state-sequences. More specifically, we compare the results of conformal prediction to a likelihood-based prediction strategy when the data are assumed to come from a discrete-state Markov process. A point-prediction may not supply sufficient information because the penalty for a wrong prediction is extreme, and so we consider a machine learning alternative that gives valid uncertainty quantification and is robust to model misspecification. In the data analysis, we present real forecasts of conflict dynamics across multiple countries. Lastly, we assess the differences between the conformal and likelihood-based prediction sets applied to training data which fall under a pervasive edge-case, not discussed in existing literature.

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Conflict Forecasting via Conformal Prediction for Markov Processes".

Jane: Here's the gist of the paper. Whether or not a country is at war, or experiencing escalating or deescalating levels of conflict, has massive ramifications on a country’s national and foreign policy.

Tom: In this paper, we present the use of conformal prediction on temporally-dependent data to obtain prediction sets of possible future conflict state-sequences.

Jane: The motivation for this work stems from the growing recognition that uncertainty quantification is equally critical in forecasting conflict dynamics, as incorrect predictions carry severe consequences and decision-makers require a calibrated sense of the range of plausible outcomes.

Tom: Early efforts focused on point predictions, but there is a growing recognition that uncertainty quantification is equally critical.

Paper summary: Tom: So we're talking about this paper here called "Conflict Forecasting via Conformal Prediction for Markov Processes," and the big idea is using conformal prediction on time-dependent data to get sets of possible future conflict sequences <ref:2604.25139#pg1>. Jane, can you lay out the core thesis for us?

Jane: Well, essentially, this paper addresses how we can predict conflict states—like whether a country is at war or escalating—when that data has a time component <ref:2604.25139#pg1>. The authors are comparing two main strategies: conformal prediction against a likelihood-based prediction strategy when the underlying data follows a discrete-state Markov process <ref:2604.25139#pg1>.

Tom: What does that comparison mean for us on the ground? Like, why not just pick one method?

Lu: Because point predictions just don't give enough information, and the penalty for being wrong is really high in conflict forecasting, so they need a way to show the range of plausible outcomes <ref:2604.25139#pg2>.

Meng: I'm wondering about that uncertainty quantification part. If we get a single forecast, how do we know if it's actually reliable when the underlying process is so messy?

Jane: Conformal prediction offers an alternative because it only needs an exchangeability assumption instead of having to perfectly specify the complex model of conflict dynamics <ref:2604.25139#pg2>. This approach produces prediction sets with guaranteed coverage for finite sample data <ref:2604.25139#pg3>.

Tom: But the paper immediately flags a problem, right? They say Markov processes aren't inherently exchangeable, which is the standard assumption for conformal methods <ref:2604.25139#pg3>. That sounds like a major hurdle.

Lu: Exactly, and they tackle this by looking at two different frameworks for Markov processes <ref:2604.25139#pg3>. The first idea is treating each state sequence as an individual observation, assuming those observations are independent and thus exchangeable <ref:2604.25139#pg3>.

Meng: So they are trying to force a form of exchangeability onto something that is supposed to be sequential and dependent over time, which is a tough spot to navigate <ref:2604.25139#pg3>.

Jane: And the second framework leverages the properties of Markov processes and permutation groups to create this notion of partial exchangeability for a single state sequence <ref:2604.25139#pg3>. That's where they build their prediction set methodology <ref:2604.25139#pg4>.

Tom: Okay, so they are trying to get a rigorous way to handle that temporal dependency while still using the principled approach of conformal prediction <ref:2604.25139#pg1>. How does this translate into actually predicting future conflict sequences?

Jane: The ultimate goal isn't just one future state sequence, but a prediction set C one-alpha for any level from zero to one, such that the true sequence is inside that set with at least a probability of one minus alpha <ref:2604.25139#pg5>. They compare this against the likelihood-based approach which constructs sets based on highest probability mass <ref:2604.25139#pg4>.

Paper summary: Lu: The likelihood-based method just fits a Markov model to the calibration data and picks the highest probability mass set, defined by finding all future sequences that have a probability greater than or equal to one minus alpha <ref:2604.25139#pg4>. It's straightforward math, but it relies heavily on having the correct model <ref:2604.25139#pg1>.

Meng: If that model is wrong, the likelihood-based prediction set can get really wide when things get uncertain <ref:2604.25139#pg4>. That sounds risky when you're trying to deploy a forecast in a real system <ref:2604.25139#pg7>.

Tom: And the simulation study showed something specific about this comparison, right? How did the two methods actually perform against each other?

Jane: Figure two shows that conformal prediction remains well calibrated for all confidence levels and test lengths, which aligns with what they saw in previous work <ref:2604.25139#pg7>. In contrast, the likelihood-based approach tends to over-cover as both the test length and confidence level decrease <ref:2604.25139#pg7>.

Lu: Look at Table one which compares the size of these prediction sets for different forecast lengths and target coverage levels <ref:2604.25139#pg8>. On average, the likelihood-based prediction sets contain fewer predicted state sequences than the conformal prediction sets, except in one specific case where forecast length is one and the target coverage level is zero point five five <ref:2604.25139#pg8>.

Meng: So for a typical scenario, CP gives us more options to consider than the likelihood method does in terms of the number of sequences it suggests <ref:2604.25139#pg8>. What about when the input data is very simple?

Tom: That brings up their limitations, I think. They point out that conformal prediction isn't as efficient as the likelihood-based prediction when all of the observed states in the calibration data are actually the same state <ref:2604.25139#pg7>.

Jane: That inefficiency falls dramatically when the last state in your test sequence is different from the dominant state seen during training <ref:2604.25139#pg7>. They also specifically noted that for a special case where all calibration data is just one state, CP is less efficient than the likelihood-based prediction <ref:2604.25139#pg7>.

Lu: The authors even suggest a workaround for that specific inefficiency by adding an artificial time point at the end of the sequence <ref:2604.25139#pg8>. That’s a bit of a hack, but it shows they are trying to find where CP breaks down in efficiency <ref:2604.25139#pg8>.

Meng: From an engineering standpoint, that sounds like something we'd have to build special handling for if we were deploying this at scale with very simple input data <ref:2604.25139#pg8>. It’s not a clean solution yet.

Tom: So to wrap up the summary of "Conflict Forecasting via Conformal Prediction for Markov Processes," the paper shows that conformal prediction sets maintain coverage across all forecasted steps, which is important because it preserves the temporal dynamics that matter in many applications <ref:2604.25139#pg9>.

Paper summary: Jane: That's true, and unlike likelihood-based prediction sets which might collapse to a stationary distribution when you project many steps ahead, conformal prediction is better positioned to capture the true trajectory uncertainty <ref:2604.25139#pg9>.

Lu: So the paper really advocates for CP because it’s better at reflecting genuine trajectory uncertainty rather than just fixed state probabilities <ref:2604.25139#pg9>. It’s about capturing the dynamics, not just a static probability distribution <ref:2604.25139#pg9>.

Meng: If we look at the implications for deployment, it suggests that for forecasting sequences, uncertainty quantification matters more than just getting one best guess <ref:2604.25139#pg1>. We need sets of possibilities <ref:2604.25139#pg7>.

Tom: Right, so the authors are basically arguing that for complex, time-dependent processes like conflict states, we need robust methods that don't rely on assuming perfect model accuracy <ref:2604.25139#pg2>. They give us a way to get calibrated sets of plausible outcomes <ref:2604.25139#pg1>.

Jane: And the comparison with likelihood methods shows that while they have some efficiency trade-offs, conformal prediction seems more reliable for capturing the actual evolution of the sequence over time <ref:2604.25139#pg9>. That’s a key difference for anyone trying to use this in policy decisions <ref:2604.25139#pg1>.

Lu: It’s about moving beyond just point predictions and giving decision-makers a calibrated range of outcomes that respect the temporal nature of the data <ref:2604.25139#pg5>. That's the core contribution here <ref:2604.25139#pg1>.

Meng: So, for production systems, it means we need to be careful about how we define our uncertainty because a simple model fitting approach might give us a deceptively narrow view of what could actually happen <ref:2604.25139#pg8>.

Tom: Indeed, this paper on "Conflict Forecasting via Conformal Prediction for Markov Processes" suggests that using conformal prediction to build prediction sets is a principled way to forecast sequences without needing a perfectly specified model <ref:2604.25139#pg1>. It’s about getting those plausible future conflict sequences right <ref:2604.25139#pg9>.

Jane: We should keep an eye on how these methods evolve, especially when dealing with more complex dependencies or when we move beyond the discrete-state Markov process that they tested <ref:2604.25139#pg3>. That’s where the next round of research will likely be needed <ref:2604.25139#pg3>.

Lu: Definitely, because handling those violations of exchangeability in a way that remains efficient is still an open problem in this area <ref:2604.25139#pg3>.

Meng: And for us, it means if we build forecasting tools, we have to account for the possibility that our simplest models might be misleading when they're projecting long sequences <ref:2604.25139#pg7>.

Tom: That’s the practical implication—uncertainty quantification isn't just academic; it directly affects how decisions are made about national security and foreign policy <ref:2604.25139#pg1>.

Jane: So, to wrap up this discussion on "Conflict Forecasting via Conformal Prediction for Markov Processes," the paper gives us a tool for generating prediction sets of possible future conflict state sequences using conformal prediction <ref:2604.25139#pg1>. It shows that CP is better at capturing temporal dynamics than likelihood-based approaches when projecting many steps ahead [ref:two thousand six hundred four point two five one three nine#pg9

Conclusion: Tom: So we're wrapping up this paper "Conflict Forecasting via Conformal Prediction for Markov Processes" <ref:2604.25139#pg9>. It’s about using conformal prediction to get sets of possible future conflict sequences <ref:2604.25139#pg1>.

Jane: Right, it's taking something that changes state over time—like war status—and giving us a range of what could happen <ref:2604.25139#pg5>. The authors are really comparing this conformal prediction approach against a likelihood-based method <ref:2604.25139#pg4>.

Lu: What they’re showing is that the conformal prediction sets keep their coverage across all the future steps <ref:2604.25139#pg7>. That's important because it preserves the actual temporal movement of the conflict state <ref:2604.25139#pg9>.

Meng: But they also admitted a limitation, which is that if all their training data is just one state, conformal prediction isn't as efficient as the likelihood-based method <ref:2604.25139#pg7>. They even have to add an extra time point to fix that <ref:2604.25139#pg8>.

Tom: So, the big picture here is that for forecasting conflict dynamics, we need methods that respect those temporal dependencies and provide a range of plausible outcomes <ref:2604.25139#pg1>.

Jane: Exactly. Likelihood-based methods can sometimes collapse to just fixed probabilities when you look way out into the future, which isn't helpful for understanding actual trajectory uncertainty <ref:2604.25139#pg9>.

Lu: The authors argue that conformal prediction is better at capturing that true trajectory uncertainty over many steps <ref:2604.25139#pg1>. It's about the sequence itself, not just a static probability distribution <ref:2604.25139#pg9>.

Meng: For me, that means if I’m building a system to predict conflict, I can't just pick the simplest model because it might be missing crucial long-term shifts in the state <ref:2604.25139#pg7>.

Tom: So, the paper suggests that using conformal prediction provides a principled way to get those calibrated sets of plausible outcomes for conflict sequences without needing a perfectly specified model <ref:2604.25139#pg1>. It's about getting those future conflict sequences right <ref:2604.25139#pg9>.

Jane: That’s the main message, and it sets up the next question: can this work in real-world policy decisions where time is measured in human terms?

More episodes

← Home