Validation of Satellite Lifetime Predictions at Leonid Space

summary

The gist

Leonid Space provides real-time satellite lifetime predictions for low Earth orbit missions by validating its prediction pipeline through comprehensive backtesting against 1283 deorbited satellites…

In short

Leonid Space validated its real-time satellite lifetime predictions using backtesting against 1283 deorbited satellites from 1968 to mid-2026. The validation used four progressive scenarios, showing that improved solar cycle forecasting and density correction significantly enhance prediction accuracy, demonstrating operational readiness for commercial use.

Key concepts

Perfect Knowledge Scenario
This simulation assumes the system has perfect information, using actual space weather data and density corrections from the final year of flight. It serves as the gold standard to establish the theoretical maximum performance achievable by the prediction pipeline.
Historical Atmosphere Scenario
This test uses actual space weather data but estimates density scale factors based on observations from 30 days before simulation start. This tests how well the system can handle imperfect, near-term atmospheric conditions.
Fully Predictive Scenario
This is the most challenging simulation, using forecasted space weather and predicted density scale factors. It proves the system's robustness by showing that solar cycle forecasting becomes the dominant factor in prediction accuracy.
Ballistic Coefficient Modeling (OU Process)
The system models ballistic coefficients as Ornstein–Uhlenbeck processes in log space. This mathematical approach ensures that the modeled coefficient remains positive and maintains multiplicative symmetry, preventing integration errors over time.

Terminology used across episodes

This episode discusses

The paper

Validation of Satellite Lifetime Predictions at Leonid Space · Read on arXiv

Scott Shambaugh

Leonid Space

This paper validates Leonid Space's satellite lifetime prediction pipeline through a backtesting campaign of 1283 deorbited satellites spanning six decades of spaceflight, showing that it accurately recreates their last year of flight. The pipeline combines a custom high-speed orbit propagator, ballistic coefficient estimation from on-orbit TLE data, atmospheric density corrections from data assimilation, custom space weather forecasts, and probabilistic Monte Carlo analysis. Validation proceeds in four steps that progressively remove the benefit of hindsight: starting from a "perfect knowledge" scenario using actual space weather data and ephemerides, we gradually replace historical data with contemporary predictions to arrive at a fully predictive scenario analogous to a deorbit prediction performed for a spacecraft flying today. These forecasts are scored using the continuously ranked probability score and root mean square calibration error, yielding a median one-year prediction accuracy of 2.4 days (0.7%) with perfect knowledge and 35.3 days (9.7%) in the fully predictive scenario. Solar flux forecasts contribute 75% of the variance in our predictions, dominating all other sources of error. To our knowledge, this is the first comprehensive backtesting of long-term satellite lifetime predictions that reports quantitative results for the fully predictive case, and it establishes a new framework for scoring solar cycle forecasts by grounding forecast accuracy in operationally-relevant satellite lifetime predictions.

Transcript

Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.

Vera: We're looking at the paper: "Validation of Satellite Lifetime Predictions at Leonid Space".

Jocelyn: Here's the gist of the paper. Leonid Space provides real-time satellite lifetime predictions for low Earth orbit missions by validating its prediction pipeline through comprehensive backtesting against 1283 deorbited satellites that deorbited between 1968 and mid-2026, demonstrating operational readiness for commercial deployment.

Vera: The validation process employs a three-stage approach designed to progressively remove hindsight to arrive at fully predictive operational conditions.

Jocelyn: This methodology involves simulating the final year of flight and comparing predicted deorbit dates against actuals.

Vera: The validation scenarios are structured as follows: 1.

Title and authors: Vera: Okay, so what they're summarizing is that this validation process used three stages to remove hindsight and get to fully predictive conditions. They started with perfect knowledge using actual space weather data and density corrections from the last year of flight, then moved to historical atmosphere, and finally simulated with forecasted space weather.

Jocelyn: That progression is smart; it shows how the models evolve as you move from knowing everything to just predicting based on what you know now. It’s a way of proving that their forecasting pipeline can handle the uncertainty of future conditions.

Subrahmanyan: From a theoretical standpoint, this method is essentially testing if the coupling between solar cycle variations and orbital decay is captured correctly across different levels of input fidelity. If it works well in the 'Fully Predictive' scenario, it suggests that solar cycle forecasting dominates all other error sources and higher fidelity propagators are of low importance <ref:2601.02453#pg4>.

Vera: It really points to the fact that the solar cycle forecast is doing most of the heavy lifting here. The paper explicitly states that in the fully predictive case, solar cycle forecasting dominates all other error sources and higher fidelity propagators are of low importance <ref:2601.02453#pg4>.

Jocelyn: So for someone listening who doesn't deal with orbital mechanics, it means the system is basically saying that getting the space weather prediction right is more important than having a super-precise drag calculation for every single satellite at every moment.

Subrahmanyan: That makes sense because we know the density changes are massive; they are two to three times higher than what pre-cycle consensus predicted <ref:2601.02453#pg3>. It's about grounding operational decisions in that statistically superior space weather prediction <ref:2601.02453#pg1>.

Vera: And I also want to talk about the improvements they made, because that’s where the real technical depth is. They didn't just run a test; they built better tools for the test itself.

The paper's summary: Jocelyn: Right, so beyond just running the backtest, there are several specific technical improvements they introduced to make their prediction pipeline more robust. One is improving solar cycle forecasting integration.

Vera: They took their internal solar flux forecast models and scored them against fourteen other public archives, and they outperformed them across all time horizons up to five years <ref:2601.02453#pg2>. That’s a lot of comparison work.

Subrahmanyan: Outperforming other models in that way is significant because it grounds operational decisions in a space weather prediction that "outperforms all others" <ref:2601.02453#pg1>. It suggests the new scoring mechanism is genuinely better at capturing the complex physics of solar activity.

Jocelyn: Then there’s the density correction improvement, which they did by generating a first-order global density scale factor by correlating per-satellite scale factors at a three-hour cadence, similar to what Space Force does <ref:2601.02453#pg2>.

Vera: That correlation is powerful because it leads to a reduction in average drag variance since one thousand nine hundred seventy by seventy percent and over the past decade by about ninety percent <ref:2601.02453#pg2>. That level of reduction in error is substantial for long-term mission planning.

Subrahmanyan: And the modeling of ballistic coefficients is also getting tighter; they model these as Ornstein–Uhlenbeck processes in log space, which ensures the dispersed ballistic coefficient at every timestep t remains strictly positive and has multiplicative symmetry over its distribution <ref:2601.02453#pg2>. That prevents constant bias from integrating up over time.

Jocelyn: That mathematical modeling of the process noise is what’s making their results more stable, which I think is crucial for someone trying to predict long-term orbital decay reliably.

Vera: And they also improved uncertainty quantification by using a global dispersion factor g tuned to minimize the Root Mean Square Calibration Error, or RMSCE, for each scenario <ref:2601.02453#pg2>. They showed that this optimal global drag dispersion g rose from zero percent in the perfect knowledge scenario up to thirty-one percent when the ballistics coefficient was estimated in the historical atmosphere <ref:2601.02453#pg2>.

The paper's improvements: Subrahmanyan: So, to wrap up these improvements, what we see is that they’ve established a performance baseline by simulating four progressive scenarios, starting with "Perfect Knowledge" and moving all the way to a "Fully Predictive" retrospective forecast. This shows that solar cycle forecasting dominates all other error sources and higher fidelity propagators are of low importance in the fully predictive case <ref:2601.02453#pg4>.

Jocelyn: It confirms that when you have the best input data—the forecasted space weather and predicted density scale factors—the prediction pipeline is very reliable for predicting deorbit dates. But we have to remember they also flagged their limitations.

Vera: They did mention that the method they used to estimate density correction, by correlating scale factors at a three-hour cadence, is based on a method similar to Space Force’s HASDM <ref:2601.02453#pg2>. So while it works well for reducing variance, it still depends on that correlation being accurate.

Subrahmanyan: That's the caveat; the accuracy of the density scale factor correction is what underpins a lot of their performance gains in terms of reducing drag variance <ref:2601.02453#pg2>. It’s not a perfect fix, just a very powerful one.

Jocelyn: What this means for us listening who aren't in the orbital mechanics world is that we can trust the real-time lifetime predictions they provide because they have rigorously tested how well those predictions hold up against actual deorbit data over several solar cycles.

Vera: Exactly; it’s a strong validation of their operational readiness. The paper "Validation of Satellite Lifetime Predictions at Leonid Space" shows a system that is ready for commercial deployment because it has been tested against nearly twelve hundred satellites from one thousand nine hundred sixty-eight to two thousand twenty-six <ref:2601.02453#pg1>.

Subrahmanyan: It’s important to see this kind of comprehensive backtesting, especially given the real impact drag is having on constellations now <ref:2601.02453#pg3>. We need these kinds of methods to figure out the long-term implications for where we place satellites in orbit.

Jocelyn: Well, that's what we had on this validation paper. It’s a solid piece of work showing how to build a reliable system when you’re dealing with complex, evolving space weather effects.

Conclusion: Jocelyn: So, we just looked at "Validation of Satellite Lifetime Predictions at Leonid Space," and basically, they ran a big test to see if their prediction pipeline actually works for real missions.

Vera: Right, they used one thousand two hundred eighty-three deorbited satellites from one thousand nine hundred sixty-eight until mid-two thousand twenty-six to check how accurate their lifetime predictions are when you throw in the real data.

Subrahmanyan: It’s about testing the whole system against known outcomes, which is necessary before you think it can handle a live constellation.

Jocelyn: They used a three-stage validation method to strip away any guesswork, starting with perfect knowledge and moving to something fully predictive using forecasts.

Vera: That progression shows how the model handles changing information, from knowing everything now to just predicting what's coming based on forecasts.

Subrahmanyan: The core finding seems to be that the solar cycle forecasting part actually dominates all the error sources in those more complex scenarios.

Jocelyn: Yeah, they found that even when you use forecasted data and predicted density factors, the solar cycle forecast is what's doing most of the heavy lifting for accuracy.

Vera: That means if you want to improve predictions, focusing on better space weather forecasting is probably a bigger win than trying to get every single drag coefficient calculation absolutely perfect at every tiny time step.

Subrahmanyan: I agree with that, because the density changes themselves are huge; they’re two to three times higher than what previous consensus predicted <ref:2601.02453#pg3>. Grounding it in that superior space weather prediction is key.

Jocelyn: That makes sense for someone just listening—it means the system is betting its accuracy on knowing the sun’s activity better than anything else.

Vera: Exactly, and they did some serious engineering work too, like using data assimilation to generate a global density scale factor by correlating those per-satellite factors <ref:2601.02453#pg2>.

Subrahmanyan: That correlation method is powerful because it leads to a reduction in average drag variance since one thousand nine hundred seventy by seventy percent and over the last decade by about ninety percent <ref:2601.02453#pg2>. That’s a massive reduction in uncertainty.

Jocelyn: And they modeled the ballistic coefficients using Ornstein–Uhlenbeck processes in log space, which keeps that dispersion strictly positive and symmetric over time <ref:2601.02453#pg2>. That prevents that constant bias issue from building up over the integration time.

Vera: Plus, they tuned a global dispersion factor g to minimize the Root Mean Square Calibration Error for each scenario, and you saw it rise from zero percent in the perfect knowledge case up to thirty-one percent in the historical atmosphere <ref:2601.02453#pg2>.

Subrahmanyan: The limitation they point out is that this whole method relies on that density scale factor correlation being accurate, which is a big assumption for real deployment <ref:2601.02453#pg4>.

Jocelyn: So, overall, the Leonid Space validation paper confirms the system’s operational readiness by showing it can handle those complex uncertainty levels through rigorous backtesting.

Vera: It’s a solid piece of work demonstrating how to build a reliable system when you’re dealing with evolving space weather effects.

Subrahmanyan: It shows that for long-term mission planning, grounding the prediction in high-fidelity solar cycle forecasts is the most impactful step we can take right now <ref:2601.02453#pg1>.

Jocelyn: We’ll keep an eye on how these improvements translate into live operational use and see if they hold up over the next few solar cycles.

Vera: Next time, we're looking at how this relates to those mock observations for the CSST mission, CPI-C.

More episodes

← Home