On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics".
Jane: Here's the gist of the paper. On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics, this paper provides Bayesian probabilistic projections of progress toward SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children The focus of this paper is to provide Bayesian probabilistic projections of progress to SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children The authors employ a novel combination of Bayesian latent growth curve modeling and Bayesian model averaging to obtain optimal estimates of the rate of progress in minimum proficiency percentages They use data from the OECD PISA, as well as indicators drawn from the World Bank, the OECD, UNDP, and UNESCO The process involves first deriving an estimate of the rate of progress in minimum proficiency across countries using Bayesian latent growth curve modeling Subsequently, they apply Bayesian model averaging (BMA) to account for model uncertainty in the selection of indicators of rate of progress Framework for Growth Curve Modeling The within-country model is framed as a Bayesian hierarchical latent variable model where the trajectory is modeled by yi = Λ ηi + ϵy(i) The between-country model relates the random growth parameters to time-invariant predictors, allowing for correlation conditional on those predictors Flexible models are examined, including a simple linear growth curve model, a latent basis model with the last time point freed to reflect pandemic impact (M1), and another latent basis model where the last two terms are freed to capture pre-pandemic trends (M2) Model Uncertainty Management Bayesian decision theory is used to structure the problem of predictive modeling in the context of minimizing expected loss, where the optimal action a∗ minimizes the posterior expected loss E(y˜D) Bayesian model averaging (BMA) is employed to account for model uncertainty by taking a weighted average over a selected set of models, with weights being the posterior model probabilities (PMPs) The BMA algorithm uses a birth-death (BD) algorithm within the BMS software to sample models rather than enumerating all possible regression models Data Sources and Indicator Selection The study utilizes data from PISA 2009 to PISA 2022, focusing on the percentage of students at the country level who have reached or exceeded minimum proficiency in reading and mathematics Indicators are selected based on a Context-Input-Process-Output (CIPO) model of national educational systems, drawing from OECD, UNDP, UNESCO, and the World Bank The analysis uses difference variables between 2022 and 2009 values to estimate the rate of progress in percent minimum proficiency Projections and Findings Overall country projections show relatively small amounts of change in the percentage of students who have reached or exceeded the PISA-defined minimum proficiency level, with a projection to 2033 appearing relatively stable at about 65% Case studies illustrate that while one or two variables are deemed important, their impact on the rate of progress in minimum proficiency was quite modest The analysis suggests that without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.1c targets by 2030 Limitations and Future Research A clear limitation concerns the sample of countries used in this analysis, as it relied on data from OECD countries or those that partnered with the OECD Future research should focus on harmonizing data from ILSAs such as PISA with existing regional assessments to provide additional information The paper concludes that quality education and providing equal learning opportunities for all must be an urgent focus of education policy efforts around the globe Case Study Results In case studies, the trajectory plots reveal that for Croatian girls in reading, the projected decline is much greater with 88% of the girls at or above minimum proficiency in 2009 but projected to be at 66% by 2033 for a drop of 22% The trajectory plots also reveal that for German girls in reading, the decline is much greater at 20% compared to German boys For the United States, the projected percentage meeting or exceeding minimum proficiency for US girls in reading in 2033 is 68% representing an 18% decline relative to the baseline 2009 percentage Overall Projections Summary An average of approximately 35% of 15-year-old students across these countries will remain below minimum proficiency in reading and mathematics beyond 2030, independent of gender The BMA results indicate that the top predictor for boys in reading minimum proficiency is the change from 2009 to 2022 in the percentage of teachers in pre-primary education who are female The top predictor for girls in mathematics minimum proficiency is the change from 2009 to 2022 in the percent of children out of school of primary school age The projection to 2033 appears to be relatively stable at about 65% at or above minimum proficiency across these countries Final Conclusion Without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.
Paper summary: Tom: To get into the thesis, this paper argues that existing statistical measures for country progress haven't used methodologies explicitly designed to get optimally predictive measures for trends >
Jane: Right. So, they introduce Bayesian probabilistic projections as a better way to forecast where these countries will be regarding reading and math proficiency >
Tom: The paper focuses specifically on lower secondary school children’s competencies, which is a key focus because that’s where the real challenges lie >
Jane: It claims this work contributes to monitoring progress for UNESCO, the OECD, the World Bank, and other educational goals by providing these sophisticated analyses at the population level >
Lu: The authors are using a combination of Bayesian latent growth curve modeling and Bayesian model averaging to get those optimal estimates of how fast these proficiency percentages are changing >
Meng: I see they’re pulling data from PISA, plus indicators from the World Bank, UNDP, OECD, and UNESCO to feed into their process >
Lalam: They derive an estimate of the rate of progress first using the latent growth curve modeling and then apply Bayesian model averaging to account for uncertainty in selecting those indicators >
Tom: So they set up a hierarchical model where you have a within-country model modeling the trajectory, and a between-country model relating random growth parameters to time invariant predictors >
Jane: They test several flexible models, including a simple linear growth curve, and two different latent basis models that are set up differently to capture things like pandemic impacts or pre-pandemic trends >
Lu: And for managing the uncertainty in those projections, they employ Bayesian decision theory to minimize expected loss by finding an optimal action >
Meng: They use Bayesian model averaging by taking a weighted average over a selected set of models, where the weights are the posterior model probabilities, or PMPs >
Lalam: The BMA algorithm samples models using a birth-death algorithm inside the BMS software instead of trying to list every possible regression model >
Conclusion: Tom: So, looking at the title, "On the Development of Probabilistic Projections," this isn't just a static forecast; it’s about giving you a probability distribution for where progress might land >
Jane: And the authors are David Kaplan and his team, who are putting this methodology forward to help monitor those international education goals >
Tom: The implication is that without tailoring interventions to the specific needs of individual countries, reaching those SDG four point one point one targets by two thousand thirty seems unlikely >
Jane: It suggests that the current situation, even with global efforts, isn't leading to the necessary shifts in a predictable way for everyone >
Lu: The paper shows that while some variables might seem important in case studies, their actual impact on the rate of progress is often quite modest when you look at the overall trends >
Meng: From a deployment standpoint, this means that broad, one-size-fits-all educational policies might not be hitting the mark unless they are very precisely targeted to what’s happening locally >
Lalam: The overall projection summary shows that an average of approximately thirty-five percent of fifteen-year-old students across these countries will remain below minimum proficiency in reading and mathematics beyond two thousand thirty regardless of gender >
Tom: And the top predictors they found for boys in reading proficiency is the change from two thousand nine to two thousand twenty-two in the percentage of teachers who are female > <ref:2511.06107#pg1>
Jane: That’s a concrete data point showing that even specific demographic shifts have a measurable effect on these projections, but overall, we still need focused action >
David Kaplan, Nina Jude, Kjorte Harra, Jonas Stampka
Department of Educational Psychology, University of Wisconsin - Madison · Institute for Bildungswissenshaft, Universität Heidelberg
stat.AP
Submitted: 2025-11-08
Updated: 2025-11-08
DOI: 10.1002/sd.71755
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 79/100
The gist: On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics…
Key concepts
- Bayesian latent growth curve modeling
- This statistical technique models how proficiency levels change over time within a country. It treats the underlying rate of progress as an unknown (latent) variable that is estimated using Bayesian methods, allowing for uncertainty to be quantified in the projections.
- Bayesian model averaging (BMA)
- BMA is used to handle uncertainty in which specific growth models are best. Instead of picking one model, BMA calculates a weighted average across several potential models, where the weights reflect how likely each model is based on the data, providing a more robust prediction.
- SDG Indicator 4.1.1
- This is the United Nations Sustainable Development Goal indicator focusing on achieving minimum proficiency in reading and mathematics for students. The paper specifically examines progress toward this goal among lower secondary school children across various countries.
- Context-Input-Process-Output (CIPO) model
- The CIPO model is a framework used to select relevant indicators for national educational systems. It helps ensure that the chosen data points cover the necessary context, inputs (resources), processes (teaching methods), and outputs (student performance) needed to accurately measure educational progress.
Terminology
Summary
On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics, this paper provides Bayesian probabilistic projections of progress toward SDG indicator 4. p. 0 On the Development of Probabilistic Projections of
Country-level Progress to the UN SDG Indicator of
Minimum Proficiency in Reading and Mathematics
David Kaplan1*, Nina Jude2
, Kjorte Harra1
, Jonas Stampka2
1*Department of Educational Psychology, University of Wisconsin -
Madison, 1025 W. Johnson St. The
focus of this paper is to provide Bayesian probabilistic projections of progress
to SDG indicator 4. 1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school
children.Where the paper says it
Where the paper says it
Methodology for Projection Estimation
The authors employ a novel combination of Bayesian latent growth curve modeling and Bayesian model averaging to obtain optimal estimates of the rate of progress in minimum proficiency percentages They use data from the OECD PISA, as well as indicators drawn from the World Bank, the OECD, UNDP, and UNESCO The process involves first deriving an estimate of the rate of progress in minimum proficiency across countries using Bayesian latent growth curve modeling Subsequently, they apply Bayesian model averaging (BMA) to account for model uncertainty in the selection of indicators of rate of progress
Framework for Growth Curve Modeling
The within-country model is framed as a Bayesian hierarchical latent variable model where the trajectory is modeled by yi = Λ ηi + ϵy(i) The between-country model relates the random growth parameters to time-invariant predictors, allowing for correlation conditional on those predictors Flexible models are examined, including a simple linear growth curve model, a latent basis model with the last time point freed to reflect pandemic impact (M1), and another latent basis model where the last two terms are freed to capture pre-pandemic trends (M2)
Model Uncertainty Management
Bayesian decision theory is used to structure the problem of predictive modeling in the context of minimizing expected loss, where the optimal action a∗ minimizes the posterior expected loss E(y˜D) Bayesian model averaging (BMA) is employed to account for model uncertainty by taking a weighted average over a selected set of models, with weights being the posterior model probabilities (PMPs) The BMA algorithm uses a birth-death (BD) algorithm within the BMS software to sample models rather than enumerating all possible regression models
Data Sources and Indicator Selection
The study utilizes data from PISA 2009 to PISA 2022, focusing on the percentage of students at the country level who have reached or exceeded minimum proficiency in reading and mathematics Indicators are selected based on a Context-Input-Process-Output (CIPO) model of national educational systems, drawing from OECD, UNDP, UNESCO, and the World Bank The analysis uses difference variables between 2022 and 2009 values to estimate the rate of progress in percent minimum proficiency
Projections and Findings
Overall country projections show relatively small amounts of change in the percentage of students who have reached or exceeded the PISA-defined minimum proficiency level, with a projection to 2033 appearing relatively stable at about 65% Case studies illustrate that while one or two variables are deemed important, their impact on the rate of progress in minimum proficiency was quite modest The analysis suggests that without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.1c targets by 2030
Limitations and Future Research
A clear limitation concerns the sample of countries used in this analysis, as it relied on data from OECD countries or those that partnered with the OECD Future research should focus on harmonizing data from ILSAs such as PISA with existing regional assessments to provide additional information The paper concludes that quality education and providing equal learning opportunities for all must be an urgent focus of education policy efforts around the globe
Case Study Results
In case studies, the trajectory plots reveal that for Croatian girls in reading, the projected decline is much greater with 88% of the girls at or above minimum proficiency in 2009 but projected to be at 66% by 2033 for a drop of 22% The trajectory plots also reveal that for German girls in reading, the decline is much greater at 20% compared to German boys For the United States, the projected percentage meeting or exceeding minimum proficiency for US girls in reading in 2033 is 68% representing an 18% decline relative to the baseline 2009 percentage
Overall Projections Summary
An average of approximately 35% of 15-year-old students across these countries will remain below minimum proficiency in reading and mathematics beyond 2030, independent of gender The BMA results indicate that the top predictor for boys in reading minimum proficiency is the change from 2009 to 2022 in the percentage of teachers in pre-primary education who are female The top predictor for girls in mathematics minimum proficiency is the change from 2009 to 2022 in the percent of children out of school of primary school age The projection to 2033 appears to be relatively stable at about 65% at or above minimum proficiency across these countries
Final Conclusion
Without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Bayesian probabilistic projections of country-level progress to UN SDG indicator 4.1.1 can be developed by combining Bayesian latent growth curve modeling with Bayesian model averaging (BMA) to obtain optimal estimates of the rate of progress, as described in the abstract and section 2.1.
-
The improved AI system can generate probabilistic projections into the future overall for all countries in the analysis, using
Bayesian model averaged predictors of the rate of progress
to account for model uncertainty, as stated in section 3 and Section 6. -
The system can perform sophisticated secondary analyses of large-scale educational assessments by utilizing Bayesian probabilistic prediction models to study educational trends at the population level, demonstrating
the richness of policy information that can be obtained when using Bayesian probabilistic prediction models to study educational trends at the population level,
as mentioned in section 2.1. -
The system can identify key drivers of progress by calculating posterior inclusion probabilities (PIPs) for predictors, which represent
the proportion of the total posterior model probability mass that includes that predictor across all models considered in the averaging process,
as detailed in section 8.2. -
The AI can provide out-of-sample predictions for minimum proficiency percentage to future years like 2033 by plugging in
the optimally predicted growth rate into our growth model and projecting the next two cycles of PISA, namely 2029 and 2033,
as described in section 7.1. -
The system can assess the quality of predictions for specific countries by comparing BMA predictive densities against unconditional growth rates, allowing examination
of problems with the prediction model and/or potential outlier countries,
as noted in section 9.2.
Abstract
As of this writing, there are five years remaining for countries to reach their Sustainable Development Goals deadline of 2030 as agreed to by the member countries of the United Nations. Countries are, therefore, naturally interested in projections of progress toward these goals. A variety of statistical measures have been used to report on country-level progress toward the goals, but they have not utilized methodologies explicitly designed to obtain optimally predictive measures of rate of progress as the foundation for projecting trends. The focus of this paper is to provide Bayesian probabilistic projections of progress to SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children. Using data from the OECD PISA, as well as indicators drawn from the World Bank, the OECD, UNDP, and UNESCO, we employ a novel combination of Bayesian latent growth curve modeling Bayesian model averaging to obtain optimal estimates of the rate of progress in minimum proficiency percentages and then use those estimate to develop probabilistic projections into the future overall for all countries in the analysis. Four case study countries are also presented to show how the methods can be used for individual country projections.
Related papers
- Unraveling time-varying causal effects of multiple exposures: integrating Functional Data Analysis with Multivariable Mendelian Randomization
- Strategic Causal Policy Learning with Manipulable Eligibility
- BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments
- Automating structural reliability analysis with a multi-agent large language model framework
- Improving Minority Population Sampling with BISG Probabilities: Evidence from a Survey of Jewish Americans
- When Do Sports Leagues Stop Being Competitive? Bayesian Detection of Competitive-Balance Eras from Match-Level Data