On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics
summary
The gist
On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics…
In short
This study used Bayesian probabilistic projections to forecast progress toward SDG indicator 4.1.1 (minimum proficiency in reading and math) for lower secondary students using OECD PISA data. The method combines latent growth curves and model averaging to estimate rates of change, suggesting that without tailored interventions, countries are unlikely to meet their targets by 2030.
Key concepts
- Bayesian latent growth curve modeling
- This statistical technique models how proficiency levels change over time within a country. It treats the underlying rate of progress as an unknown (latent) variable that is estimated using Bayesian methods, allowing for uncertainty to be quantified in the projections.
- Bayesian model averaging (BMA)
- BMA is used to handle uncertainty in which specific growth models are best. Instead of picking one model, BMA calculates a weighted average across several potential models, where the weights reflect how likely each model is based on the data, providing a more robust prediction.
- SDG Indicator 4.1.1
- This is the United Nations Sustainable Development Goal indicator focusing on achieving minimum proficiency in reading and mathematics for students. The paper specifically examines progress toward this goal among lower secondary school children across various countries.
- Context-Input-Process-Output (CIPO) model
- The CIPO model is a framework used to select relevant indicators for national educational systems. It helps ensure that the chosen data points cover the necessary context, inputs (resources), processes (teaching methods), and outputs (student performance) needed to accurately measure educational progress.
Terminology used across episodes
This episode discusses
- On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics · Paper Radio
The paper
On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics · Read on arXiv
David Kaplan, Nina Jude, Kjorte Harra, Jonas Stampka
Department of Educational Psychology, University of Wisconsin - Madison · Institute for Bildungswissenshaft, Universität Heidelberg
As of this writing, there are five years remaining for countries to reach their Sustainable Development Goals deadline of 2030 as agreed to by the member countries of the United Nations. Countries are, therefore, naturally interested in projections of progress toward these goals. A variety of statistical measures have been used to report on country-level progress toward the goals, but they have not utilized methodologies explicitly designed to obtain optimally predictive measures of rate of progress as the foundation for projecting trends. The focus of this paper is to provide Bayesian probabilistic projections of progress to SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children. Using data from the OECD PISA, as well as indicators drawn from the World Bank, the OECD, UNDP, and UNESCO, we employ a novel combination of Bayesian latent growth curve modeling Bayesian model averaging to obtain optimal estimates of the rate of progress in minimum proficiency percentages and then use those estimate to develop probabilistic projections into the future overall for all countries in the analysis. Four case study countries are also presented to show how the methods can be used for individual country projections.
DOI: 10.1002/sd.71755
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics".
Jane: Here's the gist of the paper. On the Development of Probabilistic Projections of Country-level Progress to the UN SDG Indicator of Minimum Proficiency in Reading and Mathematics, this paper provides Bayesian probabilistic projections of progress toward SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children The focus of this paper is to provide Bayesian probabilistic projections of progress to SDG indicator 4.1.1, attaining minimum proficiency in reading and mathematics, with particular emphasis on competencies among lower secondary school children The authors employ a novel combination of Bayesian latent growth curve modeling and Bayesian model averaging to obtain optimal estimates of the rate of progress in minimum proficiency percentages They use data from the OECD PISA, as well as indicators drawn from the World Bank, the OECD, UNDP, and UNESCO The process involves first deriving an estimate of the rate of progress in minimum proficiency across countries using Bayesian latent growth curve modeling Subsequently, they apply Bayesian model averaging (BMA) to account for model uncertainty in the selection of indicators of rate of progress Framework for Growth Curve Modeling The within-country model is framed as a Bayesian hierarchical latent variable model where the trajectory is modeled by yi = Λ ηi + ϵy(i) The between-country model relates the random growth parameters to time-invariant predictors, allowing for correlation conditional on those predictors Flexible models are examined, including a simple linear growth curve model, a latent basis model with the last time point freed to reflect pandemic impact (M1), and another latent basis model where the last two terms are freed to capture pre-pandemic trends (M2) Model Uncertainty Management Bayesian decision theory is used to structure the problem of predictive modeling in the context of minimizing expected loss, where the optimal action a∗ minimizes the posterior expected loss E(y˜D) Bayesian model averaging (BMA) is employed to account for model uncertainty by taking a weighted average over a selected set of models, with weights being the posterior model probabilities (PMPs) The BMA algorithm uses a birth-death (BD) algorithm within the BMS software to sample models rather than enumerating all possible regression models Data Sources and Indicator Selection The study utilizes data from PISA 2009 to PISA 2022, focusing on the percentage of students at the country level who have reached or exceeded minimum proficiency in reading and mathematics Indicators are selected based on a Context-Input-Process-Output (CIPO) model of national educational systems, drawing from OECD, UNDP, UNESCO, and the World Bank The analysis uses difference variables between 2022 and 2009 values to estimate the rate of progress in percent minimum proficiency Projections and Findings Overall country projections show relatively small amounts of change in the percentage of students who have reached or exceeded the PISA-defined minimum proficiency level, with a projection to 2033 appearing relatively stable at about 65% Case studies illustrate that while one or two variables are deemed important, their impact on the rate of progress in minimum proficiency was quite modest The analysis suggests that without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.1c targets by 2030 Limitations and Future Research A clear limitation concerns the sample of countries used in this analysis, as it relied on data from OECD countries or those that partnered with the OECD Future research should focus on harmonizing data from ILSAs such as PISA with existing regional assessments to provide additional information The paper concludes that quality education and providing equal learning opportunities for all must be an urgent focus of education policy efforts around the globe Case Study Results In case studies, the trajectory plots reveal that for Croatian girls in reading, the projected decline is much greater with 88% of the girls at or above minimum proficiency in 2009 but projected to be at 66% by 2033 for a drop of 22% The trajectory plots also reveal that for German girls in reading, the decline is much greater at 20% compared to German boys For the United States, the projected percentage meeting or exceeding minimum proficiency for US girls in reading in 2033 is 68% representing an 18% decline relative to the baseline 2009 percentage Overall Projections Summary An average of approximately 35% of 15-year-old students across these countries will remain below minimum proficiency in reading and mathematics beyond 2030, independent of gender The BMA results indicate that the top predictor for boys in reading minimum proficiency is the change from 2009 to 2022 in the percentage of teachers in pre-primary education who are female The top predictor for girls in mathematics minimum proficiency is the change from 2009 to 2022 in the percent of children out of school of primary school age The projection to 2033 appears to be relatively stable at about 65% at or above minimum proficiency across these countries Final Conclusion Without interventions tailored to the needs of the individual countries, it seems unlikely that countries will reach their SDG 4.1.
Paper summary: Tom: To get into the thesis, this paper argues that existing statistical measures for country progress haven't used methodologies explicitly designed to get optimally predictive measures for trends >
Jane: Right. So, they introduce Bayesian probabilistic projections as a better way to forecast where these countries will be regarding reading and math proficiency >
Tom: The paper focuses specifically on lower secondary school children’s competencies, which is a key focus because that’s where the real challenges lie >
Jane: It claims this work contributes to monitoring progress for UNESCO, the OECD, the World Bank, and other educational goals by providing these sophisticated analyses at the population level >
Lu: The authors are using a combination of Bayesian latent growth curve modeling and Bayesian model averaging to get those optimal estimates of how fast these proficiency percentages are changing >
Meng: I see they’re pulling data from PISA, plus indicators from the World Bank, UNDP, OECD, and UNESCO to feed into their process >
Lalam: They derive an estimate of the rate of progress first using the latent growth curve modeling and then apply Bayesian model averaging to account for uncertainty in selecting those indicators >
Tom: So they set up a hierarchical model where you have a within-country model modeling the trajectory, and a between-country model relating random growth parameters to time invariant predictors >
Jane: They test several flexible models, including a simple linear growth curve, and two different latent basis models that are set up differently to capture things like pandemic impacts or pre-pandemic trends >
Lu: And for managing the uncertainty in those projections, they employ Bayesian decision theory to minimize expected loss by finding an optimal action >
Meng: They use Bayesian model averaging by taking a weighted average over a selected set of models, where the weights are the posterior model probabilities, or PMPs >
Lalam: The BMA algorithm samples models using a birth-death algorithm inside the BMS software instead of trying to list every possible regression model >
Conclusion: Tom: So, looking at the title, "On the Development of Probabilistic Projections," this isn't just a static forecast; it’s about giving you a probability distribution for where progress might land >
Jane: And the authors are David Kaplan and his team, who are putting this methodology forward to help monitor those international education goals >
Tom: The implication is that without tailoring interventions to the specific needs of individual countries, reaching those SDG four point one point one targets by two thousand thirty seems unlikely >
Jane: It suggests that the current situation, even with global efforts, isn't leading to the necessary shifts in a predictable way for everyone >
Lu: The paper shows that while some variables might seem important in case studies, their actual impact on the rate of progress is often quite modest when you look at the overall trends >
Meng: From a deployment standpoint, this means that broad, one-size-fits-all educational policies might not be hitting the mark unless they are very precisely targeted to what’s happening locally >
Lalam: The overall projection summary shows that an average of approximately thirty-five percent of fifteen-year-old students across these countries will remain below minimum proficiency in reading and mathematics beyond two thousand thirty regardless of gender >
Tom: And the top predictors they found for boys in reading proficiency is the change from two thousand nine to two thousand twenty-two in the percentage of teachers who are female > <ref:2511.06107#pg1>
Jane: That’s a concrete data point showing that even specific demographic shifts have a measurable effect on these projections, but overall, we still need focused action >
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates