Learning to Detect Recessions from Pseudo-out-of-sample Tests

arXiv:2506.09664 · econ.GN, q-fin.EC · Submitted 2025-06-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Learning to Detect Recessions from Pseudo-out-of-sample Tests".

Warren: Here's the gist of the paper. This paper develops an algorithm for detecting US recessions in real time by constructing hundreds of millions of recession classifiers from unemployment and vacancy data because this approach identifies all 15 historical recessions without any false positives and selects classifiers on the anticipation-precision frontier to optimize both earliness and accuracy Algorithm Construction The algorithm constructs hundreds of millions of recession classifiers by combining the unemployment and vacancy data These indicators are constructed in four steps: smoothing the data, detecting turning points, scaling variations, and combining single-variable indicators The process involves constructing 95,832 recession indicators from the unemployment and vacancy rates These indicators are created by processing unemployment and vacancy data in many different ways.

Ruth: Classifier Selection Criteria The algorithm selects perfect classifiers, which avoid both false negatives (undetected true recessions) and false positives (falsely detected recessions) between 1929 and 2021 These perfect classifiers are constructed by combining all the recession indicators with all thresholds ζ = 0.0001, 0.0002, 0.3,..., 1 In total, this procedure produces 4,481,622 perfect classifiers that detect all the recessions that occurred between 1929 and 2021 without false positives.

Warren: Optimization of Detection The algorithm selects classifiers on the anticipation-precision frontier to optimize jointly the earliness and accuracy of detection The frontier comprises the classifiers with lowest mean error (highest anticipation) and lowest standard deviation of errors (highest precision).

Ruth: The high-precision segment of this frontier is selected because it has a standard deviation of errors below 3 months.

Paper summary: Ruth: To wrap up, this paper "Learning to Detect Recessions from Pseudo-out-of-sample Tests" developed an algorithm that builds hundreds of millions of recession classifiers by combining unemployment and vacancy data to find real-time recession signals >

Warren: The core contribution seems to be the optimization process. They select perfect classifiers and then narrow them down onto the high precision segment of the anticipation-precision frontier to get earliness without losing accuracy >

Iris: It’s important that they establish this method works by showing it identifies all fifteen historical recessions from one thousand nine hundred twenty-nine to two thousand twenty-one without any false positives, and then uses a placebo test to confirm those are genuine economic signals > <ref:2506.09664#pg1>

Ruth: And the implication is that labor market conditions characteristic of a recession aren't just coming; they are already here, evidenced by the sixty-four percent probability in September two thousand twenty-five > <ref:2506.09664#pg1>

Warren: The paper suggests this approach is better than traditional methods because it filters the data more intelligently, extracting more information than just using arbitrary rules like the Michez or Sahm rule >

Iris: The method’s limitation they flag is that if you were to retrain it on a longer period, specifically including two thousand twenty-two through two thousand twenty-five only classifiers that don't detect recessions in those later years would be selected > <ref:2506.09664#pg1>

Ruth: So the paper shows how optimizing the filtering process can lead to faster and more accurate detection than older approaches like the NBER Business Cycle Dating Committee >

Conclusion: Ruth: So we've been looking at how they built this massive set of classifiers to find recessions using unemployment and job vacancy data across nearly a century, and now we're getting to the end of this paper, "Learning to Detect Recessions from Pseudo-out-of-sample Tests."

Warren: Right. The core idea here is taking those hundreds of millions of potential recession signals and really filtering them down onto this specific frontier between earliness and accuracy. It sounds like they’ve optimized a system where the cost of running that detection strategy is being carefully managed against how early they catch things.

Iris: What I see in the conclusion is that these results are quite robust because they tested it on data before, not just data after, which adds a layer of confidence to their findings about those perfect classifiers. The authors are saying this isn't some fluke; it’s systematic identification based on how the variables behave.

Ruth: Exactly. The paper argues that this method improves upon older ways of filtering labor market data because it’s not just picking arbitrary rules; they are mathematically optimizing the selection process itself to get a signal that is both early and precise.

Warren: And what this means for us, as someone who deals with trading or forecasting, is that if you can build a system like this—one that keeps the error standard deviation low, below three months—you’re looking at a detection time much faster than established bodies like the NBER Business Cycle Dating Committee.

Iris: That speed difference you mentioned earlier is key. They found this ensemble of eleven classifiers signals recessions about two point one months after they start, with an error standard deviation of one point eight months. That’s a significant improvement over that committee’s average six point three months.

Ruth: And it points to something deeper, doesn't it? The conclusion suggests that the conditions we see now in the labor market aren't just coming; they are actually here, given those sixty-four percent probabilities they calculated for September twenty twenty-five.

Warren: That’s a strong statement. They conclude that only classifiers not detecting recessions from two thousand twenty-two through two thousand twenty-five would have been selected if they had retrained the model on that extended period, which is an interesting way to test the stability of the signal.

Iris: It also flags a limitation, though. The authors state clearly that if you were to retrain this model using a longer historical window, specifically including two thousand twenty-two through two thousand twenty-five only those classifiers that failed to detect recessions in that later period would be chosen.

Ruth: So the implication is that the current structure of the labor market is already displaying characteristics of a recession according to these optimized rules, and we’re looking at a highly reliable way to track those changes in real time.

University of California, Santa Cruz

econ.GN, q-fin.EC

Submitted: 2025-06-11

Updated: 2026-10-28

Project page: http://probml.github.io/book1

Importance score: 90/100

The gist: This paper develops an algorithm for detecting US recessions in real time by constructing hundreds of millions of recession classifiers from unemployment and vacancy data because this approach

Key concepts

Recession Classifiers: These are individual models built from unemployment and vacancy data that are trained to identify whether a recession is happening. The algorithm constructs hundreds of millions of these classifiers by combining various ways of processing the raw data.
Anticipation-Precision Frontier: This is a selection strategy used to choose the best classifiers. It balances two goals: earliness (how early it detects a recession) and accuracy (how precise the detection is). The algorithm selects classifiers that offer both low mean error and low standard deviation of errors.
Ensemble Detection: Instead of relying on one classifier, the algorithm combines 11 high-precision classifiers into an ensemble. This combination creates a more reliable signal, resulting in a detection lead of about 2.1 months with a low error standard deviation.
Placebo Test Validation: The model is tested using a placebo test where it fails to detect random events, such as the deaths of US first ladies. This confirms that the algorithm is identifying genuine economic signals instead of just fitting random data patterns.

Terminology

False Negatives — 9× in this paper · explained in 2 episodes
Accuracy — 8× in this paper · explained in 1 episode
Papers — 8× in this paper · explained in 3 episodes
Overfitting — 6× in this paper · explained in 2 episodes
Adam — 3× in this paper · explained in 3 episodes
Core — 2× in this paper · explained in 2 episodes
Robustness — 2× in this paper · explained in 6 episodes
Regularization — 1× in this paper · explained in 3 episodes

Summary

This paper develops an algorithm for detecting US recessions in real time by constructing hundreds of millions of recession classifiers from unemployment and vacancy data because this approach identifies all 15 historical recessions without any false positives and selects classifiers on the anticipation-precision frontier to optimize both earliness and accuracy

Algorithm Construction

The algorithm constructs hundreds of millions of recession classifiers by combining the unemployment and vacancy data These indicators are constructed in four steps: smoothing the data, detecting turning points, scaling variations, and combining single-variable indicators The process involves constructing 95,832 recession indicators from the unemployment and vacancy rates These indicators are created by processing unemployment and vacancy data in many different ways.

Classifier Selection Criteria

The algorithm selects perfect classifiers, which avoid both false negatives (undetected true recessions) and false positives (falsely detected recessions) between 1929 and 2021 These perfect classifiers are constructed by combining all the recession indicators with all thresholds ζ = 0.0001, 0.0002, 0.3,..., 1 In total, this procedure produces 4,481,622 perfect classifiers that detect all the recessions that occurred between 1929 and 2021 without false positives.

Optimization of Detection

The algorithm selects classifiers on the anticipation-precision frontier to optimize jointly the earliness and accuracy of detection The frontier comprises the classifiers with lowest mean error (highest anticipation) and lowest standard deviation of errors (highest precision). The high-precision segment of this frontier is selected because it has a standard deviation of errors below 3 months. A final ensemble of 11 classifiers is chosen from this high-precision segment to provide a robust signal.

Performance and Validation

On average, the selected classifier ensemble signals recessions 2.1 months after their true onset, with a standard deviation of detection errors of 1.8 months This ensemble is much faster than the NBER Business Cycle Dating Committee: between 1979 and 2021, the committee takes on average 6.3 months to determine recession starts, while the classifier ensemble only takes 1.2 months. The algorithm is validated through a placebo test where it fails to detect random events like the deaths of US first ladies, confirming that the model identifies genuine economic signals rather than spuriously fitting the data.

Real-Time Application

As of September 2025, the recession probability given by the 11 classifiers is 64%, suggesting a high likelihood that a recession has begun at this point in time The algorithm's reliability is further confirmed by backtests, which show that classifier ensembles selected on limited historical data all detect out-of-sample recessions without any false positives. The final output is a single recession risk score, computed by averaging the probabilities from the 11 classifiers using equal weights.

Comparative Analysis

The paper finds that the combination of unemployment and vacancy rates outperforms industrial production in providing early and accurate recession signals The most precise labor-market classifier has a standard deviation of detection error of 1.6 months, while the most precise industrial-production classifier has a standard deviation of detection error of 2.5 months. This suggests that the unemployment-vacancy combination is better than industrial production to detect recession starts early and accurately. The algorithm also shows that labor market conditions characteristic of a recession are already here, as evidenced by the 64% probability in September 2025.

Conclusion

The paper develops a new algorithm for detecting US recessions from unemployment and vacancy data It improves upon traditional approaches like the Sahm and Michez rules, which arbitrarily determine how labor market data are filtered By optimizing the filtering process, it is possible to detect recessions earlier and more accurately. The algorithm shows that labor market conditions characteristic of a recession are not on the horizon—they are already here. Only classifiers that do not detect a recession in 2022–2025 would be selected by the algorithm if it were retrained on the extended period that includes 2022–2025 The algorithm developed in the paper shows that labor market conditions characteristic of a recession are not on the horizon—they are already here.

References

Abraham, Katharine G. 1987. “Help-Wanted Advertising, Job Vacancies, and Unemployment.” Brookings Papers on Economic Activity 18 (1): 207–248

Barnichon, Regis. 2010. “Building a Composite Help-Wanted Index.” Economics Letters 109 (3): 175–178

BLS. 2020. “Labor Force Statistics from the Current Population Survey: Overview.” https://perma.cc/RN3P-S4SL

BLS. 2023. “Labor Force Statistics from the Current Population Survey: Concepts and Definitions.” https://perma.cc/6LGU-AEU6

BLS. 2024a. “Job Openings and Labor Turnover Survey Overview Page.” https://perma.cc/

BLS. 2024b. “JOLTS Frequently Asked Questions.” https://perma.cc/22QW-ASAL

BLS. 2025a. “Civilian Labor Force Level.” FRED, Federal Reserve Bank of St. Louis

BLS. 2025b. “Job Openings: Total Nonfarm.” FRED, Federal Reserve Bank of St. Louis https://fred.stlouisfed.org/series/JTSJOL

BLS. 2025c. “Technical Documentation (CPS).” https://perma.cc/DZ9K-AJP8

Chen, Zhihong, Azhar Iqbal, and Huiwen Lai. 2011. “Forecasting the Probability of US Recessions: A Probit and Dynamic Factor Modelling Approach.” Canadian Journal of Economics 44 (2): 651–672 https://doi.org/10.1111/j.1540-5982.2011.01648.x

Crump, Richard, Domenico Giannone, and David Lucca. 2020a. “Reading the Tea Leaves of the US Business Cycle – Part One.” Liberty Street Economics, Federal Reserve Bank of New York https://perma.cc/W79A-EFPF

Crump, Richard, Domenico Giannone, and David Lucca. 2020b. “Reading the Tea Leaves of the US Business Cycle – Part Two.” Liberty Street Economics, Federal Reserve Bank of New York https://perma.cc/8CEL-TK37

Federal Reserve Board. 2025. “Industrial Production: Total Index.” FRED, Federal Reserve Bank of St. Louis https://fred.stlouisfed.org/series/INDPRO

Fornaro, Paolo. 2016. “Forecasting US Recessions with a Large Set of Predictors.” Journal of Forecasting 35 (6): 477–492 https://doi.org/10.1002/for.2388

Garimella, Rohit, Oscar Jorda, and Sanjay R. Singh. 2025. “Tracking Labor Market Stress.” FRBSF Economic Letter 2025-19 https://perma.cc/LXQ5-HSE4

Hamilton, James D. 2011. “Calling Recessions in Real Time.” International Journal of Forecasting 27 (4): 1006–1026 https://doi.org/10.1016/j.ijforecast.2010.09.001

Hatzius, Jan and Sven Jari Stehn. 2012. “Comment on ‘The Ins and Outs of Forecasting Unemployment: Using Labor Force Flows to Forecast the Labor Market’.” Brookings Papers on Economic Activity 43 (2): 118–131 https://doi.org/10.1353/eca.2012.0024

Leamer, Edward E. 2008. “What’s a Recession, Anyway?” NBER Working Paper 14221 https://doi.org/10.3386/w14221

McCracken, Michael W. and Serena Ng. 2016. “FRED-MD: A Monthly Database for Macroeconomic Research.” Journal of Business & Economic Statistics 34 (4): 574–589 https://doi.org/1080/07350015.2015.1086655

Michaillat, Pascal and Emmanuel Saez. 2015. “Aggregate Demand, Idle Time, and Unemployment.” Quarterly Journal of Economics 130 (2): 507–569 http://doi.org/10.1093/qje/qjv006

Michaillat, Pascal and Emmanuel Saez. 2022. “An Economical Business-Cycle Model.” Oxford Economic Papers 74 (2): 382–411 https://doi.org/10.1093/oep/gpab021

Michaillat, Pascal and Emmanuel Saez. 2024a. “Beveridgean Phillips Curve.” arXiv:2401.12475 https://doi.org/10.48550/arXiv.2401.12475

Michaillat, Pascal and Emmanuel Saez. 2024b. “u∗ = √uv: The Full-Employment Rate of Unemployment in the United States.” Brookings Papers on Economic Activity 55 (2) https://doi.org/10.1353/eca.2024.a964373

Michaillat, Pascal and Emmanuel Saez. 2025. “Has the Recession Started?” Oxford Bulletin of Economics and Statistics 87 (6): 1047–1058 https://doi.org/10.1111/obes.12685

Murphy, Kevin P. 2022. Probabilistic Machine Learning: An Introduction. Cambridge, MA: MIT Press http://probml.github.io/book1

NBER. 1979. “Business Cycle Dating Committee Announcement December 31, 1979.” https://perma.cc/7XKA-526Z

NBER. 1980. “Business Cycle Dating Committee Announcement June 3, 1980.” https://perma.cc/8MNK-GM2S

NBER. 1982. “Business Cycle Dating Committee Announcement January 6, 1982.” https://perma.cc/TL5X-9M8D

NBER. 2001. “Business Cycle Dating Committee Announcement November 26, 2001.” https://perma.cc/T6T8-46KQ

NBER. 2008a. “Business Cycle Dating Committee Announcement December 1, 2008.” https://perma.cc/488B-ZCGY

NBER. 2008b. “Business Cycle Dating Committee Announcement January 7, 2008.” https://perma.cc/U737-K42A

NBER. 2020. “Business Cycle Dating Committee Announcement June 8, 2020.” https://perma.cc/9HDG-A7QL

NBER. 2021. “Business Cycle Dating Committee Announcements.” https://perma.cc/JL65-XDWH

NBER. 2023. “US Business Cycle Expansions and Contractions.” https://perma.cc/B4LHBVT8

NBER. 2024. “Business Cycle Dating Procedure: Frequently Asked Questions.” https://perma.cc/32QQ-D5NW

NBER. 2025. “Business Cycle Dating.” https://perma.cc/2DQB-AUAX

Ng, Serena. 2014. “Viewpoint: Boosting Recessions.” Canadian Journal of Economics 47 (1): 1–34 https://doi.org/10.1111/caje.12070

O’Trakoun, John and Adam Scavette. 2025. “A Better Sahm Rule? Introducing the SOS Recession Indicator.” Economics Letters 247: 112175 https://doi.org/10.1016/j.econlet.2025

Petrosky-Nadeau, Nicolas and Lu Zhang. 2021. “Unemployment Crises.” Journal of Monetary Economics 117: 335–353 https://doi.org/10.1016/j.jmoneco.2020.01.009

Philips, Thomas K. 2024. “A Simple Real-Time Algorithm to Identify Turning Points in US Business Cycles.” https://ssrn.com/abstract=4897450

Piger, Jeremy. 2020. “Turning Points and Classification.” In Macroeconomic Forecasting in the Era of Big Data, edited by Peter Fuleky, chap. 18. New York: Springer

Sahm, Claudia. 2019. “Direct Stimulus Payments to Individuals.” In Recession Ready: Fiscal Policies to Stabilize the American Economy, edited by Heather Boushey, Ryan Nunn, and Jay Shambaugh, chap. 3. Washington: Brookings Institution https://www.brookings.edu/articles/direct-stimulus-payments-to-individuals

Schannep, Jack. 2008. Dow Theory for the 21st Century. Hoboken, NJ: John Wiley & Sons

Shimer, Robert. 2005. “The Cyclical Behavior of Equilibrium Unemployment and Vacancies.” American Economic Review 95 (1): 25–49 https://doi.org/10.1257/0002828053828572

Stock, James H. and Mark W. Watson. 2014. “Estimating Turning Points Using Large Data Sets.” Journal of Econometrics 178: 368–381 https://doi.org/10.1016/j.jeconom.2013.08

Sun, Jiandong, Shuaizhang Feng, and Yingyao Hu. 2021. “Misclassification Errors in Labor Force Statuses and the Early Identification of Economic Recessions.” Journal of Asian Economics 75: 101319 https://doi.org/10.1016/j.asieco.2021.101319

Weir, David R. 1992. “A Century of US Unemployment: 1890–1990. Revised Estimates and Evidence for Stabilization.” Research in Economic History 14: 301–346

Zagorsky, Jay L. 1998. “Job Vacancies in the United States: 1923 to 1994.” Review of Economics and Statistics 80 (2): 338–345 https://doi.org/10.1162/003465398557438

Appendix References

Appendix A. Announcements by the NBER Business Cycle Dating Committee

Appendix B. Classifier ensemble selected from 1929–2021 data

Appendix C. Additional results for 1965–1985 backtests

Table References

TABLE A1. NBER Business Cycle Dating Committee announcements

TABLE A2. Classifier ensemble selected in the 2015 backtest

TABLE A3. Classifier ensemble selected in the 2005 backtest

TABLE A4. Classifier ensemble selected in the 1995 backtest

TABLE A5. Classifier ensemble selected in the 1985 backtest

TABLE A6. Classifier ensemble selected in the 1975 backtest

TABLE A7.

Improvements for AI systems

Optimize recession detection by using an ensemble of classifiers selected from the anticipation-precision frontier, which optimizes jointly earliness and accuracy of detection. This system can detect recessions 2.1 months after their true onset on average between 1929 and 2021, outperforming the NBER committee's average delay of 6.3 months by only taking 1.2 months to determine recession starts.
Implement real-time risk assessment by computing a single recession risk score as the average of probabilities from the ensemble, where P(t) = 1/K. K sum k=1 P(k,t). This allows for a current assessment of economic health as seen in the September 2025 data showing a 64% probability of a recession.
Enhance policy readiness by selecting classifiers on the high-precision segment of the frontier where the standard deviation of errors below 3 months, ensuring that the 95% confidence interval for recession start dates has a width below 1 year. This provides policymakers with highly accurate timing information to implement stabilization policies rapidly.
Improve model robustness by using backtesting across multiple historical training periods (e.g., 1929-2014, 1929-2004) to ensure reliability; for instance, the classifier ensemble trained on 1929-1984 data gives the highest current recession probability of 67% in September 2025.
Develop a mechanism to handle potential future economic shifts by retraining the model: Only classifiers that do not detect a recession in 2022–2025 would be selected by the algorithm, allowing for adaptation if historical patterns change.

Sources

Beveridgean Phillips Curve