LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting
cs.CL
Submitted: 2026-11-04
Updated: 2026-11-04
Project page: https://ranaroussi.github.io/yfinance/.3https://fred.stlouisfed.org/series/DFEDTAR
Terminology
Sources
- Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
- Scaling Open-Ended Reasoning to Predict the Future
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Detecting Lookahead Bias in LLM Forecasts
- Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis
- Approaching Human-Level Forecasting with Language Models
- Chronologically Consistent Large Language Models
- Is the Future Predictable? Modeling Future Predictability for Reliable Temporal Reasoning
- ChatGPT and Corporate Policies
- HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities
- Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models
- The Memorization Problem: Can We Trust LLMs' Economic Forecasts?
- OracleProto: A Reproducible Framework for Benchmarking LLM Forecasting via Knowledge Cutoffs and Temporal Masking
- GPT-4o System Card
- Pitfalls in Evaluating Language Model Forecasters
- Herculean: An Agentic Benchmark for Financial Intelligence
- LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models
- When Agents Trade: Live Multi-Market Trading Benchmark for LLM Agents
- AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrieval
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data