What Can Language Models Inherit from Solomonoff Induction? Limits, Attainable Properties, and What Served Conditionals Reveal
cs.LG, cs.AI, cs.IT, math.IT
Submitted: 2026-09-28
Updated: 2026-09-28
Sources
- What learning algorithm is in-context learning? Investigations with linear models
- Bayesian scaling laws for in-context learning
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Theoretical guarantees on the best-of-n alignment policy
- Chinchilla Scaling: A replication attempt
- The Description Length of Deep Learning Models
- Tokenization Multiplicity Leads to Arbitrary Price Variation in LLM-as-a-service
- LLMs are Bayesian, In Expectation, Not in Realization
- In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
- The Llama 3 Herd of Models
- In-context learning and Occam's razor
- Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal reference
- Logits of API-Protected LLMs Leak Proprietary Information
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
- Enough Coin Flips Can Make LLMs Act Bayesian
- Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data
- Distilling the Knowledge in a Neural Network
- Training Compute-Optimal Large Language Models
- RULER: What's the Real Context Size of Your Long-Context Language Models?
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks