The Numerical Linear Algebra of Large Language Models
cs.LG, cs.NA, math.NA
Submitted: 2026-10-03
Updated: 2026-10-03
Sources
- Random Projections and Dimension Reduction
- gpt-oss-120b & gpt-oss-20b Model Card
- Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
- The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
- Layer Normalization
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
- Emergence of a High-Dimensional Abstraction Phase in Language Transformers
- Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
- The Llama 3 Herd of Models
- The State of Sparsity in Deep Neural Networks
- Which Algorithms Have Tight Generalization Bounds?
- Gemini: A Family of Highly Capable Multimodal Models
- Distilling the Knowledge in a Neural Network
- LoRA: Low-Rank Adaptation of Large Language Models
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Scaling Laws for Neural Language Models
- Measuring Intrinsic Dimension of Token Embeddings
- Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
- Adam: A Method for Stochastic Optimization
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks