Efficient LLM Adaptation with Forward-Only Passes
cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Code: https://github.com/kingoflolz/mesh-transformer-jax
Terminology
Sources
- GPT-4 Technical Report
- Gradients without Backpropagation
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- Learning without training: The implicit dynamics of in-context learning
- Dynamic Model Merging Made Slim
- NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Models
- Measuring the environmental impact of delivering AI at Google Scale
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- The Forward-Forward Algorithm: Some Preliminary Investigations
- Distilling the Knowledge in a Neural Network
- Scaling Laws for Neural Language Models
- Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
- AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Artificial Intelligence Index Report 2025
- Transmuting prompts into weights
- Layer-wise dynamic rank for compressing large language models
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- End-to-End Test-Time Training for Long Context
- Gemma: Open Models Based on Gemini Research and Technology
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents