When Can Prefixes Compile LoRA? Exact Resource-Capped Tests for Frozen Attention
cs.LG
Submitted: 2026-09-29
Updated: 2026-09-29
Sources
- Learning without training: The implicit dynamics of in-context learning
- ReasonCACHE: Teaching LLMs To Reason Without Weight Updates
- Training-Free Universal Approximation by Prompting Random Transformers
- Adaptive Task Vectors for Large Language Models
- Latent Context Compilation: Distilling Long Context into Compact Portable Memory
- Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning
- SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
- Transmuting prompts into weights
- Memory Limitations of Prompt Tuning in Transformers
- A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
- In-context Learning and Induction Heads
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks