On the Capability and Limitation of Hard Prompt
cs.LG
Submitted: 2026-09-26
Updated: 2026-09-26
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- What's the Magic Word? A Control Theory of LLM Prompting
- In-Context Learning Creates Task Vectors
- Training Compute-Optimal Large Language Models
- An Information-Theoretic Analysis of In-Context Learning
- Scaling Laws for Neural Language Models
- GPT Understands, Too
- Memorization Capacity of Multi-Head Attention in Transformers
- Memory Limitations of Prompt Tuning in Transformers
- A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
- Prompting a Pretrained Transformer Can Be a Universal Approximator
- GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models
- Automatic Prompt Optimization with "Gradient Descent" and Beam Search
- It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
- On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
- An Explanation of In-context Learning as Implicit Bayesian Inference
- TEMPERA: Test-Time Prompting via Reinforcement Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks