Explaining Attention with Program Synthesis
cs.LG, cs.AI
Submitted: 2026-06-17
Updated: 2026-10-06
Terminology
Sources
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- What Does BERT Look At? An Analysis of BERT's Attention
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- The Llama 3 Herd of Models
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- Learning Transformer Programs
- Attention is not Explanation
- Can Interpretation Predict Behavior on Unseen Data?
- Opening the AI black box: program synthesis via mechanistic interpretability
- Illuminating search spaces by mapping elites
- Towards Automated Circuit Discovery for Mechanistic Interpretability
- Is Self-Repair a Silver Bullet for Code Generation?
- Crowdsourcing Multiple Choice Science Questions
- Which Attention Heads Matter for In-Context Learning?
- TinyLlama: An Open-Source Small Language Model
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks