Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution
cs.CL
Submitted: 2026-06-11
Updated: 2026-11-05
Terminology
Sources
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Studying Large Language Model Generalization with Influence Functions
- Distilling the Knowledge in a Neural Network
- Large-Scale Data Selection for Instruction Tuning
- LoRIF: Pushing the Scalability Frontier of Training Data Attribution
- A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
- Seq vs Seq: An Open Suite of Paired Encoders and Decoders
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data