Representational drift in a code model: behaviour settles, which neurons carry it does not
cs.CL, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/piotrwilam/AtlasRepresentationalDrift
Terminology
Sources
- Emergence and Localisation of Semantic Role Circuits in LLMs
- Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining
- Word Acquisition in Neural Language Models
- Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
- Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training?
- Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler
- Evolution of Concepts in Language Model Pre-Training
- Universal Neurons in GPT2 Language Models
- Loss Landscape Degeneracy and Stagewise Development in Transformers
- Capability Emergence Can Be Forecast: Per-Seed, In Advance, With Calibrated Intervals, Certified False Alarms, and a Blind Pre-Registered Gate
- OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training
- Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
- Universal Neurons in GPT-2: Emergence, Persistence, and Functional Impact
- Instability in Downstream Task Performance During LLM Pretraining
- Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network
- Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
- How Do Large Language Models Learn Concepts During Continual Pre-Training?
- The Emergence and Sparsification of a Syntactic Circuit
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data