Shallow Encoding Emits Deep Memories
cs.LG, cs.DC
Submitted: 2026-11-04
Updated: 2026-11-04
Code: https://github.com/huggingface/open-r1
Terminology
Sources
- OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
- LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
- Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs
- Lessons from the Trenches on Reproducible Evaluation of Language Models
- Evaluating Large Language Models Trained on Code
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Accuracy is Not All You Need
- In-context Autoencoder for Context Compression in a Large Language Model
- GLM-5: from Vibe Coding to Agentic Engineering
- When Attention Sink Emerges in Language Models: An Empirical View
- OpenThoughts: Data Recipes for Reasoning Models
- Measuring Mathematical Problem Solving With the MATH Dataset
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis