on'ona: Twin Embeddings with Tied Unembeddings
cs.CL, cs.LG
Submitted: 2026-11-07
Updated: 2026-11-13
Terminology
Sources
- Leviathan: Decoupling Input and Output Representations in Language Models
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
- Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
- Measuring Massive Multitask Language Understanding
- MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
- Attention Residuals
- MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
- Decoupled Weight Decay Regularization
- Qwen3 Technical Report
- GLU Variants Improve Transformer
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data