In-Place Indic Tokenizer Expansion and Continual Pretraining of Nemotron-3-Nano-30B-A3B
cs.CL, cs.AI
Submitted: 2026-10-11
Updated: 2026-10-11
Sources
- FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
- Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- In-Place Tokenizer Expansion for Pre-trained LLMs
- The Art of Breaking Words: Rethinking Multilingual Tokenizer Design
- DataComp-LM: In search of the next generation of training sets for language models
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages
- Decoupled Weight Decay Regularization
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
- MILU: A Multi-task Indic Language Understanding Benchmark
- IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
- IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
- Measuring Massive Multitask Language Understanding
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- HellaSwag: Can a Machine Really Finish Your Sentence?
- The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
- Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
- IndicParam: Benchmark to evaluate LLMs on low-resource Indic Languages
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering