Adapting Small Language Models to Romanized Nepali: Tokenizer Fertility, Reference Likelihood, and Generation Metrics
cs.CL, cs.AI
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/pantpujan017/romanized-nepali-slm-adaptation
Terminology
Sources
- Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- Gemma 2: Improving Open Language Models at a Practical Size
- The Llama 3 Herd of Models
- A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
- Script Gap: Evaluating LLM Triage on Indian Languages in Native vs Romanized Scripts in a Real World Setting
- Qwen2.5 Technical Report
- Script Sensitivity: Benchmarking Language Models on Unicode, Romanized and Mixed-Script Sinhala
- Benchmarking Linguistic Adaptation in Comparable-Sized LLMs: A Study of Llama-3.1-8B, Mistral-7B-v0.1, and Qwen3-8B on Romanized Nepali
- Evaluating Large Language Models' Responses to Sexual and Reproductive Health Queries in Nepali
- LLaMA Beyond English: An Empirical Study on Language Capability Transfer
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data