Compact Greek Language Models from Scratch: Pretraining 500M- and 1B-Parameter Decoder-Only Transformers
cs.CL
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/Telefonica-Scientific-Research/EuroAlpaca
Terminology
Sources
- Meltemi: The first open Large Language Model for Greek
- Krikri: Advancing Open Large Language Models for Greek
- TinyLlama: An Open-Source Small Language Model
- FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- GLU Variants Improve Transformer
- Training Compute-Optimal Large Language Models
- Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data