Opt.Gear Technical Report
Juneyoung Park, Youngwook Kwon
cs.CL
Submitted: 2026-08-02
Comments: [OptAI] OptGear Model Technical Report
Code: https://github.com/huggingface/nanotron
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: We introduce OptGear, a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability.
Terminology
Abstract
We introduce OptGear, a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability. It includes a dense model (1M, 270M, and 1B) with a context length of 64K. We designed a new hybrid architecture that combines a convolutional key-value gated mixer with local-global attention to reduce the KV-cache memory that tends to increase exponentially with long context. This architecture delivers up to X4.9 faster prefill and decoding speeds on the NPUs compared to models of a similar scale models. From a 2T tokens candidate corpus, OptGear is trained on a curated 0.5T tokens subset without knowledge distillation. This is the most data-efficient of the existing foundation models. All models are released with open weights and deployment binaries for ONNX, Qualcomm NPU, and Apple ANE making OptGear a practical base for edge applications that need fast, memory-efficient inference and strong task capabilities. Furthermore, to expand the ecosystem of on-device generative language models, we are introducing the OptGear-1M that can be deployed on Micro-Controller Units (MCUs), a Tiny Language Model (TLM). OptGear-1M is the first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU of the STM32H747I-DISCO.
Sources
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- LFM2 Technical Report
- EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes
- Generating Long Sequences with Sparse Transformers
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Scaling Vision Transformers to 22 Billion Parameters
- Learned Step Size Quantization
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- How to Train Long-Context Language Models (Effectively)
- The Llama 3 Herd of Models
- Measuring Massive Multitask Language Understanding
- Distilling the Knowledge in a Neural Network
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
- KOBEST: Korean Balanced Evaluation of Significant Tasks
- CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean
- KORMo: Korean Open Reasoning Model for Everyone
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- Decoupled Weight Decay Regularization
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering