Mobile AI Factory: Building Specialized Language Models for Your Mobile Device
cs.CL
Submitted: 2026-11-10
Updated: 2026-11-10
Code: https://github.com/evanly-gh/SLM_Factory
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Small language models (SLMs) running on mobile devices can keep user data private and work without connectivity.
Terminology
Abstract
Small language models (SLMs) running on mobile devices can keep user data private and work without connectivity. But they perform well only when fine-tuned for the target task and sized for the target device, a process that today requires repeated expert tuning. We present Mobile AI Factory, an agentic system that builds a fine-tuned, quantized SLM for a given task and phone. Given a task specification and a target phone, an orchestrator LLM chooses whether to tune hyperparameters, mine real data, or generate synthetic examples, while fixed rules roll back changes that hurt accuracy. The factory searches from small to large models across base models and quantizations, adapts its accuracy goal as it searches, and uses real measurements of the target phone across inference engines and backends. We evaluate Mobile AI Factory on ten tasks and four Android phones. On an unseen testset, its on-device models outperform two five-shot frontier models on five tasks, by up to 0.34, and match them on three more. Its search comes within 0.03 of exhaustively training all 27 model variants on an entity extraction task, while using 19 times less GPU time. Phone measurements also matter: the best configuration changes across phones. Without phone measurements, Claude Opus 5.5, given the full hardware and model specifications, misses the app's requirements in 25% of easy and moderate cases and in 60% of hard ones. Code is available at: https://github.com/evanly-gh/SLM Factory.
Sources
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- Pioneer Agent: Continual Improvement of Small Language Models in Production
- Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference
- MobileFineTuner: A Mobile-Native Framework for On-Device LLM Fine-Tuning in Real-World Embedded AI Applications
- AIDE: AI-Driven Exploration in the Space of Code
- VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
- Towards Active Synthetic Data Generation for Finetuning Language Models
- DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
- FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents
- Skilled AI Agents for Embedded and IoT Systems Development
- Exploring Autonomous Agentic Data Engineering for Model Specialization
- Orca: Progressive Learning from Complex Explanation Traces of GPT-4
- MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases
- MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
- PostTrainBench: Can LLM Agents Automate LLM Post-Training?
- EmbedAgent: Benchmarking Large Language Models in Embedded System Development
- PowerInfer-2: Fast Large Language Model Inference on a Smartphone
- LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls
- Embedded Arena: Iterative Optimization via Hardware Feedback
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data