PTC-Decoder: Towards Intelligent SLMs on Offline Resource-Constrained Edge Devices
cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Code: https://github.com/yuminghui/llm-toolconstrained-decoder
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge: their limited reasoning capacity hinders
Terminology
Abstract
Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge: their limited reasoning capacity hinders reliable execution of multi-step agent tasks requiring complex tool orchestration. Existing plan-solve paradigms rely on prompt-based enforcement, which our experiments show SLMs almost entirely disregard: weak models fail to invoke the plan. We propose PTC-Decoder (Plan-Tool Constrained Decoder), a training-free, plug-and-play decoder framework that combines (1) a Plan-to-Act paradigm, which elevates planning to an atomic tool and forces its invocation at the first inference step, and (2) TC-Decoder, a deterministic finite automaton that imposes token-level hard constraints on tool names while preserving freedom over parameter generation, thereby retaining SLM reasoning capability. Evaluated on 200 real remote-sensing satellite tasks across 7 SLMs, PTC-Decoder yields a statistically significant mean overall score gain of +1.21 (p<0.01), 95% CI [+1.13, +1.29]), with consistent improvements across models and other datasets. An ablation study that removes TC-Decoder causes substantial performance degradation across all quality metrics without reducing computational cost, confirming TC-Decoder as the primary driver. PTC-Decoder thus offers a lightweight yet effective solution for improving step-level reliability, with final-answer accuracy remaining an open challenge. In essence, we enforce plan adherence by constraining the permissible output vocabulary during inference, without requiring retraining.
Sources
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Lightweight Transformer Architectures for Edge Devices in Real-Time Applications
- Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs
- OnionEval: An Unified Evaluation of Fact-conflicting Hallucination for Small-Large Language Models
- ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases
- Gemma 4 Technical Report
- Gemma 3 Technical Report
- Near-realtime Earth Observation Via Starlink LEO Satellite Constellation
- Qwen3 Technical Report
- SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
- A Survey of Large Language Models
- CE-LSLM: Efficient Large-Small Language Model Inference and Communication via Cloud-Edge Collaboration
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems