DFIP: Decision-Focused Inference Path for System One Models
cs.PF
Submitted: 2026-10-27
Updated: 2026-10-27
Comments: 11 pages, 1 figure. Code and reproducible results: https://github.com/superagent-ai/dfip
License: http://creativecommons.org/licenses/by/4.0/
The gist: System One (decision) models return small structured answers, but can retain host dispatch, GPU launch overhead and full-vocabulary projection.
Abstract
System One (decision) models return small structured answers, but can retain host dispatch, GPU launch overhead and full-vocabulary projection. We present DFIP (Decision-Focused Inference Path), an empirical systems study of established inference techniques in PPLX-Decider, Liquid d1-3B, Clef-Flash and GLiNER2.5-Decide. Selective CUDA graph replay, candidate-token projection and schema-compatible length-aware batching are applied at compatible model boundaries. We compare resident requests with stronger native baselines, audit exposed answers and measure changing arrivals. With optional native Qwen kernels installed, short Boolean resident speedups over strengthened native are 2.47x for Decider across five containers and 1.45x for Clef across four of five planned containers; Clef's fifth fails in native preflight. Liquid is effectively tied with released compilation; replay supplies most of its eager-baseline gain, while projection alone adds little full-request benefit. In a one-worker GLiNER follow-up, length-aware batching is 1.40x faster than compiled native FIFO batching on a diverse resident trace, but has worse arrival p95. Decider and Clef audits cover 3,024 decisions per model and preserve all 1,112 returned choices, while Clef changes scores, probabilities and serving confidence thresholds. Diverse external arrivals reverse fixed-shape replay gains and expose overload losses and a replay-path CUDA error. The results identify workload-specific acceleration and its limits, rather than a universal optimizer or production readiness.
Related papers
- Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100
- denet, A lightweight command-line tool for process monitoring in benchmarking and beyond
- HybridQC: Hardware-Grounded Simulation of Tightly Integrated Hybrid Quantum-Classical Systems
- TileBench: A Controlled Benchmark for Performance Evaluation and Bottleneck Diagnosis of Tile-Based Programming Models
- The M/PH 2/1 Queue: Explicit Distributions, Sojourn Times and Numerical Benchmarks
- Rolling Round-Robin Rate: Standard Heterogeneous Throughput for SPEC CPU