Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services
cs.DC, cs.NI, cs.PF
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 13 pages, 7 figures; 41-page supplementary material in ancillary files. Submitted to IEEE Transactions on Services Computing. Data and code: https://doi.org/10.5281/zenodo.23792887
Code: https://github.com/Azure/AzurePublicDataset
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Request-Level Energy Attribution for Batched LLM Serving
- Ensuring Fair LLM Serving Amid Diverse Applications
- AI Token Futures Market: Commoditization of Compute and Derivatives Contract Design
- Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation
- Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
- Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling
- SLOs-Serve: Optimized Serving of Multi-SLO LLMs
- Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving
- Equinox: Holistic Fair Scheduling in Serving Large Language Models
- Justitia: Fair and Efficient Scheduling of Task-parallel LLM Agents with Selective Pampering
- Token Inflation: How Dishonest Providers Can Overcharge for Large Language Model Usage
- Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs
- Which Model Is Actually Serving You? IRIS: Budgeted Black-Box Auditing of Model Substitution and Routing Dilution in LLM Gateways
- The Tokenizer Tax: Quantifying and Explaining the Cross-Lingual Cost of Subword Tokenization for Indian Languages
- How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference
- The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation
- MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation
- BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends