OmniCast: Interaction-Aware Serving for Real-Time Omni-Modal LLMs
cs.DC
Submitted: 2026-06-22
Updated: 2026-11-01
Code: https://github.com/sgl-project/sglang-omni
Terminology
Sources
- GPT-4 Technical Report
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility Applications
- Gemini: A Family of Highly Capable Multimodal Models
- EPD-Serve: A Flexible Multimodal EPD Disaggregation Inference Serving System On Ascend
- HunyuanImage 3.0 Technical Report
- Cornserve: A Distributed Serving System for Any-to-Any Multimodal Models
- HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
- Ming-Omni: A Unified Multimodal Model for Perception and Generation
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
- VoxServe: Streaming-Centric Serving System for Speech Language Models
- OmniGAIA: Towards Native Omni-Modal AI Agents
- Baichuan-Omni-1.5 Technical Report
- StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
- Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation
- Rocks, Pebbles and Sand: Modality-aware Scheduling for Multimodal Large Language Model Inference
- Qwen3.5-Omni Technical Report
- LongCat-Flash-Omni Technical Report
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends