MeshRT: Compile-Time Governed Wafer-Scale Runtime for Low-Latency High-Throughput Inference
cs.OS, cs.AI, cs.DC
Submitted: 2026-09-20
Updated: 2026-09-20
Code: https://github.com/NVIDIA/TensorRTLLM
Terminology
Sources
- MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens
- Generating Long Sequences with Sparse Transformers
- OpenThoughts: Data Recipes for Reasoning Models
- Dissecting the Graphcore IPU Architecture via Microbenchmarking
- Mixtral of Experts
- Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
- SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
- SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference