RW-TTT: Batched Serving for Request-Owned Test-Time Training State

Name: RW-TTT: Batched Serving for Request-Owned Test-Time Training State
Uploaded: 2026-09-05T20:28:45.047957+00:00
Duration: 440 s
Description: The episode details the paper RW-TTT, which solves the challenge of maintaining unique, mutable state for every user during large-scale AI inference. The authors introduce a system using sophisticated batching planners and owner maps to ensure data integrity while achieving high throughput (274.61 tokens/sec), proving that personalized adaptation can run efficiently at massive scale.

The episode details the paper RW-TTT, which solves the challenge of maintaining unique, mutable state for every user during large-scale AI inference. The authors introduce a system using sophisticated batching planners and owner maps to ensure data integrity while achieving high throughput (274.61 tokens/sec), proving that personalized adaptation can run efficiently at massive scale.

Open the full episode