CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM
The episode discusses the paper CHIME, which proposes a system for efficient long-context AI inference by disaggregating attention and placing compute inside memory sticks using DIMM-PIM. The hosts explain how this approach balances memory capacity and bandwidth, leading to up to five point one five times faster performance compared to previous systems.