Accelerating Cross-Model LLM Inference via Attention Map Reuse
MapFlow is a system that reuses attention maps between different, heterogeneous LLMs to speed up inference. It addresses the inefficiency of standard prefix caching when multiple models share context but differ in architecture. By transferring and adapting these attention patterns, MapFlow reduces prefilling latency by 15-20% and boosts throughput by up to 25% while maintaining generation quality.