FORGE: Procedural Weight Generation for Sparse MoE Inference
FORGE replaces storing expert weights in high-bandwidth memory with generating them procedurally during inference. This system uses a Rotor Generation Unit to create FFN weight tiles on-the-fly from rotor parameters instead of loading dense tensors. This results in significant throughput gains and lower latency, trading off some downstream accuracy.