OrBIT: Structure-Guided Embedding Compression
OrBIT is a structure-guided method for compressing large language model embedding tables by discovering reusable local geometry from orbit dynamics. Instead of fixing compression structures, OrBIT learns local patterns to constrain shared codewords, directing global residual allocation, and using redundant charts for error correction. It achieves significant compression gains over existing methods while maintaining competitive rate-distortion performance.