Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learning
QCOC compresses tabular In-Context Learning examples by compiling their full Key-Value (KV) cache into compact memory once. It achieves near full context accuracy while drastically reducing online inference costs for repeated queries. The method groups example states using K-means and fits the resulting prototypes with value vectors derived from attention queries, effectively approximating the function q -> AD(q).