Training-Only Heterogeneous Image-Patch-Text Graph Supervision for Advancing Few-Shot Learning Adapters
The TOGA framework introduces an asymmetric training-only method to improve few-shot learning adapters like Tip-Adapter. It uses a high-capacity auxiliary Heterogeneous Graph Teacher, which reasons over visual patches and text prompts to generate fine-grained class features. This knowledge is distilled into the adapter's cache during training, upgrading performance without adding any extra inference cost.