Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness

Name: Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness
Uploaded: 2026-10-04T01:28:08.989465+00:00
Duration: 345 s
Description: The Contrastive-Context strategy balances in-context learning (ICL) and in-weights learning (IWL) by creating contrast within examples and across contexts. This method outperforms standard fine-tuning by enabling the model to switch between ICL and IWL based on how relevant the context is to the target, ensuring robust adaptation without further training.

The Contrastive-Context strategy balances in-context learning (ICL) and in-weights learning (IWL) by creating contrast within examples and across contexts. This method outperforms standard fine-tuning by enabling the model to switch between ICL and IWL based on how relevant the context is to the target, ensuring robust adaptation without further training.

Open the full episode