Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering
summary
The gist
Large Language Models (LLMs) are highly sensitive to their input contexts, which motivates automated context engineering.
In short
This work shifts context engineering from finding one best global prompt to a dynamic recommendation system. It uses Neural Collaborative Context Engineering (NCCE) to iteratively build and refine a catalog of specialized contexts by treating context selection as an instance-wise routing problem, leading to better performance across various LLM tasks.
Key concepts
- Context Engineering as Recommendation Problem
- The paper reframes context engineering not as a search for one perfect prompt for the whole dataset, but as recommending the best context strategy for each individual input instance. This allows the system to dynamically select a tailored context instead of using a single, averaged strategy.
- Context-CF Co-Evolution
- This mechanism creates a self-improving catalog of contexts. It uses a Neural Collaborative Filtering model to find instances where current contexts fail and then adjusts context embeddings based on those failures, generating new, specialized contexts that improve future performance.
- Instance-wise Context Routing
- At inference time, the trained system uses a Neural Collaborative Filtering model to look at an unseen input instance and instantly route it to the most suitable context strategy from the catalog. This dynamic assignment ensures each prompt gets its best possible context.
- Cluster-based Initialization
- The training data is first grouped into semantic clusters using embeddings. For each cluster, a specific set of anchor contexts is generated. This provides a strong starting point by creating specialized initial contexts relevant to different types of inputs.
Terminology used across episodes
This episode discusses
- Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering · Paper Radio
- GPT-4 Technical Report
- PromptWizard: Task-Aware Prompt Optimization Framework
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
- EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers
- LLMs Are In-Context Bandit Reinforcement Learners
- GPT-4o System Card
- Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
- POLCA: Stochastic Generative Optimization with LLM
- BPR: Bayesian Personalized Ranking from Implicit Feedback
- Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
- Reflexion: Language Agents with Verbal Reinforcement Learning
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models as Optimizers
- TextGrad: Automatic "Differentiation" via Text
- Inductive Matrix Completion Based on Graph Neural Networks
- A Survey of Large Language Models
- Large Language Models Are Human-Level Prompt Engineers
The paper
Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering · Read on arXiv
Jiachen Zhu, Zhuoying Ou, Congmin Zheng, Yuxiang Chen, Zeyu Zheng, Rong Shan, Lingyu Yang, Lionel Z. WANG
Shanghai Jiao Tong University · “Univ. College London” Research Institute of Technology and Science and Carnegie Mellon University
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Contexting as Recommendation".
Jane: Here's the gist of the paper. Large Language Models (LLMs) are highly sensitive to their input contexts, which motivates automated context engineering.
Tom: Existing methods primarily treat this as a global search problem, seeking a single context strategy that maximizes average performance across a dataset.
Jane: This paper proposes a paradigm shift by formulating context engineering as an instance-wise recommendation problem, introducing Neural Collaborative Context Engineering (NCCE).
Tom: NCCE transitions optimization from static global search to dynamic, instance-wise routing.
Title and authors: Tom: Let’s talk about that title again, "Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering." It tells you right away they’re using recommendation systems and an iterative process to improve the contexts. It suggests a continuous loop rather than a one-time optimization effort.
Jane: And the authors are from Shanghai Jiao Tong University and Carnegie Mellon University, so we have some strong AI representation learning expertise here, which is where I think this idea fits best. It’s connecting LLM instruction tuning with established recommender system techniques.
Lu: Right, and the authors are really pushing for this instance-wise optimization instead of just optimizing that single global context strategy they call p star in the traditional setting. They’re saying that one strategy doesn't fit all instances, which is what they point out on page two > Context Engineering asRecommendation Global Optimization Global Optimizer Context Components Instruction Few-shot Examples Reasoning Formats Output Constraint Instance-wiseOptimization SingleContext Components Different Context Components <ref:2605.15721#pg3>... InstancesasUsers Contextsas Items... CollaborativeFiltering Matrix... Instances Contexts Success Failure Chosen Context Figure one: Context engineering as recommendation: learning to assign instance-specific composite contexts instead of optimizing a single global context strategy <ref:2605.15721#pg2,Instances Contexts Success Failure Chosen Context Figure 1: Context engineering as recommendation>.
Meng: So they’re treating the context strategies like items in a catalog, and instances like users. That mapping is what makes it sound so much more practical for deployment at scale, rather than just theoretical optimization on a test set.
Lalam: It really is about making that process dynamic, not static. They propose this iterative feedback loop where the context catalog and the neural recommender improve each other synergistically > guide, we iteratively generate new context variants to address failure instances, creating a feedback loop where the context catalog and the neural recommender improve each other synergistically.
Tom: So they’re essentially saying that when an LLM fails on an instance, we don't just note it down; we use that failure to generate a better context idea next time > guide, we iteratively generate new context variants to address failure instances, creating a feedback loop where the context catalog and the neural recommender improve each other synergistically.
The paper's summary: Jane: So what’s the actual process they laid out? They start by taking all their training instances and partitioning them into semantic clusters using KMeans on their frozen text encoder embeddings, which gives them a warm-up optimizer to create initial anchor contexts for each cluster.
Tom: That initialization step seems really smart because it gives you specialized starting points for different groups of data, instead of just one generic context to start with > Cluster-based Initialization: The training instances are partitioned into K semantic clusters using KMeans on their frozen text encoder embeddings phi(xi). For each cluster Ck, a warm-up optimizer generates specialized anchor contexts P0,k = A(Ck), creating an informative initial catalog P0.
Lu: And then they move into the core of the paper with Context-CF Co-Evolution. This is where they use a Neural Collaborative Filtering model to estimate how suitable an instance is for each context, projecting their embeddings into a shared latent space to compute an interaction vector > A lightweight Neural Collaborative Filtering NCF model estimates instance-context suitability using instance embeddings ei and context embeddings hj, projecting them into a shared latent space to compute an interaction vector zij.
Meng: So they have this model that learns the relationship between an input and a potential context, and it uses that to find the best fit for a failing example. That sounds like something we could actually integrate into our production pipeline for fine-tuning inputs on the fly > From a sampled batch of these failures, gradient ascent is performed on context embeddings h to maximize their NCF-predicted suitability for that batch.
Lalam: And then they map those optimized context embeddings back to the nearest discrete context in their catalog, and finally an LLM reflector revises that new strategy based on the failures, generating a new strategy > These optimized embeddings h tilde are mapped back to the nearest discrete context in the catalog ppot. Finally, an LLM reflector revises ppot based on its failures, generating a new strategy pnew, which updates both the context catalog Pt+one and the interaction set omega t+one for subsequent NCF training rounds <ref:2605.15721#pg2>.
Tom: So it’s this whole cycle: cluster initialization, using an NCF model to score suitability, finding failures that need fixing, and then using gradient ascent on those embeddings to improve the catalog > Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering Jiachen Zhu* Shanghai Jiao Tong Univ. gebro13@sjtu.edu.cn Zhuoying Ou* Shanghai Jiao Tong Univ. zoeouzy23@sjtu.edu.cn Congmin Zheng Shanghai Jiao Tong Univ. desp.zcm@sjtu.edu.cn Yuxiang Chen Univ College London yuxiangchen25@ucl, Zeyu Zheng Carnegie Mellon Univ zeyuzhen@andrewcmu, Rong Shan Shanghai Jiao Tong Univ shanrong@sjtu.edu.cn Lingyu Yang Shanghai Jiao Tong Univ jlnhbyu.yang@sjtu.edu.cn Lionel Z WANG Hong Kong Polytechnic Univ lionel-z wang @connectpolyuhk, Weiwen Liu Shanghai Jiao Tong Univ wwliu@sjtu, Yong Yu Shanghai Jiao Tong Univ yyu@sjtu, Weinan Zhang Shanghai Jiao Tong Univ wnzhang@sjtu, Jianghao Lin† Shanghai Jiao Tong Univ linjianghao@sjtu.
The paper's improvements: Jane: They highlight a few key improvements over the old methods. First, they’re shifting from that static global optimization to dynamic instance-wise context routing > Context Engineering asRecommendation Global Optimization Global Optimizer Context Components Instruction Few-shot Examples Reasoning Formats Output Constraint Instance-wiseOptimization SingleContext Components Different Context Components... InstancesasUsers Contextsas Items... CollaborativeFiltering Matrix... Instances Contexts Success Failure Chosen Context Figure one: Context engineering as recommendation: learning to assign instance-specific composite contexts instead of optimizing a single global context strategy <ref:2605.15721#pg2,Instances Contexts Success Failure Chosen Context Figure 1: Context engineering as recommendation>.
Lu: The second major part is the iterative generation of new context variants through the feedback loop, which creates a self-improving catalog > guide, we iteratively generate new context variants to address failure instances, creating a feedback loop where the context catalog and the neural recommender improve each other synergistically. This is where they really show that personalization helps significantly.
Meng: The third improvement is how they start things off: robust initialization via semantic clustering using KMeans on text encoder embeddings to build those cluster-specific anchor contexts > Cluster-based Initialization: The training instances are partitioned into K semantic clusters using KMeans on their frozen text encoder embeddings phi(xi). For each cluster Ck, a warm-up optimizer generates specialized anchor contexts P0,k = A(Ck), creating an informative initial catalog P0.
Tom: And they back that up with some theoretical work too. They provide a PAC style bound for the instance-wise routing regret, which basically gives them a theoretical guarantee on how much we can expect to gain from this dynamic approach > Theoretical analysis, based on Assumption one (Cluster Lipschitz Preference), provides a PAC-style bound for instance-wise routing, confirming the theoretical justification of the cluster initialization and co-evolution process <ref:2605.15721#pg2>.
Lalam: And they show that this is very sample efficient too, achieving strong gains even when there’s sparse interaction data, which is important for real world scenarios > Experiments also show that performance is highly sample-efficient, achieving strong gains even with sparse interaction data.
Conclusion: Jane: So to wrap up the main point of "Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering," they’ve shown that moving to instance-wise routing beats relying on a single global context strategy by using an evolutionary collaborative filtering approach > Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering Jiachen Zhu* Shanghai Jiao Tong Univ. gebro13@sjtu.edu.cn Zhuoying Ou* Shanghai Jiao Tong Univ. zoeouzy23@sjtu.edu.cn Congmin Zheng Shanghai Jiao Tong Univ. despzcm@sjtu, Yuxiang Chen Univ College London yuxiangchen25@ucl, Zeyu Zheng Carnegie Mellon Univ zeyuzhen@andrewcmu, Rong Shan Shanghai Jiao Tong Univ shanrong@sjtu.
Tom: It boils down to this: we are building a system that dynamically routes each input instance to its most suitable context strategy instead of relying on a single context strategy for everything > Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering Jiachen Zhu* Shanghai Jiao Tong Univ. gebro13@sjtu.edu.cn Zhuoying Ou* Shanghai Jiao Tong Univ. zoeouzy23@shangtonguniv, Congmin Zheng Shanghai Jiao Tong Univ. despzcm@sjtu, Yuxiang Chen Univ College London yuxiangchen25@ucl, Zeyu Zheng Carnegie Mellon Univ zeyuzhen@andrewcmu.
Lu: I think what this means for the world is that context engineering becomes a personalized process rather than a one-size-fits-all effort, allowing LLMs to be more reliable across a much wider variety of inputs > guide, we iteratively generate new context variants to address failure instances, creating a feedback loop where the context catalog and the neural recommender improve each other synergistically.
Meng: From an engineering standpoint, it means we can focus our tuning efforts on improving the feedback mechanism rather than trying to get that perfect global context which is often just too hard to find > Cluster-based Initialization: The training instances are partitioned into K semantic clusters using KMeans on their frozen text encoder embeddings phi(xi). For each cluster Ck, a warm-up optimizer generates specialized anchor contexts P0,k = A(Ck), creating an informative initial catalog P0.
Lalam: And the paper also flags that the method can handle sparse interaction data well, which means we don't need massive amounts of labeled data to get significant performance gains in real applications > Experiments also show that performance is highly sample-efficient, achieving strong gains even with sparse interaction data.
More episodes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI
- 2605.23500-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation