Retrieve, Match, Escalate: Accurate and Scalable Product Linking with VLM-Distilled Cross-Encoders and Agentic VLMs
cs.AI, cs.CL, cs.DB, cs.IR
Submitted: 2026-08-25
Updated: 2026-08-25
Comments: 9 pages, 3 figures, 5 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Product linking, the entity-resolution task of mapping merchant product records to canonical catalog products, consolidates fragmented listings so downstream search, recommendation, and advertising
Terminology
Abstract
Product linking, the entity-resolution task of mapping merchant product records to canonical catalog products, consolidates fragmented listings so downstream search, recommendation, and advertising see one clean entry per product. At marketplace scale, billions of noisy, multi-category records must be resolved against tens of millions of canonical products, where scoring every candidate with a single model is either too weak for the hard cases or too costly for the easy ones. We present a production retrieve-then-match cascade that spends computation in proportion to difficulty: retrieval surfaces plausible matches, a lightweight text cross-encoder auto-resolves the high-confidence majority, and an agentic multimodal vision-language model settles the ambiguous remainder by inspecting product images and issuing web searches for evidence that is in neither record. The cross-encoder is distilled from millions of dual-VLM-consensus labels, retiring human annotation from the training set, and is calibrated to auto-accept links at a 98% precision bar validated against a smaller operator-certified audit. The agent is a self-hosted open-weight model that reaches a closed frontier VLM's precision at a four-point recall cost (88% versus 92%) for roughly one-seventh the per-pair cost, with no fine-tuning. Per-pair cost spans nearly five orders of magnitude from the cheap cross-encoder to the frontier VLM, so escalating only the hard tail to the agent raises end-to-end link coverage from the cheap stage's 68% to 77%.
Sources
- Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback
- Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR
- VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Annotation-Free Pathological Image Classification
- Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents
- Build, Judge, Optimize: A Blueprint for Continuous Improvement of Multi-Agent Consumer Assistants
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- Distilling the Knowledge in a Neural Network
- Scaling Down to Scale Up: A Cost-Benefit Analysis of Replacing OpenAI's LLM with Open Source SLMs in Production
- Optimizing Product Deduplication in E-Commerce with Multimodal Embeddings
- AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
- Leveraging the Power of Large Language Models in Entity Linking via Adaptive Routing and Targeted Reasoning
- Adaptive Vision-Language Model Routing for Computer Use Agents
- AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
- Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs
- End-to-end multi-modal product matching in fashion e-commerce
- Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- Closing the Auto-Research Loop: An AI Co-Scientist for Production Search Ranking
- The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems