From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction
The research addresses slow and expensive relevance prediction using large LLMs by distilling their performance into smaller, efficient multimodal student models. The method uses a high-quality human dataset to train an LLM-as-a-judge system as a teacher. This teacher then generates labeled data used to train compact student models, achieving near-teacher accuracy while drastically reducing inference cost and latency for production search systems.