Evaluating the Retrieval Robustness of Large Language Models

Name: Evaluating the Retrieval Robustness of Large Language Models
Uploaded: 2026-10-06T14:34:49.061390+00:00
Duration: 368 s
Description: This study tested how robust large language models are when using Retrieval-Augmented Generation (RAG). It measured three robustness metrics: No-Degradation Rate, Retrieval Size Robustness, and Retrieval Order Robustness. Findings show models are generally strong but suffer from 'imperfect robustness,' meaning performance can fluctuate depending on retrieval size or document order.

This study tested how robust large language models are when using Retrieval-Augmented Generation (RAG). It measured three robustness metrics: No-Degradation Rate, Retrieval Size Robustness, and Retrieval Order Robustness. Findings show models are generally strong but suffer from 'imperfect robustness,' meaning performance can fluctuate depending on retrieval size or document order.

Open the full episode