RAG vs. GraphRAG: A Systematic Evaluation and Key Insights
cs.IR
Submitted: 2025-02-17
Updated: 2026-11-07
Code: https://github.com/haoyuhan1/RAGvsGraphRAG2https:
Project page: https://microsoft.github.io/graphrag
License: http://creativecommons.org/licenses/by/4.0/
The gist: Retrieval-Augmented Generation (RAG) improves large language models (LLMs) by retrieving relevant information from external sources and has been widely adopted for text-based tasks.
Terminology
Abstract
Retrieval-Augmented Generation (RAG) improves large language models (LLMs) by retrieving relevant information from external sources and has been widely adopted for text-based tasks. For structured data, such as knowledge graphs, Graph Retrieval-Augmented Generation (GraphRAG) retrieves and aggregates information along graph structures. More recently, GraphRAG has been extended to general text settings by organizing unstructured text into graph representations, showing promise for reasoning and grounding. Despite these advances, existing GraphRAG systems for text data are often tailored to specific tasks, datasets, and system designs, resulting in heterogeneous evaluation protocols. Consequently, a systematic understanding of the relative strengths, limitations, and trade-offs between RAG and GraphRAG on widely used text benchmarks remains limited. In this paper, we present a comprehensive benchmark study comparing RAG and GraphRAG on established text-based tasks, including question answering and query-based summarization. We introduce a unified evaluation protocol that standardizes data preprocessing, retrieval configurations, and generation settings, enabling fair and reproducible comparisons. Our results highlight the distinct strengths of RAG and GraphRAG across different tasks and evaluation perspectives. Building on these findings, we explore selection and integration strategies that combine the strengths of both paradigms, leading to consistent performance improvements. We further analyze failure modes, efficiency trade-offs, and evaluation biases, and highlight key considerations for designing and evaluating retrieval-augmented generation systems.
Sources
- Don't Forget to Connect! Improving RAG with Graph-based Reranking
- A Survey on In-context Learning
- The Llama 3 Herd of Models
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Ragas: Automated Evaluation of Retrieval Augmented Generation
- The Chronicles of RAG: The Retriever, the Chunk and the Generator
- Retrieval-Augmented Generation for Large Language Models: A Survey
- From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
- Retrieval-Augmented Generation with Graphs (GraphRAG)
- Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- Active Retrieval Augmented Generation
- Dense Passage Retrieval for Open-Domain Question Answering
- FactKG: Fact Verification via Reasoning on Knowledge Graphs
- Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
- Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
- Query Rewriting for Retrieval-Augmented Large Language Models
- Nomic Embed: Training a Reproducible Long Context Text Embedder
- Graph Retrieval-Augmented Generation: A Survey
- ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Related papers
- UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA