Revisiting Text Ranking in Deep Research

arXiv:2602.21456 · cs.IR, cs.AI, cs.CL · Submitted 2026-02-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Revisiting Text Ranking in Deep Research".

Title and authors: Tom: So, let's talk about who wrote this paper and what it’s called. It’s titled "Revisiting Text Ranking in Deep Research."

Jane: That title tells you right away that they aren't just looking at existing ranking methods; they are re-examining them specifically for the context of deep research agents.

Tom: Right. And the authors include Chuan Meng, Litu Ou, and Sean MacAvaney from Edinburgh and Glasgow universities.

Lu: They bring a strong background in both core text ranking theory and working with large language models, which should give them a solid foundation for this kind of evaluation.

Meng: I wonder if they focused too much on the academic setup and not enough on how these ranking methods actually behave when we put them into a production environment with real web search traffic.

Lalam: Based on what I see, they are setting up a framework to systematically check the effectiveness of these techniques across different scenarios.

The paper's summary: Jane: So, the summary explains that deep research agents use iterative reasoning and search to find answers, and the authors are providing a look at how established text ranking methods perform in this setup.

Tom: They highlight three specific areas they investigate: retrieval units, pipeline configurations involving re-rankers, and query characteristics.

Lu: The summary points out a key gap they are filling: we don't really know if the re-ranking methods we usually use actually give consistent benefits when the consumer is an LLM agent instead of a human.

Meng: That inconsistency sounds bad for deployment; you want predictable performance, not something that works well on paper but breaks in practice.

Lalam: They are also looking at how the queries issued by these agents—the queries they generate—might not match what was used to train the ranking models initially.

The paper's improvements: Tom: Now, let's get into what they actually suggest is better than what we currently know. They investigate retrieval units, pipeline configurations, and query characteristics as their main points of improvement.

Jane: For retrieval units, they found that passage-level information units are quite efficient for agents with limited context windows because their concise nature makes them more efficient under limited context-window budgets.

Lu: And on the other side, they saw that BM25, a lexical retriever, often outperforms neural retrievers when dealing with out-of-domain data compared to single-vector dense models.

Meng: That’s interesting because if BM25 is better and simpler to run at scale, that might suggest we should lean on it more for initial retrieval stages.

Lalam: They also focused heavily on re-ranking, showing it consistently lifts documents to the top of ranked lists and reduces the number of search calls needed by the agent.

Conclusion: Tom: So, to wrap up, they found that lexical retrievers like BM25 with specific settings can outperform neural rankers in most cases, and passage-level units help agents manage smaller context windows better.

Jane: They also proposed a query-to-question method to bridge the gap between agent queries and the training queries used for neural rankers, which seems important for improving their effectiveness.

Lu: The main thing they point out is that when an agent issues a search query, it can cause distribution drift for neural rankers, which limits their performance in deep research settings.

Meng: From my side, I see the Q2Q method as something that could stabilize performance during deployment because it handles that mismatch between the agent's real-time need and the model's training data better.

Lalam: They conclude that while established methods are still effective, addressing query mismatch and using passage-level units offers concrete ways to improve how these agents find information.

Chuan Meng, Litu Ou, Sean MacAvaney, Jeff Dalton

University of Edinburgh · University of Glasgow

cs.IR, cs.AI, cs.CL

Submitted: 2026-02-25

Updated: 2026-06-19

Comments: Accepted at the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026)

Journal ref: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026), pp. 3006-3016

DOI: 10.1145/3805712.3808557

Code: https://github.com/ChuanMeng/text-ranking-in-deep-research

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

The gist: Deep research has emerged as an important task that aims to address hard queries that need extensive open-web exploration The paper examines key findings and best practices for text ranking methods…

Key concepts

Passage-level Information Units
These units focus on ranking individual passages rather than entire documents. They are efficient for agents with limited context windows because their concise nature allows for more search and reasoning iterations before hitting context limits, leading to higher accuracy when memory is constrained.
Query-to-Question (Q2Q) Method
This technique translates raw agent-issued web search queries into natural-language questions. This is crucial because the mismatch between the agent's query and the training queries used for neural rankers severely degrades performance; Q2Q helps bridge this gap by using a more effective question format.
BM25 Retrieval
BM25 is a lexical retrieval method that performs well, especially when agent queries follow a web-search style with keywords and phrases. Optimizing BM25 parameters, such as setting k1=10 and b=1 for document indexing, substantially improves performance by reducing sensitivity to length normalization.
Re-ranking Efficacy
A re-ranking stage consistently enhances ranking effectiveness and accuracy while simultaneously reducing the number of search calls. Deeper re-ranking depths further amplify these gains, confirming that this step is vital for deep research agents.

Terminology

Summary

Deep research has emerged as an important task that aims to address hard queries that need extensive open-web exploration The paper examines key findings and best practices for text ranking methods in deep research from three perspectives: retrieval units, pipeline configurations, and query characteristics

Research Gaps Addressed

The authors identify three research gaps in text ranking for deep research that have received limited attention First, passage-level information units have received little attention in deep research because most existing studies build agents that search and read at the document level Second, the understanding of re-ranking in deep research remains limited regarding whether widely-used re-ranking methods provide consistent benefits when the content consumer is an LLM-based agent Third, the potential mismatch between agent-issued queries and training queries used for text ranking methods remains underexplored

Retrieval Unit Effectiveness

RQ1 investigates how existing retrievers perform under passage-level and document-level retrieval units The findings show fivefold results regarding passage-level units Specifically, the concise nature of passage-level units enables more search and reasoning iterations before reaching context-window limits, resulting in higher answer accuracy than document-level units without a full-document reader Furthermore, BM25 on the passage corpus outperforms neural retrievers in most cases, achieving the highest accuracy of 0.572 across all retrieval settings in their study Learned sparse and multi-vector dense retrievers with only millions of parameters generalise better to web-search-style queries than 7B/8B single-vector dense models

Re-ranking Efficacy and Mismatch Mitigation

RQ2 addresses the effectiveness of a re-ranking stage in deep research under different initial retrievers, re-ranker types, and re-ranking cutoffs Re-ranking consistently improves ranking effectiveness and answer accuracy while reducing search calls, confirming its important role in deep research The BM25–monoT5-3B pipeline with gpt-oss-20b achieves the best results in their work, reaching 0.716 recall and 0. p. 1

Where the paper says it

Notably, the BM25–monoT5-3B pipeline with gpt-oss-20b achieves the best results in our work, reaching 0. 716 recall and 0.

689 accuracy For RQ3, the authors propose a query-to-question (Q2Q) method to translate agent-issued web search queries into natural-language questions, significantly improving neural retrieval and re-ranking performance This indicates that the mismatch between agent-issued queries and queries used to train neural rankers can severely degrade neural ranking effectiveness

Key Findings on Query Mismatch

The Q2Q (Q+R) variant consistently achieves significant improvements over raw agent-issued queries for both SPLADE-v3 and Qwen3-Embed-8B retrievers These results indicate that mismatch between agent-issued queries and the natural-language questions used to train neural rankers severely limits their effectiveness in deep research Q2Q (Q) provides little to no improvement over raw agent-issued queries, as illustrated by examples where the raw query is keyword-driven and under-specified Conversely, for BM25, Q2Q-generated questions even hurt performance, indicating that web-search-style queries are better suited to BM25 than natural-language questions

Conclusion and Contributions

The main contributions include reproducing a comprehensive set of text ranking methods in deep research They demonstrate that the lexical retriever BM25 with appropriate setup outperforms neural rankers in most cases, achieving the highest answer accuracy across all retrieval settings in their study They also find that passage-level units benefit agents with limited context windows and reduce sensitivity to document-length normalisation in lexical retrieval The authors propose the Q2Q method as a way to mitigate the training–inference query mismatch, which is critical for improving neural rankers in deep research They open-sourced their code, data, and all agent-generated traces at https://github.com/ChuanMeng/text-ranking-in-deep-research

Future Directions

Future directions identified include evaluating additional model families and larger model sizes to assess the generalisability of their findings Exploring additional rankers and configurations, such as scaling laws, is an important direction Extending deep research to conversational scenarios is also a promising avenue The paper was supported by a Turing AI Acceleration Fellowship funded by the Engineering and Physical Sciences Research Council (EPSRC), grant number EP/V025708/1

References

[1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b Model Card. p. 9

Where the paper says it

References [1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b Model Card.

p. 9
Where the paper says it

References [1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b Model Card.

arXiv preprint arXiv:2508. p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

10925 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

[4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary and Tong Wang. p. 9

Where the paper says it

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang.

2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. p. 9
Where the paper says it

[4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.

p. 9
Where the paper says it

[4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.

arXiv preprint arXiv:1611. p. 9
Where the paper says it

MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv preprint arXiv:1611.

p. 9
Where the paper says it

MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv preprint arXiv:1611.

09268 (2018). p. 9
Where the paper says it

arXiv preprint arXiv:1611. 09268 (2018).

p. 9
Where the paper says it

arXiv preprint arXiv:1611. 09268 (2018).

[5] Antoine Chaffin and Raphaël Sourty. p. 9

Where the paper says it

09268 (2018). [5] Antoine Chaffin and Raphaël Sourty.

p. 9
Where the paper says it

09268 (2018). [5] Antoine Chaffin and Raphaël Sourty.

2025. PyLate: Flexible Training and Retrieval for Late Interaction Models. p. 9
Where the paper says it

[5] Antoine Chaffin and Raphaël Sourty. PyLate: Flexible Training and Retrieval for Late Interaction Models.

p. 9
Where the paper says it

[5] Antoine Chaffin and Raphaël Sourty. PyLate: Flexible Training and Retrieval for Late Interaction Models.

In CIKM. 6334–6339. p. 9
Where the paper says it

PyLate: Flexible Training and Retrieval for Late Interaction Models. 6334–6339.

[6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al. p. 9

Where the paper says it

6334–6339. [6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al.

2025. BrowseCompPlus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent. p. 9
Where the paper says it

[6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al. BrowseCompPlus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent.

arXiv preprint arXiv:2508.06600 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 06600 (2025).

[7] Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. p. 9

Where the paper says it

06600 (2025). [7] Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M.

Voorhees. 2019. Overview of the TREC 2019 Deep Learning Track. p. 9
Where the paper says it

[7] Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Overview of the TREC 2019 Deep Learning Track.

In REC 2019. p. 9
Where the paper says it

Overview of the TREC 2019 Deep Learning Track. In REC 2019.

[8] Zhuyun Dai and Jamie Callan. p. 9

Where the paper says it

In REC 2019. [8] Zhuyun Dai and Jamie Callan.

2019. Deeper Text Understanding for IR with Contextual Neural Language Modeling. p. 9
Where the paper says it

[8] Zhuyun Dai and Jamie Callan. Deeper Text Understanding for IR with Contextual Neural Language Modeling.

In SIGIR. 985–988.

[10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. p. 9

Where the paper says it

TREC CAsT 2021: The Conversational Assistance Track Overview. [10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.

2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. p. 9
Where the paper says it

[10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.

In NAACL. 4171–4186. p. 9
Where the paper says it

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 4171–4186.

[13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi and et al. p. 9

Where the paper says it

2288–2292. [13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.

2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. p. 9
Where the paper says it

[13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.

arXiv preprint arXiv:2501. p. 9
Where the paper says it

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2501.

12948 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2501. 12948 (2025).

[14] Ben He and Iadh Ounis. p. 9

Where the paper says it

12948 (2025). [14] Ben He and Iadh Ounis.

2005. Term Frequency Normalisation Tuning for BM25 and DFR Models. p. 9
Where the paper says it

[14] Ben He and Iadh Ounis. Term Frequency Normalisation Tuning for BM25 and DFR Models.

In ECIR. 200–214.

[17] Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, and Chen Zhao. p. 9

Where the paper says it

LoRA: Low-Rank Adaptation of Large Language Models. [17] Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, and Chen Zhao.

2026. SAGE: Benchmarking and Improving Retrieval for Deep Research Agents. p. 9
Where the paper says it

[17] Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, and Chen Zhao. SAGE: Benchmarking and Improving Retrieval for Deep Research Agents.

arXiv preprint arXiv:2602. p. 9
Where the paper says it

jina-embeddings-v5-text: Task-Targeted Embedding Distillation. arXiv preprint arXiv:2602.

05975 (2026). p. 9
Where the paper says it

arXiv preprint arXiv:2602. 05975 (2026).

[18] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani and Jiawei Han. p. 9

Where the paper says it

05975 (2026). [18] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han.

2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. p. 9
Where the paper says it

[18] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning.

In COLM.

[19] Marcin Kaszkiel and Justin Zobel. p. 9

Where the paper says it

Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [19] Marcin Kaszkiel and Justin Zobel.

1997. Passage Retrieval Revisited. p. 9
Where the paper says it

[19] Marcin Kaszkiel and Justin Zobel. Passage Retrieval Revisited.

In ACM SIGIR Forum, Vol. p. 9
Where the paper says it

Passage Retrieval Revisited. In ACM SIGIR Forum, Vol.

31. ACM New York, NY, USA, 178–185. p. 9
Where the paper says it

In ACM SIGIR Forum, Vol. ACM New York, NY, USA, 178–185.

[20] Omar Khattab and Matei Zaharia. p. 9

Where the paper says it

ACM New York, NY, USA, 178–185. [20] Omar Khattab and Matei Zaharia.

2020. Colbert: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. p. 9
Where the paper says it

[20] Omar Khattab and Matei Zaharia. Colbert: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT.

In SIGIR. 39–48.

[21] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin et al. p. 9

Where the paper says it

Colbert: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. [21] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al.

2019. Natural Questions: A Benchmark for Question Answering Research. p. 9
Where the paper says it

[21] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural Questions: A Benchmark for Question Answering Research.

TACL 7 (2019), 453–466. p. 9
Where the paper says it

Natural Questions: A Benchmark for Question Answering Research. TACL 7 (2019), 453–466.

[22] Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. p. 9

Where the paper says it

TACL 7 (2019), 453–466. [22] Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant.

2024. SPLADE-v3: New baselines for SPLADE. p. 9
Where the paper says it

[22] Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. SPLADE-v3: New baselines for SPLADE.

arXiv preprint arXiv:2403. p. 9
Where the paper says it

SPLADE-v3: New baselines for SPLADE. arXiv preprint arXiv:2403.

06789 (2024). p. 9
Where the paper says it

arXiv preprint arXiv:2403. 06789 (2024).

[31] Chuan Meng, Jiqun Liu, Mohammad Aliannejadi, Fengran Mo, Jeff Dalton and Maarten de Rijke. p. 9

Where the paper says it

Ranked List Truncation for Large Language Model-based Re-Ranking. [31] Chuan Meng, Jiqun Liu, Mohammad Aliannejadi, Fengran Mo, Jeff Dalton, and Maarten de Rijke.

2026. Re-Rankers as Relevance Judges. p. 9
Where the paper says it

[31] Chuan Meng, Jiqun Liu, Mohammad Aliannejadi, Fengran Mo, Jeff Dalton, and Maarten de Rijke. Re-Rankers as Relevance Judges.

arXiv preprint arXiv:2601. p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

p. 9
Where the paper says it

Re-Rankers as Relevance Judges. arXiv preprint arXiv:2601.

04455 (2026). p. 9
Where the paper says it

arXiv preprint arXiv:2601. 04455 (2026).

[38] Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. p. 9

Where the paper says it

04085 (2019). [38] Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin.

2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. p. 9
Where the paper says it

[38] Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. Document Ranking with a Pretrained Sequence-to-Sequence Model.

In EMNLP. 708–718.

[45] Sahel Sharifymoghaddam and Jimmy Lin. p. 9

Where the paper says it

20595 (2025). [45] Sahel Sharifymoghaddam and Jimmy Lin.

2026. Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents. p. 9
Where the paper says it

[45] Sahel Sharifymoghaddam and Jimmy Lin. Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents.

arXiv preprint arXiv:2601.14224 (2026). p. 9
Where the paper says it

arXiv preprint arXiv:2601. 14224 (2026).

[58] Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie and Benjamin Van Durme. p. 9

Where the paper says it

ArXiv (2023). [26] Jimmy Lin, Rodrigo Nogueira, and Andrew Yates.

2025. Rank1: Test-Time Compute for Reranking in Information Retrieval. p. 9
Where the paper says it

[39] Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Pengjun Xie, Jingren Zhou, and Yong Jiang. BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents. p. 9

Where the paper says it

[39] Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Pengjun Xie, Jingren Zhou, and Yong Jiang. BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents.

In COLM.

[69] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang and Dayiheng Liu et al. p. 9

Where the paper says it

05366 (2025). [25] Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang.

2025. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. p. 2
Where the paper says it

The series features fine-tuning of Qwen3 LLMs [62] on synthetic training data across multiple domains and languages, generated by the Qwen3 models themselves. It follows RepLLaMA to obtain query and document embeddings and shares the same embedding dimension.

arXiv preprint arXiv:2506. p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

05176 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

[71] Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon and Daxin Jiang. p. 9

Where the paper says it

6334–6339. [6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al.

p. 9
Where the paper says it

6334–6339. [6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al.

2022. Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation. p. 9
Where the paper says it

[33] Chuan Meng, Francesco Tonolini, Fengran Mo, Nikolaos Aletras, Emine Yilmaz, and Gabriella Kazai. Bridging the Gap: From Ad-hoc to Proactive Search in Conversations.

p. 9
Where the paper says it

[33] Chuan Meng, Francesco Tonolini, Fengran Mo, Nikolaos Aletras, Emine Yilmaz, and Gabriella Kazai. Bridging the Gap: From Ad-hoc to Proactive Search in Conversations.

arXiv preprint arXiv:2206. p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

p. 9
Where the paper says it

gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.

10128 (2022). p. 9
Where the paper says it

Trippas, and Svitlana Vakulenko. TREC CAsT 2022: Going Beyond User Ask and System Retrieve with Initiative and Response Generation.

p. 9
Where the paper says it

Trippas, and Svitlana Vakulenko. TREC CAsT 2022: Going Beyond User Ask and System Retrieve with Initiative and Response Generation.

[75] Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus and Amelia Glaese. 2025. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. p. 7

Where the paper says it

, the BrowseComp benchmark [55] includes queries that humans typically cannot solve within ten minutes using search engines. Solving these queries typically requires LLM-based agents to perform multiple rounds of search and reasoning to gather, synthesise,

p. 7
Where the paper says it

, the BrowseComp benchmark [55] includes queries that humans typically cannot solve within ten minutes using search engines. Solving these queries typically requires LLM-based agents to perform multiple rounds of search and reasoning to gather, synthesise,

arXiv preprint arXiv:2504. p. 9
Where the paper says it

ReasonIR: Training Retrievers for Reasoning Tasks. arXiv preprint arXiv:2504.

p. 9
Where the paper says it

ReasonIR: Training Retrievers for Reasoning Tasks. arXiv preprint arXiv:2504.

p. 9
Where the paper says it

ReasonIR: Training Retrievers for Reasoning Tasks. arXiv preprint arXiv:2504.

12516 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

[76] Yibo Wang, Lei Wang, Yue Deng, Keming Wu, Yao Xiao, Huanjin Yao, Liwei Kang, Hai Ye and Yongcheng Jing et al. p. 9

Where the paper says it

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang.

2026. DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation. p. 0
Where the paper says it

CCS Concepts • Information systems → Information retrieval. Keywords Text ranking, Retrieval, Re-ranking, Deep research, Agentic search ACM Reference Format: Chuan Meng, Litu Ou, Sean MacAvaney, and Jeff Dalton.

arXiv preprint arXiv:2601.09688 (2026). p. 0
Where the paper says it

Revisiting Text Ranking in Deep Research. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia.

[77] Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin Hao Zeng et al. p. 9

Where the paper says it

Bridging the Gap: From Ad-hoc to Proactive Search in Conversations. [34] Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, et al.

p. 9
Where the paper says it

Bridging the Gap: From Ad-hoc to Proactive Search in Conversations. [34] Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, et al.

2025. GLM-4.5: Agentic, Reasoning and Coding (ARC) Foundation Models. p. 9
Where the paper says it

[24] Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic Search-Enhanced Large Reasoning Models.

arXiv preprint arXiv:2508.06471 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

[78] Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong et al. p. 9

Where the paper says it

02592 (2025). [24] Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou.

2025. BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese. p. 0
Where the paper says it

Recently, the IR community has witnessed the emergence of deep research [46] scenarios. Deep research aims to answer multi-hop, reasoning-intensive queries that require extensive exploration of the open web and are difficult for both humans and large language models (LLMs) [55].

arXiv preprint arXiv:2504.19314 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2508. 10925 (2025).

[79] Chuan Meng, Mohammad Aliannejadi and Maarten de Rijke. p. 9

Where the paper says it

2421–2425. [29] Chuan Meng, Mohammad Aliannejadi, and Maarten de Rijke.

p. 9
Where the paper says it

2421–2425. [29] Chuan Meng, Mohammad Aliannejadi, and Maarten de Rijke.

2023. System Initiative Prediction for Multi-turn Conversational Information Seeking. p. 9
Where the paper says it

[29] Chuan Meng, Mohammad Aliannejadi, and Maarten de Rijke. System Initiative Prediction for Multi-turn Conversational Information Seeking.

p. 9
Where the paper says it

[29] Chuan Meng, Mohammad Aliannejadi, and Maarten de Rijke. System Initiative Prediction for Multi-turn Conversational Information Seeking.

In CIKM. 1807–1817. p. 9
Where the paper says it

System Initiative Prediction for Multi-turn Conversational Information Seeking. 1807–1817.

p. 9
Where the paper says it

System Initiative Prediction for Multi-turn Conversational Information Seeking. 1807–1817.

[80] Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I Hsu, Jun Yan, Vishy Tirumalashetty and Eunsol Choi. p. 9

Where the paper says it

05592 (2025). [48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu.

p. 9
Where the paper says it

05592 (2025). [48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu.

2026. SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback. p. 0
Where the paper says it

CCS Concepts • Information systems → Information retrieval. Keywords Text ranking, Retrieval, Re-ranking, Deep research, Agentic search ACM Reference Format: Chuan Meng, Litu Ou, Sean MacAvaney, and Jeff Dalton.

p. 0
Where the paper says it

CCS Concepts • Information systems → Information retrieval. Keywords Text ranking, Retrieval, Re-ranking, Deep research, Agentic search ACM Reference Format: Chuan Meng, Litu Ou, Sean MacAvaney, and Jeff Dalton.

p. 0
Where the paper says it

CCS Concepts • Information systems → Information retrieval. Keywords Text ranking, Retrieval, Re-ranking, Deep research, Agentic search ACM Reference Format: Chuan Meng, Litu Ou, Sean MacAvaney, and Jeff Dalton.

arXiv preprint arXiv:2601.18202 (2026). p. 0
Where the paper says it

Revisiting Text Ranking in Deep Research. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia.

p. 0
Where the paper says it

Revisiting Text Ranking in Deep Research. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia.

p. 0
Where the paper says it

Revisiting Text Ranking in Deep Research. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia.

[9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. p. 9

Where the paper says it

Deeper Text Understanding for IR with Contextual Neural Language Modeling. [9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan.

p. 9
Where the paper says it

Deeper Text Understanding for IR with Contextual Neural Language Modeling. [9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan.

2021. TREC CAsT 2021: The Conversational Assistance Track Overview. p. 9
Where the paper says it

[9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. TREC CAsT 2021: The Conversational Assistance Track Overview.

p. 9
Where the paper says it

[9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. TREC CAsT 2021: The Conversational Assistance Track Overview.

In TREC.

[39] Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye et al. p. 9

Where the paper says it

Document Ranking with a Pretrained Sequence-to-Sequence Model. [39] Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Pengjun Xie, Jingren Zhou, and Yong Jiang.

2025. BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents. arXiv preprint arXiv:2510. p. 9
Where the paper says it

BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents. arXiv preprint arXiv:2510.

23458 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2510. 23458 (2025).

[46] Zhengliang Shi, Yiqun Chen, Haitao Li, Weiwei Sun, Shiyu Ni, Yougang Lyu, Run-Ze Fan et al. p. 9

Where the paper says it

14224 (2026). [46] Zhengliang Shi, Yiqun Chen, Haitao Li, Weiwei Sun, Shiyu Ni, Yougang Lyu, Run-Ze Fan, Bowen Jin, Yixuan Weng, Minjun Zhu, et al.

2025. Deep Research: A Systematic Survey. p. 9
Where the paper says it

[46] Zhengliang Shi, Yiqun Chen, Haitao Li, Weiwei Sun, Shiyu Ni, Yougang Lyu, Run-Ze Fan, Bowen Jin, Yixuan Weng, Minjun Zhu, et al. Deep Research: A Systematic Survey.

arXiv preprint arXiv:2512. p. 9
Where the paper says it

Deep Research: A Systematic Survey. arXiv preprint arXiv:2512.

02038 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2512. 02038 (2025).

[47] Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang and Ji-Rong Wen. p. 9

Where the paper says it

02038 (2025). [47] Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen.

2025. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. p. 9
Where the paper says it

[47] Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning.

arXiv preprint arXiv:2503. p. 9
Where the paper says it

R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2503.

05592 (2025). p. 9
Where the paper says it

arXiv preprint arXiv:2503. 05592 (2025).

[48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu et al. p. 9

Where the paper says it

05592 (2025). [48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu.

2025. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval. p. 9
Where the paper says it

[48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval.

In ICLR.

[51] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. p. 9

Where the paper says it

[6] Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al. BrowseCompPlus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent.

In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).

[55] Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung et al. p. 9

Where the paper says it

05592 (2025). [48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu.

2025. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. arXiv preprint arXiv:2504.12516 (2025).

[60] Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I Hsu, Jun Yan, Vishy Tirumalashetty and Eunsol Choi. p. 9

Where the paper says it

05592 (2025). [48] Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu.

2026. SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback. arXiv preprint arXiv:2601.18202 (2026).

[71] Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon and Daxin Jiang. 2022. Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation. arXiv preprint arXiv:2206.10128 (2022).

[77] Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin Hao Zeng et al. 2025. GLM-4.5: Agentic Reasoning and Coding (ARC) Foundation Models. p. 9

Where the paper says it

[24] Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic Search-Enhanced Large Reasoning Models.

arXiv preprint arXiv:2508.06471 (2025).

[79] Chuan Meng, Mohammad Aliannejadi and Maarten de Rijke. 2023. System Initiative Prediction for Multi-turn Conversational Information Seeking. In CIKM. 1807–1817.

[80] Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I Hsu, Jun Yan, Vishy Tirumalashetty and Eunsol Choi. 2026. SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback. arXiv preprint arXiv:2601.18202 (2026).

[9] Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2021. TREC CAsT 2021: The Conversational Assistance Track Overview. In TREC.

Conclusion

Overall, text ranking methods developed by the IR community are still highly effective in deep research Several well-established findings continue to hold, including the effectiveness of BM25 with appropriate setup and the generalisation of BERT-based learned sparse and multi-vector dense retrievers over LLM-based single-vector dense retrievers However, web-search-style syntax of agent-issued queries induces distribution drift for neural rankers, limiting their effectiveness in deep research The proposed Q2Q method significantly mitigates this drift, improving neural ranking performance Passage-level units benefit agents with limited context windows and reduce sensitivity to document-length normalisation in lexical retrieval The paper was supported by a Turing AI Acceleration Fellowship funded by the Engineering and Physical Sciences Research Council (EPSRC), grant number EP/V025708/1

Acknowledgments

We thank Zijian Chen and Xueguang Ma for their guidance on using BrowseComp-Plus This research was supported by a Turing AI Acceleration Fellowship funded by the Engineering and Physical Sciences Research Council (EPSRC), grant number EP/V025708/1

[1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao et al. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.10925 (2025).

[4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder et al. p. 9

Where the paper says it

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [4] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang.

2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv preprint arXiv:1611.09268 (2018).

[5] Antoine Chaffin and Raphaël Sourty. 2025. PyLate: Flexible Training and Retrieval for Late Interaction Models.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Passage-level units improve efficiency under limited context windows because their concise nature makes them more efficient under limited context-window budgets. This allows agents to perform more search and reasoning iterations before reaching context-window limits, resulting in higher answer accuracy for agents with smaller contexts.

  2. Query-to-question (Q2Q) reformulation significantly reduces the mismatch between agent-issued queries and training queries, which is critical because the mismatch between agent-issued queries and the training queries used for text ranking methods can severely degrade neural ranking effectiveness. This is achieved by translating search queries into natural-language questions or including the reasoning trace to capture the agent’s search intent.

  3. Using document-oriented BM25 parameters, such as k1 = 10 and b = 1, substantially improves performance on documents by addressing sensitivity to length normalisation. This configuration can yield the highest recall (0.647) across all settings in this section for the gpt-oss-20b agent when indexing the full document.

  4. Re-ranking stages consistently improve ranking effectiveness and answer accuracy while reducing search calls, with deeper re-ranking depths further amplifying these gains. This suggests that increasing the number of reranking iterations from 10 to 50 provides further gains in recall and accuracy, alongside a reduction in search calls.

  5. The system can be optimized by employing retrieval units suited to agent output; for example, lexical retrievers such as BM25 are particularly effective when agent-issued queries follow a web-search style with keywords, phrases, and quotation marks for exact matching.

Abstract

Deep research has emerged as an important task that aims to address hard queries that need extensive open-web exploration. To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it. Despite search's essential role in deep research, black-box web search APIs leave the behaviour of established text ranking methods in deep research largely unclear. To fill this gap, we reproduce key findings and best practices for text ranking methods in deep research. We examine their effectiveness from three perspectives: (i) retrieval units (documents vs. passages), (ii) pipeline configurations (different retrievers, re-rankers, and re-ranking depths), and (iii) query characteristics (the mismatch between agent-issued queries and the training queries of text rankers). We perform experiments on BrowseComp-Plus, a deep research dataset with a fixed corpus, evaluating 2 open-source agents, 5 retrievers, and 3 re-rankers. We find that agent-issued queries typically follow web-search-style syntax (e.g., quoted exact matches), favouring lexical, learned sparse, and multi-vector retrievers; passage-level units are more efficient under limited context windows, and avoid the difficulties of document length normalisation in lexical retrieval; re-ranking is highly effective. We further propose a query-to-question (Q2Q) method that translates agent-issued queries into natural language questions, significantly reducing the query mismatch.

Sources

Related papers