DS@GT ARC at Touch'e: Large Language Models for Retrieval-Augmented Debate

arXiv:2608.08143 · cs.IR, cs.CL · Submitted 2026-08-08 · Read on arXiv

Anthony Miyaguchi, Conor Johnston

cs.IR, cs.CL

Submitted: 2026-08-08

Updated: 2026-08-11

Comments: 12 pages, 4 figures. Accepted for publication in the CLEF 2026 Best of Labs proceedings

Code: https://github.com/dsgt-arc/touche-2025-rad

License: http://creativecommons.org/licenses/by/4.0/

The gist: We extend the DS@GT ARC working-note submission to the Touch'e 2025 Retrieval-Augmented Debate task.

Terminology

LLM judge — 7× in this paper · explained in 2 episodes
Accuracy — 6× in this paper · explained in 1 episode
Calibration — 6× in this paper · explained in 9 episodes
Large Language Models — 5× in this paper
RAG — 3× in this paper · explained in 3 episodes
Verification — 3× in this paper · explained in 2 episodes
Consistency — 2× in this paper · explained in 3 episodes
F1 Score — 2× in this paper · explained in 4 episodes
Generator — 2× in this paper · explained in 2 episodes
Verifier — 2× in this paper · explained in 2 episodes
Augmentation — 1× in this paper · explained in 2 episodes
Distillation — 1× in this paper · explained in 2 episodes
Embeddings — 1× in this paper · explained in 2 episodes
Evaluation Framework — 1× in this paper · explained in 3 episodes
Language Models (LLMs) — 1× in this paper · explained in 2 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 9 episodes
Papers — 1× in this paper · explained in 3 episodes
Redundancy — 1× in this paper · explained in 2 episodes
Retrieval-Augmented Debating — 1× in this paper

Abstract

We extend the DS@GT ARC working-note submission to the Touch'e 2025 Retrieval-Augmented Debate task. The task has two subtasks: generating the next utterance in a simulated debate, and evaluating debate responses according to the Gricean maxims of Quantity, Quality, Relation, and Manner. The DS@GT ARC submission consisted of six leading LLMs from three providers through a retrieval-augmented prompting pipeline. We summarize the results from the working paper and explore whether multi-LLM evaluator agreement is a reliable proxy for official evaluation performance. The analysis shows that frontier LLM systems are strong response generators, and as evaluators they agree strongly within model families. However this consensus does not reliably track the official evaluation target, with the largest gap on the Quality maxim. The accompanying source code for this paper is located at https://github.com/dsgt-arc/touche-2025-rad and https://github.com/dsgt-arc/touche-2025-rad-analysis.

DS@GT ARC at Touch'e: Large Language Models for Retrieval-Augmented Debate

Terminology

Abstract

Sources

Related papers