Improving Information Extraction with Learned Queries

arXiv:2608.31058 · cs.CL · Submitted 2026-08-31 · Read on arXiv

cs.CL

Submitted: 2026-08-31

Updated: 2026-08-31

Comments: Accepted in EMNLP-2026, 21 Pages

Code: https://github.com/ZhaoyueSun/PHEE

License: http://creativecommons.org/licenses/by/4.0/

The gist: When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning.

Terminology

Abstract

When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.

Sources

Related papers