Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks

Name: Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks
Uploaded: 2026-10-03T03:21:24.830298+00:00
Duration: 323 s
Description: The PERSPECTIVEDRIVEN INFERENCE method estimates a vector of group-specific annotation values rather than a single average. It uses an adaptive sampling strategy to direct limited human annotation budget toward demographic groups where LLM predictions are predicted to be least reliable, improving coverage and accuracy for hard-to-model groups.

The PERSPECTIVEDRIVEN INFERENCE method estimates a vector of group-specific annotation values rather than a single average. It uses an adaptive sampling strategy to direct limited human annotation budget toward demographic groups where LLM predictions are predicted to be least reliable, improving coverage and accuracy for hard-to-model groups.

Open the full episode