Boundaries Agree, Labels Do Not: Intra-Annotator Dynamics as a Kind of Training Data
cs.CL
Submitted: 2026-10-03
Updated: 2026-10-03
Code: https://github.com/Malificenta883/intra-annotator-dynamics
Sources
- Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization
- Training Compute-Optimal Large Language Models
- Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias
- Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark
- Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks
- Beyond Consensus: Perspectivist Modeling and Evaluation of Annotator Disagreement in NLP
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering