How Much Human Label Variation Does Formal Semantic Structure Explain?: Group-Level Effects and Item-Level Ceilings in NLI
Researchers investigated how much formal semantic structure explains human disagreement in Natural Language Inference (NLI) using a large dataset. They found a group-level boundary where hypotheses not purely increasing in monotonicity show higher label entropy, but this structure does not predict high disagreement at the item level. Furthermore, the composition of disagreements remains unchanged across this boundary.