ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring
Shengjie Li, Vincent Ng
cs.CL
Submitted: 2026-07-30
Comments: Accepted as a long paper to NAACL 2024
DOI: 10.18653/v1/2024.naacl-long.468
Code: https://github.com/samlee946/ICLE-PlusPlus
License: http://creativecommons.org/licenses/by/4.0/
The gist: The majority of the recently-developed models for automated essay scoring (AES) are evaluated solely on the ASAP corpus.
Terminology
Abstract
The majority of the recently-developed models for automated essay scoring (AES) are evaluated solely on the ASAP corpus. However, ASAP is not without its limitations. For instance, it is not clear whether models trained on ASAP can generalize well when evaluated on other corpora. In light of these limitations, we introduce ICLE++, a corpus of persuasive student essays annotated with both holistic scores and trait-specific scores. Not only can ICLE++ be used to test the generalizability of AES models trained on ASAP, but it can also facilitate the evaluation of models developed for newer AES problems such as multi-trait scoring and cross-prompt scoring. We believe that ICLE++, which represents a culmination of our long-term effort in annotating the essays in the ICLE corpus, contributes to the set of much-needed annotated corpora for AES research.
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering