Evaluating the Evaluator: Problems with SemEval-2020 Task 1 for Lexical Semantic Change Detection
This episode reviews a paper titled "Evaluating the Evaluator," which critically examines SemEval-2020 Task 1 for Lexical Semantic Change Detection. Hosts discuss major flaws, including narrow definitions of change, noisy input data (OCR errors), and limited test sets. The discussion concludes with a call for future AI research to adopt continuous modeling and greater data transparency to accurately track language evolution.