On Benchmarking Human-like Intelligence in Machines
The episode discusses a paper titled "On Benchmarking Human-like Intelligence in Machines." The hosts explain how current AI benchmarks often fail to match human judgment, finding that labels are often flawed or ambiguous. They conclude that moving beyond single 'correct' answers and focusing on collecting real human data is necessary for building trustworthy, human-like AI.