Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans
cs.AI, cs.LG
Submitted: 2026-07-13
Updated: 2026-11-17
Terminology
Sources
- Medical Large Language Model Benchmarks Should Prioritize Construct Validity
- When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks
- Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework
- Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks
- On the Measure of Intelligence
- ARC Prize 2024: Technical Report
- ARC Prize 2025: Technical Report
- The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
- Machine Psychology
- A Definition of AGI
- Fast and flexible: Human program induction in abstract reasoning tasks
- Watson & Holmes: A Naturalistic Benchmark for Comparing Human and LLM Reasoning
- Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
- AI and the Everything in the Whole Wide World Benchmark
- BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
- Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
- M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark
- Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead
- Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents