ARC Prize

Founded by Gregory Kamradt in 2024

Benchmark and evaluate machine learning models on complex abstract reasoning puzzles

Pricing model

Demo

Rating

★ People love it

Time to value

Requires Expertise

You can use ARC Prize to evaluate artificial general intelligence models against fluid reasoning benchmarks that resist standard brute-force memorization. The organization runs open competitions where researchers submit novel algorithms to solve visual puzzle tasks designed to test human-like learning efficiency. You can track performance across global leaderboards, test models against ARC-AGI benchmarks, compete for multi-million-dollar rewards, and share open-source code to advance machine reasoning. It provides a standardized measurement environment to determine whether automated agents can adapt to novel tasks without prior direct training data.

What ARC Prize does

Download open benchmark datasets for offline model testingSubmit prediction outputs to competition leaderboards on KaggleInspect visual grid reasoning tasks to analyze failure modesTrack model accuracy scores across public and private evaluation splitsApply program synthesis techniques to generate puzzle transformation rulesEvaluate agentic systems on multi-turn reasoning challengesMeasures fluid reasoning ability rather than memorized web knowledgeHosts public leaderboards tracking state-of-the-art model performanceProvides verified evaluation sets resistant to training data contaminationDistributes multi-million dollar cash prize pools for open-source solutionsFeatures ARC-AGI-3 for evaluating multi-step autonomous agent behaviorsServes as an official benchmark resource for NIST CAISI standards