AGI progress has stalled.
New ideas are needed.
Competition Details
ARC Prize 2024 is hosted on Kaggle.
Goals
- Increase the number of AI researchers exploring new ideas
- Open source progress towards AGI
- Accelerate the discovery of AGI
Key Dates
- Jun. 11, 2024 - ARC Prize 2024 Launch
- Nov. 10, 2024 - Code Submission Deadline
- Nov. 12, 2024 - Paper Submission Deadline
- Dec. 6, 2024 - 2024 Winners Announced
- Q1 2025 - ARC Prize 2025 Update
Prizes
The spirit of ARC Prize is to open source progress towards AGI. To win prize money, you will be required to publish reproducible code/methods into public domain.
ARC Prize measures AGI progress using the ARC-AGI private evaluation set, the leaderboard is here, and the Grand Prize is unlocked once the first team reaches at least 85%.
Note: the private evaluation set imposes limitations on solutions (eg. no internet access, so no GPT-4/Claude/etc). There is a secondary leaderboard called ARC-AGI-Pub, it measures the public evaluation set and imposes no limits but it is not part of ARC Prize 2024 at this time.
Grand Prize: $600,000!
Awarded to the top teams (up to 5) which score at least 85% during an annual competition period. If not won, the Grand Prize will continue during the next annual competition.
2024 Top Scores: $50,000
Awarded to the highest-scoring submissions during the 2024 competition.
- 1st: $25,000
- 2nd: $10,000
- 3rd: $5,000
- 4th: $5,000
- 5th: $5,000
2024 Paper Awards: $75,000
Awarded to the submitted paper that best advances our understanding of how to achieve strong performance on ARC-AGI according to the Paper Award Rubric.
- 1st: $50,000
- 2nd: $20,000
- 3rd: $5,000
2024 Extra Prize Pool: $375,000
If any team reaches the Grand Prize 85% mark during ARC Prize 2024, we’ll increase the Grand Prize pool with all remaining ARC Prize funds up to $375k.
Initiatives
To solve key competition challenges, we have committed funds to a few impactful initiatives.
ARC-AGI-Pub: $150,000
To help support those contributing to the public leaderboard, we are committing funds to offset costs incurred for submission verification.
Events: $75,000
In a proactive effort to grow the number of contestants and increase progress towards conceptual breakthroughs, we will embark on a university tour in October 2024 to visit more than a dozen US universities with top-tier AI programs on the east and west coasts. We will also disseminate learnings at NeurIPS 2024.
ARC-AGI v2: $75,000
To solve known limitations and make ARC-AGI the best AGI research target, we will create the next iteration of the benchmark through new data creation and curation and human difficulty testing.
Join the Community
No matter where you are in the world, what background you have, or your current programming skill level, if you're at all interested in AGI, you should join the ARC Prize community!
Join our Discord server to ask questions, learn about what's going on, and hang out with fellow contestants and interested onlookers.
Receive updates about ARC-AGI Progress on our blog.
Rules
See the ARC Prize 2024 official rules on Kaggle.
Prizes are awarded at the sole discretion of ARC Prize Inc. and are subject to review by our Technical Team. We may issue more or fewer awards based on the spirit of the prize and the received submissions.
Open Source License
In order for a submission to be eligible, all code and methods authored by the submitter must be made open source under a permissive public domain license (eg. CC0 or MIT-0). Additionally, any 3rd party code or methods not authored by the submitter must be available under, at least, an open source license which allows public sharing (eg. Apache-2.0, GPLv3).
Grand Prize
The Grand Prize will be unlocked when a team surpasses a score of 85% accuracy on ARC-AGI private evaluation tasks. At the end of the annual competition, the Grand Prize will be divided among the Top 5 (or fewer) teams that have surpassed 85% accuracy.
If no submission reaches the 85% mark during ARC Grand Prize 2024, there will be no ARC Grand Prize winner in 2024 and prizes will be held for future competitions.
2024 Progress Prizes
Top Score Award
The Top Score Award will be awarded to the submission with the highest score on the private evaluation set during the 2024 competition. The top 5 teams with the highest scores will each be awarded with the Top Score Award.
2024 Paper Award
To be eligible for a Paper Award, separately submit a paper (PDF, arXiv, txt, etc.) documenting and describing the conceptual approach of an eligible ARC Prize 2024 Kaggle submission. Papers must be submitted within 48 hours of the competition ending.
Paper awards are evaluated equally based on the following rubric, consisting of six components, with a score ranging from 0 (lowest) to 5 (highest) given in each category.
| Category | Description |
|---|---|
| Accuracy | How accurate is the submission based on its performance on the leaderboard? (Note: you will be asked to provide your submission ID to match your notebook to your submission) |
| Universality | How general and universal is the Submission approach beyond the competition? Does your submission translate to other similar problems? How well does your method generalize? |
| Progress | How much does the paper increase the overall chance of anyone achieving 85% on ARC-AGI? |
| Theory | How well does the paper describe why the Submission works (as opposed to merely describing how it works)? |
| Completeness | How thoroughly and completely does the paper cover your submission to the leaderboard? |
| Novelty | How novel is the Submission relative to existing public research? |
The Paper Awards will be awarded to the 3 submissions with the most points during the 2024 competition. Paper rubric evaluations will not be shared. In the event of a tie, the paper that was entered first to the competition will be the winner. In the event a potential winner is disqualified for any reason, the paper that received the next highest score rank will be chosen as the potential winner.
Papers must be submitted via the Kaggle submission form.
