AGI progress has stalled.
New ideas are needed.
Competition Details
ARC Prize 2024 is being hosted on Kaggle.
Goals
- Increase the number of AI researchers exploring new ideas
- Open source progress towards AGI
- Accelerate the discovery of AGI
Key Dates
- June 11, 2024 - ARC Prize 2024 Launch
- November 10, 2024 - Prize Submission Deadlines
- December 3, 2024 - 2024 Prizes Announced
- Q1 2025 - ARC Prize 2025 Update
Prizes
The spirit of ARC Prize is to open source progress towards AGI. To win prize money, you will be required to publish reproducible code/methods into public domain.
ARC Prize measures AGI progress using the ARC-AGI private evaluation set, the leaderboard is here, and the Grand Prize is unlocked once the first team reaches at least 85%.
Note: the private evaluation set imposes limitations on solutions (eg. no internet access, so no GPT-4/Claude/etc). There is a secondary leaderboard (in beta) called ARC-AGI-Pub, it measures the public evaluation set and imposes no limits but it is not part of ARC Prize 2024 at this time.
Grand Prize: $500,000!
Awarded to the top teams (up to 5) which score at least 85% during an annual competition period. If not won, the Grand Prize will continue during the next annual competition.
2024 Top Scores: $50,000
Awarded to the highest-scoring submissions during the 2024 competition.
- 1st: $25,000
- 2nd: $10,000
- 3rd: $5,000
- 4th: $5,000
- 5th: $5,000
2024 Paper Award: $50,000
Awarded to the submitted paper that best advances our understanding of how to achieve strong performance on ARC-AGI according the Paper Award Rubric.
- Winner: $45,000
- Runner-Up: $5,000
Note: Additional $500K of prizes to be announced later, subscribe. All prizes in USD.
Join the Community
No matter where you are in the world, what background you have, or your current programming skill level, if you're at all interested in AGI, you should join the ARC Prize community!
Join our Discord server to ask questions, learn about what's going on, and hang out with fellow contestants and interested onlookers.
Receive updates about ARC-AGI Progress on our blog.
Rules
Prizes are awarded at the sole discretion of ARC Prize Inc. and are subject to review by our Technical Team. We may issue more or fewer awards based on the spirit of the prize and the received submissions.
See ARC Prize 2024 official rules.
Open Source License
In order for a submission to be eligible, all code and methods authored by the submitter must be made open source under a permissive public domain license (eg. CC0 or MIT-0). Additionally, any 3rd party code or methods not authored by the submitter must be available under, at least, an open source license which allows public sharing (eg. Apache-2.0, GPLv3).
Grand Prize
The Grand Prize will be unlocked when a team surpasses a score of 85% accuracy on ARC-AGI private evaluation tasks. At the end of the annual competition, the Grand Prize will be divided among the Top 5 (or fewer) teams that have surpassed 85% accuracy.
If no submission reached the 85% mark during ARC Grand Prize 2024, there will be no ARC Grand Prize winner in 2024 and prizes will be held for future competitions.
2024 Progress Prizes
Top Score Award
The Top Score Award will be awarded to the submission with the highest score on the private evaluation set during the 2024 competition. The top 5 teams with the highest scores will be awarded with the Top Score Award.
2024 Paper Award
To be eligible for a Paper Award, separately submit a paper (PDF, arXiv, txt, etc.) documenting and describing the conceptual approach of an eligible ARC Prize 2024 Kaggle submission. Papers must be submitted within 48 hours of the competition ending.
Paper Awards are evaluated equally on the following six components, where a score of 0 (lowest) and 5 (highest) is given in each category.
| Category | Description |
|---|---|
| Accuracy | How accurate is the submission based on its performance on the leaderboard? (Note: you will be asked to provide your submission ID to match your notebook to your submission) |
| Universality | How general and universal is the Submission approach beyond the competition? Does your submission translate to other similar problems? How well does your method generalize? |
| Progress | How much does the paper increase the overall chance of anyone achieving 85% on ARC Prize? |
| Theory | How well does the paper describe why the Submission works (as opposed to merely describing how it works)? |
| Completeness | How thoroughly and completely does the paper cover your submission to the leaderboard? |
| Novelty | How novel is the Submission relative to existing public research? |
The Paper Award will be awarded to the Submission with the most points and the Paper Award Runner Up will the second most points. Paper rubric evaluations will not be shared. In the event of a tie, the Paper that was entered first to the Competition will be the winner. In the event a potential winner is disqualified for any reason, the Paper that received the next highest score rank will be chosen as the potential winner.
See here for paper award submission form.
