ARC Prize Foundation - a North Star for AGI

Foreword from ARC Prize Co-founders Mike Knoop & François Chollet:

ARC Prize launched in 2024 as an experiment to promote the relatively obscure, but important, ARC-AGI benchmark. Now, after our OpenAI o3 testing, nearly everyone in tech has heard of it. But there is much more work to do to reach AGI. There remains many things that are easy/fast for humans, but hard/slow for AI.

Unlike most frontier AI benchmarks, we are not trying to measure AI risk with superhuman exam questions. Instead we are trying to inspire progress by promoting this gap in basic human capability. Future versions of the benchmark will focus on shrinking this gap towards zero.

ARC Prize has been exclusively funded by us so far, and to do this work, we’re growing ARC Prize into a proper non-profit foundation to act as a useful north star toward AGI. We're excited to announce that Greg Kamradt, who co-led ARC Prize 2024, will be taking on the role of ARC Prize President and joining the board to lead this effort.

ARC Prize 2024 Wrapped

In June we launched ARC Prize, a $1M competition to open source a ARC-AGI solution, to redirect the AI community towards AGI progress.

The result was, frankly, energy and momentum that we couldn’t have imagined.

I want to talk about what’s coming up in 2025, but before that, let’s look back on what we accomplished in 2024.

$1M Competition

Over 1.4K teams submitted 17K entries in the 2024 Kaggle competition. At launch, our $1M prize pool was the largest on Kaggle. Though the Grand Prize of $600K was not unlocked, we awarded $75K to 3 outstanding research papers and $50K to the top 5 teams who had the highest score against the ARC-AGI Private Evaluation.

Technical Paper & NeurIPS Highlight

We documented 2024's conceptual progress through our technical paper. This paper was presented at NeurIPS during the System 2 Reasoning At Scale workshop by François Chollet. He shared the stage with renowned experts Joshua Tenenbaum, Melanie Mitchell, and Jason Weston.

Media Spotlight

We were grateful to collaborate with various media partners to help spread the work about ARC Prize and its mission. This included top AI podcasts Dwarkesh Patel, No Priors, and Machine and Learning Street Talk, Cognitive Revolution, and Sequoia Capital, as well as press outlets like Time, Nature, New Scientist, The Information, and Forbes. We also saw a number of community-generated videos dedicated to ARC-AGI.

Thriving Community

As the community grows, so does ARC Prize. We wrapped 2024 with 3.4K Discord members, 15.4K Twitter followers, and 6.7K Newsletter subscribers.

ARC Prize 2024 University Tour

In Fall of ‘24, we visited 14 top AI Programs live on a US university tour (recap video). We also held a virtual event to share the tour with everyone. The final stop on the tour was a private gathering of top AI researchers at a private event co-hosted by a16z NYC.

AI Lab Focus

In 2024 we became aware of at least 8 distinct efforts to solve ARC-AGI by organizations that have greater than $1M in funding, including Basis AI, Tufa Labs, Agemo, and Symbolica + four others in stealth.

OpenAI o3 Live Stream

We partnered with OpenAI to test their new o3 system against ARC-AGI-1 and joined the final livestream for "12 Days of OpenAI" to feature ARC Prize.

Thank you to Sam and Mark for hosting us!

All of this impact was achieved with a small, self-funded team.

Bryan Landers and I led the charge as co-leads, with invaluable support from Mike Knoop and François Chollet, who volunteered their time and expertise as competition hosts.

2025: ARC Prize Foundation

Building on the momentum from 2024, we're evolving the ARC Prize Foundation to make ARC-AGI an academic, policy, and industry standard for measuring and guiding AGI.

Our unique strengths are:

1) Creating valuable, enduring benchmarks
2) Deploying benchmarks within the AI community

This combination enables us to have an outsized impact in the field.

New Leadership

To better align with this goal, I’m excited to announce a new leadership structure.

I am proud to take on the role of President of the ARC Prize Foundation. In addition to this, I will join the non-profit board with Mike & François to help guide the mission and vision of our work.

Expanding Our Impact

• Non-profit Status: We are transitioning to a 501(c)(3) non-profit organization, which will allow us to accept tax-deductible donations.
• Lab Partnerships: We're actively forging partnerships with frontier AI research labs. These collaborations will involve participation in the development of ARC-AGI-2 & 3 and model/system evaluation through early test access. If your lab wants to get involved, contact us at team@arcprize.org.
• Academic Network: We're committed to fostering a strong academic network around the ARC Prize Foundation, connecting researchers, students, and institutions to collaborate on solving the biggest challenges of abstraction and reasoning in AI.
• Fundraising Initiatives: To support our expanded vision, we will be launching a fundraise initiative mid-January to further expand our reach. If you’d like to learn more, contact us at team@arcprize.org.

Benchmark Evolution

• ARC-AGI-2: As we've previously shared, new, high-signal tasks are needed to further test the limits of AI reasoning abilities. This new dataset will be used in the 2025 competition. The updated benchmark and paper will be released in late Q1 2025.
• ARC-AGI-3: We're already hard at work on ARC-AGI-3, which will depart from the current paradigm of ARC-AGI-1. We are working in collaboration with leading labs and researchers to develop this. If you're interested in participating, please reach out to team@arcprize.org.

Get Involved

We want you to join us on this journey to AGI - whether you're a researcher, developer, student, or simply passionate about AI, there are many ways to get involved.

• Participate in the 2025 Competition: Put your skills to the test and compete for prizes and recognition. Join the newsletter to hear about when this will go out!
• Contribute to the Community: Share your ideas, collaborate with peers, and help build a vibrant research ecosystem. Join YouTube, Twitter, and Discord.
• Make a donation to the ARC Prize Foundation: Want to help the ARC Prize Foundation do more? Contact team@arcprize.org for more information.