Published 29 Jul 2025
Last week, the White House released its AI Action Plan. We’re encouraged to see the White House take transparency and measurement seriously as catalysts for AI progress.
In our March 2025 submission to the White House Office of Science and Technology Policy (OSTP), ARC Prize Foundation laid out three recommendations:
All three made it into the final plan. Here’s where our recommendations show up:
We believe open source is essential for AI progress and back a growing community of researchers pushing the field forward. As co-founder Mike Knoop put it: “We are idea-constrained for AGI. The more openness, the stronger our innovation environment.”
The AI Action Plan echoes this view, highlighting open source as essential for research, innovation, and national competitiveness.
In our submission, we urged OSTP to “accelerate America’s path to advanced AI by defining what matters… [leveraging] benchmarks to spotlight technological challenges and focus AI researcher efforts toward solving them.”
The plan reflects this, with renewed federal support for breakthrough research that can “unlock entirely new capabilities.”
We recommended that “the Administration commit to maintaining a national AI benchmarking initiative, within an existing agency such as NIST/CAISI, to coordinate AI evaluation efforts across government and provide expert guidance.”
This is now policy. OSTP has tasked NIST, CAISI, DOE, and NSF with leading evaluation science, building testbeds, and convening the community. It’s the clearest endorsement yet of what ARC Prize has long argued: definitions drive evaluations, and evaluations drive progress.
We believe evaluations are strategic infrastructure, they generate ground truth about model capabilities and progress, which shapes public and private sector decisions across research, security, procurement, and policy. For example, in December 2024 we detected a discontinuous advancement in AI capabilities while benchmarking OpenAI’s o3-preview on ARC-AGI-PUB. Our analysis detailed what led to this step function improvement in generalization and adaptability, contextualizing the shift for the wider ecosystem.
ARC Prize sees benchmarks as both technical tools and diplomatic levers. As we wrote in our submission, ‘The nation that sets AI measurement standards has significant influence over the global AI ecosystem… and China is positioning itself as a dominant force in AI standards-setting. A China-led global AI benchmarking system could disadvantage U.S. firms.’
We recommended the US anchor its international engagement in open, scientifically validated benchmarks.
That vision is reflected in the plan’s call for, “Leveraging the U.S. position in international diplomatic and standard-setting bodies to… counter authoritarian influence.”
Many of the AI Action Plan’s core ideas - support for open-source, benchmark-driven research, evaluation infrastructure, and international standards engagement - mirror the recommendations ARC Prize submitted earlier this year. We crafted these proposals to align with US strategic priorities while staying rooted in open science and global collaboration.
ARC Prize Foundation will continue designing benchmarks that stretch system capabilities, creating testbeds for evaluation, and partnering with academia, industry, and civil society. Our latest benchmark, ARC-AGI-3, embodies this mission. It’s the first ‘interactive reasoning’ eval designed to measure skill-acquisition efficiency in unfamiliar environments, a proxy for general, human-like intelligence. ARC-AGI-3 goes beyond pattern recognition to test whether models can learn, adapt, and reason across new tasks, setting a new bar for what meaningful AI progress looks like.
Learn more about ARC-AGI-3.