Gemini 4 Carbon

Gemini 4 Carbon's best semi-private scores are 98.5% on v1 and 94.2% on v2, both at medium reasoning. On v3, it reaches 65.1% with the Standard harnessStandard harness enables a model to carry forward notes it chooses to keep with it throughout the environment. at low reasoning and 99.1% with the Provider Adapter harnessThe Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work. at medium reasoning. We omitted the standard-harness result at the highest reasoning setting because it completed only 30 of 55 games. Long gaps between actions led to game-session expirations, leaving the evaluation incomplete.

ARC-AGI 3 leaderboard

Verified scores

Tasks & environments

Pass/fail per reasoning level across each benchmark.

ARC-AGI-3 Public Demo - Standard harnessStandard harness enables a model to carry forward notes it chooses to keep with it throughout the environment.

25 environments

Game⇅	Medium⇅	Low⇅
AR25	100.0%	100.0%
CN04	100.0%	100.0%
DC22	100.0%	100.0%
FT09	100.0%	100.0%
LP85	100.0%	100.0%
M0R0	100.0%	100.0%
R11L	100.0%	97.1%
RE86	100.0%	77.8%
SB26	100.0%	100.0%
SP80	100.0%	100.0%
TR87	100.0%	47.6%
VC33	100.0%	100.0%
S5I5	99.2%	100.0%
TN36	90.6%	100.0%
WA30	80.0%	73.7%
CD82	80.0%	100.0%
SK48	77.8%	100.0%
SC25	73.4%	71.4%
KA59	66.5%	73.4%
SU15	46.7%	73.4%
LS20	26.4%	62.0%
TU93	4.6%	4.0%
LF52	3.2%	3.4%
BP35	0.7%	0.9%
G50T	0.0%	0.0%

ARC-AGI-3 Public Demo - Provider Adapter harnessThe Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work.

25 environments

Game⇅	High⇅	Medium⇅	Low⇅
AR25	100.0%	100.0%	100.0%
BP35	100.0%	100.0%	100.0%
CD82	100.0%	100.0%	100.0%
CN04	100.0%	100.0%	100.0%
DC22	100.0%	100.0%	100.0%
FT09	100.0%	100.0%	100.0%
G50T	100.0%	100.0%	100.0%
KA59	100.0%	100.0%	100.0%
LF52	100.0%	100.0%	27.3%
LP85	100.0%	100.0%	100.0%
LS20	100.0%	100.0%	100.0%
M0R0	100.0%	100.0%	100.0%
RE86	100.0%	100.0%	100.0%
S5I5	100.0%	100.0%	100.0%
SB26	100.0%	100.0%	100.0%
SC25	100.0%	100.0%	100.0%
SK48	100.0%	100.0%	100.0%
SP80	100.0%	100.0%	100.0%
SU15	100.0%	100.0%	100.0%
TN36	100.0%	100.0%	100.0%
TR87	100.0%	100.0%	100.0%
TU93	100.0%	100.0%	100.0%
VC33	100.0%	100.0%	100.0%
WA30	100.0%	100.0%	100.0%
R11L	98.6%	100.0%	100.0%

ARC-AGI-2 Public Eval

120 tasks

Task	High ✓✗	Medium ✓✗	Low ✓✗
0934a4d8	✓	✓	✓
135a2760	✓	✓	✓
136b0064	✓	✓	✓
13e47133	✓	✓	✗
142ca369	✓	✓	✗
16b78196	✓	✓	✗
16de56c4	✓	✓	✗
1818057f	✓	✓	✓
195c6913	✓	✓	✗
1ae2feb7	✓	✓	✓
20270e3b	✓	✓	✗
20a9e565	✓	✓	✗
21897d95	✓	✓	✗
221dfab4	✓	✓	✗
247ef758	✓	✓	✗
269e22fb	✓	✓	✗
271d71e2	✓	✓	✓
28a6681f	✓	✓	✓
291dc1e1	✓	✓	✓
2b83f449	✓	✓	✗
2ba387bc	✓	✓	✓
2c181942	✓	✓	✗
2d0172a1	✓	✓	✗
31f7f899	✓	✓	✓
332f06d7	✓	✓	✗
35ab12c3	✓	✓	✓
36a08778	✓	✓	✓
38007db0	✓	✓	✓
3a25b0d8	✓	✓	✓
3dc255db	✓	✓	✓
3e6067c3	✓	✓	✓
409aa875	✓	✓	✗
446ef5d2	✓	✓	✗
45a5af55	✓	✓	✓
4a21e3da	✓	✓	✓
4c3d4a41	✓	✓	✗
4c416de3	✓	✓	✓
4c7dc4dd	✓	✓	✗
4e34c42c	✓	✓	✗
53fb4810	✓	✓	✓
5545f144	✓	✓	✗
581f7754	✓	✓	✓
58490d8a	✓	✓	✓
58f5dbd5	✓	✓	✓
5961cc34	✓	✓	✓
5dbc8537	✓	✓	✗
62593bfd	✓	✓	✗
64efde09	✓	✓	✓
65b59efc	✓	✓	✗
67e490f4	✓	✓	✓
6e453dd6	✓	✓	✓
6e4f6532	✓	✓	✗
6ffbe589	✓	✓	✗
71e489b6	✓	✓	✓
7491f3cf	✓	✓	✗
7666fa5d	✓	✓	✗
78332cb0	✓	✓	✗
7b0280bc	✓	✓	✗
7b3084d4	✓	✓	✓
7b5033c1	✓	✓	✓
7b80bb43	✓	✓	✗
7c66cb00	✓	✓	✓
7ed72f31	✓	✓	✗
800d221b	✓	✓	✗
80a900e0	✓	✓	✗
8698868d	✓	✓	✗
88bcf3b4	✗	✗	✗
88e364bc	✗	✗	✗
89565ca0	✓	✓	✓
898e7135	✓	✓	✓
8b7bacbf	✓	✓	✗
8b9c3697	✓	✓	✓
8e5c0c38	✓	✓	✓
8f215267	✓	✓	✓
8f3a5a89	✓	✓	✓
9385bd28	✓	✓	✓
97d7923e	✓	✓	✓
981571dc	✓	✓	✓
9aaea919	✓	✓	✗
9bbf930d	✓	✓	✗
a251c730	✓	✓	✓
a25697e4	✓	✓	✗
a32d8b75	✓	✓	✗
a395ee82	✓	✓	✓
a47bf94d	✓	✓	✓
a6f40cea	✓	✓	✓
aa4ec2a5	✓	✓	✓
abc82100	✓	✓	✗
b0039139	✓	✓	✓
b10624e5	✓	✓	✓
b5ca7ac4	✓	✓	✗
b6f77b65	✓	✓	✗
b99e7126	✓	✓	✓
b9e38dc0	✓	✓	✗
bf45cf4b	✓	✓	✓
c4d067a0	✓	✓	✗
c7f57c3e	✓	✓	✓
cb2d8a2c	✓	✓	✗
cbebaa4b	✓	✓	✗
d35bdbdc	✓	✓	✓
d59b0160	✓	✓	✓
d8e07eb2	✓	✓	✗
da515329	✗	✓	✗
db0c5428	✓	✓	✓
db695cfb	✓	✓	✗
dbff022c	✓	✓	✓
dd6b8c4b	✓	✓	✗
de809cff	✓	✓	✗
dfadab01	✓	✓	✗
e12f9a14	✗	✗	✗
e3721c99	✓	✓	✓
e376de54	✓	✓	✓
e8686506	✓	✓	✓
e87109e9	✓	✓	✗
edb79dae	✓	✓	✓
eee78d87	✓	✓	✓
f560132c	✓	✓	✓
f931b4a8	✓	✓	✗
faa9f03d	✓	✓	✗
fc7cae8d	✓	✓	✗

ARC-AGI-1 Public Eval

400 tasks

Task	High ✓✗	Medium ✓✗	Low ✓✗
00576224	✓	✓	✓
009d5c81	✓	✓	✓
00dbd492	✓	✓	✓
03560426	✓	✓	✓
05a7bcf2	✓	✓	✓
0607ce86	✓	✓	✓
0692e18c	✓	✓	✓
070dd51e	✓	✓	✓
08573cc6	✓	✓	✓
0934a4d8	✓	✓	✗
09c534e7	✓	✓	✓
0a1d4ef5	✓	✓	✓
0a2355a6	✓	✓	✓
0b17323b	✓	✓	✓
0bb8deee	✓	✓	✓
0becf7df	✓	✓	✓
0c786b71	✓	✓	✓
0c9aba6e	✓	✓	✓
0d87d2a6	✗	✗	✗
0e671a1a	✓	✓	✓
0f63c0b9	✓	✓	✓
103eff5b	✓	✓	✓
11e1fe23	✓	✓	✓
12422b43	✓	✓	✓
12997ef3	✓	✓	✓
12eac192	✓	✓	✓
136b0064	✓	✓	✓
13713586	✓	✓	✓
137f0df0	✓	✓	✓
140c817e	✓	✓	✓
14754a24	✓	✓	✗
15113be4	✓	✓	✓
15663ba9	✓	✓	✓
15696249	✓	✓	✓
16b78196	✓	✓	✗
17b80ad2	✓	✓	✓
17cae0c1	✓	✓	✓
18419cfa	✓	✓	✗
184a9768	✓	✓	✓
195ba7dc	✓	✓	✓
1990f7a8	✓	✓	✓
19bb5feb	✓	✓	✓
1a2e2828	✓	✓	✓
1a6449f1	✓	✓	✓
1acc24af	✓	✓	✗
1c02dbbe	✓	✓	✓
1c0d0a4b	✓	✓	✓
1c56ad9f	✓	✓	✓
1d0a4b61	✓	✓	✓
1d398264	✓	✓	✓
1da012fc	✓	✓	✓
1e81d6f9	✓	✓	✓
1e97544e	✓	✓	✓
2037f2c7	✓	✓	✓
2072aba6	✓	✓	✓
20818e16	✓	✓	✓
20981f0e	✓	✓	✓
212895b5	✓	✓	✗
21f83797	✓	✓	✓
22a4bbc2	✓	✓	✗
25094a63	✓	✓	✓
2546ccf6	✓	✓	✓
256b0a75	✓	✓	✓
2685904e	✓	✓	✓
2697da3f	✓	✓	✓
2753e76c	✓	✓	✓
27a77e38	✓	✓	✓
27f8ce4f	✓	✓	✓
281123b4	✓	✓	✓
292dd178	✓	✓	✓
29700607	✓	✓	✓
2a5f8217	✓	✓	✓
2b01abd0	✓	✓	✓
2c0b0aff	✓	✓	✓
2c737e39	✓	✓	✓
2f0c5170	✓	✓	✓
310f3251	✓	✓	✓
3194b014	✓	✓	✓
319f2597	✓	✓	✓
31adaf00	✓	✓	✗
31d5ba1a	✓	✓	✓
32e9702f	✓	✓	✓
332efdb3	✓	✓	✓
3391f8c0	✓	✓	✓
33b52de3	✓	✓	✓
3490cc26	✓	✓	✓
34b99a2b	✓	✓	✓
351d6448	✓	✓	✓
358ba94e	✓	✓	✓
37d3e8b2	✓	✓	✓
3979b1a8	✓	✓	✓
3a301edc	✓	✓	✓
3b4c2228	✓	✓	✓
3d31c5b3	✓	✓	✓
3ed85e70	✓	✓	✓
3ee1011a	✓	✓	✓
3f23242b	✓	✓	✓
40f6cd08	✓	✓	✓
414297c0	✓	✓	✓
423a55dc	✓	✓	✓
42918530	✓	✓	✓
42a15761	✓	✓	✓
4364c1c4	✓	✓	✓
456873bc	✓	✓	✗
45737921	✓	✓	✓
45bbe264	✓	✓	✓
477d2879	✓	✓	✓
47996f11	✓	✓	✗
48131b3c	✓	✓	✓
4852f2fa	✓	✓	✓
48f8583b	✓	✓	✓
4aab4007	✓	✓	✓
4acc7107	✓	✓	✓
4b6b68e5	✓	✓	✓
4c177718	✓	✓	✓
4cd1b7b2	✓	✓	✓
4e45f183	✓	✓	✓
4e469f39	✓	✓	✓
4f537728	✓	✓	✓
4ff4c9da	✓	✓	✗
505fff84	✓	✓	✓
506d28a5	✓	✓	✓
50a16a69	✓	✓	✓
50aad11f	✓	✓	✓
50f325b5	✗	✗	✗
516b51b7	✓	✓	✓
5207a7b5	✓	✓	✓
5289ad53	✓	✓	✓
52fd389e	✓	✓	✓
54db823b	✓	✓	✓
55059096	✓	✓	✓
551d5bf1	✓	✓	✓
55783887	✓	✓	✓
575b1a71	✓	✓	✓
5783df64	✓	✓	✓
5833af48	✓	✓	✓
58743b76	✓	✓	✓
58e15b12	✓	✓	✓
59341089	✓	✓	✓
5a5a2103	✓	✓	✓
5af49b42	✓	✓	✓
5b526a93	✓	✓	✓
5b692c0f	✓	✓	✓
5b6cbef5	✓	✓	✓
5d2a5c43	✓	✓	✓
5ffb2104	✓	✓	✓
604001fa	✓	✓	✓
60a26a3e	✓	✓	✓
60c09cac	✓	✓	✓
626c0bcc	✓	✓	✓
62ab2642	✓	✓	✓
62b74c02	✓	✓	✓
639f5a19	✓	✓	✓
642248e4	✓	✓	✓
642d658d	✓	✓	✓
64a7c07e	✓	✓	✓
66e6c45b	✓	✓	✓
66f2d22f	✓	✓	✓
67636eac	✓	✓	✓
67b4a34d	✓	✓	✓
67c52801	✓	✓	✓
68b67ca3	✓	✓	✓
692cd3b6	✓	✓	✓
695367ec	✓	✓	✓
696d4842	✓	✓	✓
69889d6e	✓	✓	✓
6a11f6da	✓	✓	✓
6ad5bdfd	✓	✓	✓
6df30ad6	✓	✓	✓
6ea4a07e	✓	✓	✓
6f473927	✓	✓	✓
7039b2d7	✓	✓	✓
705a3229	✓	✓	✓
712bf12e	✓	✓	✓
72207abc	✓	✓	✓
72a961c9	✓	✓	✓
73182012	✓	✓	✓
73c3b0d8	✓	✓	✓
73ccf9c2	✓	✓	✓
759f3fd3	✓	✓	✓
762cd429	✓	✓	✓
770cc55f	✓	✓	✓
782b5218	✓	✓	✓
79369cc6	✓	✓	✗
7953d61e	✓	✓	✓
79fb03f4	✗	✗	✗
7bb29440	✓	✓	✓
7c8af763	✓	✓	✓
7c9b52a0	✓	✓	✓
7d18a6fb	✓	✓	✓
7d1f7ee8	✓	✓	✓
7d419a02	✓	✓	✗
7e02026e	✓	✓	✓
7ee1c6ea	✓	✓	✓
817e6c09	✓	✓	✓
81c0276b	✓	✓	✓
833dafe3	✓	✓	✓
845d6e51	✓	✓	✓
84db8fc4	✓	✓	✓
84f2aca1	✓	✓	✓
8597cfd7	✓	✓	✓
85b81ff1	✓	✓	✗
85fa5666	✓	✓	✗
8719f442	✓	✓	✓
88207623	✓	✓	✓
891232d6	✓	✓	✗
896d5239	✓	✓	✗
8a371977	✓	✓	✓
8b28cd80	✓	✓	✓
8ba14f53	✓	✓	✓
8cb8642d	✓	✓	✓
8dae5dfc	✓	✓	✓
8e2edd66	✓	✓	✓
8ee62060	✓	✓	✓
8fbca751	✓	✓	✓
90347967	✓	✓	✓
903d1b4a	✓	✓	✓
9110e3c5	✓	✓	✓
917bccba	✓	✓	✓
929ab4e9	✓	✓	✓
92e50de0	✓	✓	✓
9356391f	✓	✓	✓
93b4f4b3	✓	✓	✓
93c31fbe	✓	✓	✓
94133066	✓	✓	✓
94414823	✓	✓	✓
94be5b80	✓	✓	✓
95a58926	✓	✓	✓
963f59bc	✓	✓	✓
96a8c0cd	✓	✓	✗
97239e3d	✓	✓	✓
9772c176	✓	✓	✓
981571dc	✓	✓	✓
992798f6	✓	✓	✓
99306f82	✓	✓	✓
9a4bb226	✓	✓	✓
9b2a60aa	✓	✓	✓
9b365c51	✓	✓	✓
9b4c17c4	✓	✓	✓
9bebae7a	✓	✓	✓
9c1e755f	✓	✓	✓
9c56f360	✓	✓	✓
9caba7c3	✓	✓	✓
9ddd00f0	✓	✓	✓
9def23fe	✓	✓	✓
9f27f097	✓	✓	✓
a04b2602	✓	✓	✓
a096bf4d	✓	✓	✓
a3f84088	✓	✓	✓
a406ac07	✓	✓	✓
a57f2f04	✓	✓	✓
a59b95c0	✓	✓	✓
a680ac02	✓	✓	✓
a8610ef7	✓	✓	✓
a934301b	✓	✓	✓
aa18de87	✓	✓	✓
aa300dc3	✓	✓	✓
aa4ec2a5	✓	✓	✓
aab50785	✓	✓	✓
ac0c5833	✓	✓	✗
ac2e8ecf	✓	✓	✓
ac3e2b04	✓	✓	✓
ac605cbb	✓	✓	✓
ad7e01d0	✓	✓	✓
ae58858e	✓	✓	✓
aee291af	✓	✓	✓
af22c60d	✓	✓	✓
af24b4cc	✓	✓	✓
b0722778	✓	✓	✓
b0f4d537	✓	✓	✓
b15fca0b	✓	✓	✓
b1fc8b8e	✓	✓	✗
b20f7c8b	✓	✓	✓
b457fec5	✓	✓	✓
b4a43f3b	✓	✓	✓
b7999b51	✓	✓	✓
b7cb93ac	✓	✓	✓
b7f8a4d8	✓	✓	✓
b7fb29bc	✓	✓	✓
b942fd60	✓	✓	✗
b9630600	✓	✓	✗
ba9d41b8	✓	✓	✓
baf41dbf	✓	✓	✓
bb52a14b	✓	✓	✓
bbb1b8b6	✓	✓	✓
bc4146bd	✓	✓	✓
bcb3040b	✓	✓	✓
bd14c3bf	✓	✓	✓
be03b35f	✓	✓	✓
bf32578f	✓	✓	✓
bf699163	✓	✓	✓
bf89d739	✓	✓	✓
c074846d	✓	✓	✓
c1990cce	✓	✓	✓
c3202e5a	✓	✓	✓
c35c1b4c	✓	✓	✓
c48954c1	✓	✓	✓
c62e2108	✓	✓	✓
c64f1187	✓	✓	✓
c658a4bd	✓	✓	✓
c663677b	✓	✓	✓
c6e1b8da	✓	✓	✓
c7d4e6ad	✓	✓	✓
c87289bb	✓	✓	✓
c8b7cc0f	✓	✓	✓
c92b942c	✓	✓	✓
c97c0139	✓	✓	✓
ca8de6ea	✓	✓	✓
ca8f78db	✓	✓	✓
cad67732	✓	✓	✓
cb227835	✓	✓	✓
ccd554ac	✓	✓	✓
cd3c21df	✓	✓	✓
ce039d91	✓	✓	✓
ce8d95cc	✓	✓	✓
cf133acc	✓	✓	✓
cfb2ce5a	✓	✓	✓
d017b73f	✓	✓	✗
d19f7514	✓	✓	✓
d282b262	✓	✓	✓
d2acf2cb	✓	✓	✓
d304284e	✓	✓	✓
d37a1ef5	✓	✓	✓
d47aa2ff	✓	✓	✓
d492a647	✓	✓	✗
d4b1c2b1	✓	✓	✓
d4c90558	✓	✓	✓
d56f2372	✓	✓	✓
d5c634a2	✓	✓	✓
d931c21c	✓	✓	✓
d94c3b52	✓	✓	✗
da2b0fe3	✓	✓	✓
da515329	✗	✓	✗
dc2aa30b	✓	✓	✗
dc2e9a9d	✓	✓	✓
dd2401ed	✗	✗	✗
de493100	✓	✓	✓
df8cc377	✓	✓	✓
e0fb7511	✓	✓	✓
e133d23d	✓	✓	✓
e1baa8a4	✓	✓	✓
e1d2900e	✓	✓	✗
e2092e0c	✓	✓	✗
e21a174a	✓	✓	✓
e345f17b	✓	✓	✓
e4075551	✓	✓	✓
e41c6fd3	✓	✓	✓
e57337a4	✓	✓	✓
e5790162	✓	✓	✓
e5c44e8f	✓	✓	✓
e619ca6e	✓	✓	✓
e633a9e5	✓	✓	✓
e66aafb8	✓	✓	✓
e681b708	✓	✓	✓
e69241bd	✓	✓	✓
e6de6e8f	✓	✓	✓
e74e1818	✓	✓	✓
e760a62e	✓	✓	✓
e7639916	✓	✓	✓
e78887d1	✓	✓	✗
e7a25a18	✓	✓	✓
e7b06bea	✓	✓	✓
e7dd8335	✓	✓	✓
e872b94a	✓	✓	✓
e88171ec	✓	✓	✓
e95e3d8e	✓	✓	✓
e99362f0	✓	✓	✓
e9ac8c9e	✓	✓	✓
e9b4f6fc	✓	✓	✓
e9bb6954	✓	✓	✓
e9c9d9a1	✓	✓	✓
ea959feb	✓	✓	✓
ea9794b1	✓	✓	✓
ecaa0ec1	✓	✓	✓
ed74f2f2	✓	✓	✓
ed98d772	✓	✓	✓
ef26cbf6	✓	✓	✓
f0afb749	✓	✓	✓
f0df5ff0	✓	✓	✓
f21745ec	✓	✓	✓
f3b10344	✗	✗	✗
f3cdc58f	✓	✓	✓
f3e62deb	✓	✓	✓
f4081712	✓	✓	✓
f45f5ca7	✓	✓	✓
f5aa3634	✓	✓	✓
f5c89df1	✓	✓	✓
f823c43c	✓	✓	✓
f83cb3f6	✓	✓	✓
f8be4b64	✓	✓	✗
f9a67cb5	✓	✓	✓
f9d67f8b	✓	✓	✓
fafd9572	✓	✓	✓
fb791726	✓	✓	✓
fc754716	✓	✓	✓
fd096ab6	✓	✓	✓
fd4b2b02	✓	✓	✗
fe9372f3	✓	✓	✓
fea12743	✓	✓	✓
ff72ca3e	✓	✓	✓