Nemotron 3.5 Super VL

Nemotron 3.5 Super VL's best Semi-Private scores are 22.7% on v1 at high reasoning and 1.9% on v2 at medium reasoning. We evaluated the model using NVIDIA's model-native reasoning modes: reasoning off (none), reasoning on with medium effort (medium), and reasoning on (high). In the underlying requests, none maps to chat_template_kwargs.enable_thinking: false, medium maps to enable_thinking: true with medium_effort: true, and high maps to enable_thinking: true with medium_effort: false. These labels describe the model-native chat-template controls, not Baseten's generic reasoning_effort mapping.

ARC-AGI 2 leaderboard

Verified scores

Tasks & environments

Pass/fail per reasoning level across each benchmark.

ARC-AGI-2 Public Eval

120 tasks

Task	High ✓✗	Medium ✓✗	None ✓✗
0934a4d8	✗	✗	✗
135a2760	✗	✗	✗
136b0064	✗	✗	✗
13e47133	✗	✗	✗
142ca369	✗	✗	✗
16b78196	✗	✗	✗
16de56c4	✗	✗	✗
1818057f	✓	✗	✗
195c6913	✗	✗	✗
1ae2feb7	✗	✗	✗
20270e3b	✗	✗	✗
20a9e565	✗	✗	✗
21897d95	✗	✗	✗
221dfab4	✗	✗	✗
247ef758	✗	✗	✗
269e22fb	✗	✗	✗
271d71e2	✗	✗	✗
28a6681f	✗	✗	✗
291dc1e1	✗	✗	✗
2b83f449	✗	✗	✗
2ba387bc	✓	✓	✗
2c181942	✗	✗	✗
2d0172a1	✗	✗	✗
31f7f899	✗	✗	✗
332f06d7	✗	✗	✗
35ab12c3	✗	✗	✗
36a08778	✗	✗	✗
38007db0	✗	✗	✗
3a25b0d8	✗	✗	✗
3dc255db	✗	✗	✗
3e6067c3	✗	✗	✗
409aa875	✗	✗	✗
446ef5d2	✗	✗	✗
45a5af55	✗	✗	✗
4a21e3da	✗	✗	✗
4c3d4a41	✗	✗	✗
4c416de3	✗	✗	✗
4c7dc4dd	✗	✗	✗
4e34c42c	✗	✗	✗
53fb4810	✗	✗	✗
5545f144	✗	✗	✗
581f7754	✗	✗	✗
58490d8a	✗	✗	✗
58f5dbd5	✗	✗	✗
5961cc34	✗	✗	✗
5dbc8537	✗	✗	✗
62593bfd	✗	✗	✗
64efde09	✗	✗	✗
65b59efc	✗	✗	✗
67e490f4	✗	✗	✗
6e453dd6	✗	✗	✗
6e4f6532	✗	✗	✗
6ffbe589	✗	✗	✗
71e489b6	✗	✗	✗
7491f3cf	✗	✗	✗
7666fa5d	✗	✗	✗
78332cb0	✗	✗	✗
7b0280bc	✗	✗	✗
7b3084d4	✗	✗	✗
7b5033c1	✗	✗	✗
7b80bb43	✗	✗	✗
7c66cb00	✗	✗	✗
7ed72f31	✗	✗	✗
800d221b	✗	✗	✗
80a900e0	✗	✗	✗
8698868d	✗	✗	✗
88bcf3b4	✗	✗	✗
88e364bc	✗	✗	✗
89565ca0	✗	✗	✗
898e7135	✗	✗	✗
8b7bacbf	✗	✗	✗
8b9c3697	✗	✗	✗
8e5c0c38	✗	✗	✗
8f215267	✗	✗	✗
8f3a5a89	✗	✗	✗
9385bd28	✗	✗	✗
97d7923e	✗	✗	✗
981571dc	✗	✗	✗
9aaea919	✗	✗	✗
9bbf930d	✗	✗	✗
a251c730	✗	✗	✗
a25697e4	✗	✗	✗
a32d8b75	✗	✗	✗
a395ee82	✗	✗	✗
a47bf94d	✗	✗	✗
a6f40cea	✗	✗	✗
aa4ec2a5	✗	✗	✗
abc82100	✗	✗	✗
b0039139	✗	✗	✗
b10624e5	✗	✗	✗
b5ca7ac4	✗	✗	✗
b6f77b65	✗	✗	✗
b99e7126	✗	✗	✗
b9e38dc0	✗	✗	✗
bf45cf4b	✗	✗	✗
c4d067a0	✗	✗	✗
c7f57c3e	✗	✗	✗
cb2d8a2c	✗	✗	✗
cbebaa4b	✗	✗	✗
d35bdbdc	✗	✗	✗
d59b0160	✗	✗	✗
d8e07eb2	✗	✗	✗
da515329	✗	✗	✗
db0c5428	✗	✗	✗
db695cfb	✗	✗	✗
dbff022c	✗	✗	✗
dd6b8c4b	✗	✗	✗
de809cff	✗	✗	✗
dfadab01	✗	✗	✗
e12f9a14	✗	✗	✗
e3721c99	✗	✗	✗
e376de54	✗	✗	✗
e8686506	✗	✗	✗
e87109e9	✗	✗	✗
edb79dae	✗	✗	✗
eee78d87	✗	✗	✗
f560132c	✗	✗	✗
f931b4a8	✗	✗	✗
faa9f03d	✗	✗	✗
fc7cae8d	✗	✗	✗

ARC-AGI-1 Public Eval

400 tasks

Task	High ✓✗	Medium ✓✗	None ✓✗
00576224	✓	✓	✓
009d5c81	✓	✓	✗
00dbd492	✗	✗	✗
03560426	✓	✗	✗
05a7bcf2	✗	✗	✗
0607ce86	✗	✗	✗
0692e18c	✓	✗	✗
070dd51e	✓	✗	✗
08573cc6	✗	✗	✗
0934a4d8	✗	✗	✗
09c534e7	✗	✗	✗
0a1d4ef5	✗	✗	✗
0a2355a6	✗	✗	✗
0b17323b	✓	✓	✗
0bb8deee	✓	✓	✗
0becf7df	✗	✓	✗
0c786b71	✓	✓	✗
0c9aba6e	✓	✓	✗
0d87d2a6	✗	✗	✗
0e671a1a	✗	✗	✗
0f63c0b9	✓	✓	✗
103eff5b	✗	✗	✗
11e1fe23	✗	✗	✗
12422b43	✓	✓	✗
12997ef3	✓	✓	✗
12eac192	✗	✗	✗
136b0064	✗	✗	✗
13713586	✗	✗	✗
137f0df0	✓	✗	✗
140c817e	✓	✓	✗
14754a24	✗	✗	✗
15113be4	✗	✗	✗
15663ba9	✗	✗	✗
15696249	✗	✗	✗
16b78196	✗	✗	✗
17b80ad2	✗	✗	✗
17cae0c1	✓	✓	✗
18419cfa	✗	✗	✗
184a9768	✗	✗	✗
195ba7dc	✓	✓	✗
1990f7a8	✓	✗	✗
19bb5feb	✓	✓	✗
1a2e2828	✓	✓	✓
1a6449f1	✓	✗	✗
1acc24af	✗	✗	✗
1c02dbbe	✗	✗	✗
1c0d0a4b	✗	✗	✗
1c56ad9f	✗	✗	✗
1d0a4b61	✓	✗	✓
1d398264	✗	✗	✗
1da012fc	✗	✗	✗
1e81d6f9	✗	✗	✗
1e97544e	✗	✗	✓
2037f2c7	✗	✗	✗
2072aba6	✓	✓	✓
20818e16	✗	✗	✗
20981f0e	✗	✗	✗
212895b5	✗	✗	✗
21f83797	✓	✓	✗
22a4bbc2	✗	✓	✗
25094a63	✗	✗	✗
2546ccf6	✗	✗	✗
256b0a75	✗	✗	✗
2685904e	✓	✗	✗
2697da3f	✓	✗	✗
2753e76c	✓	✓	✗
27a77e38	✗	✗	✗
27f8ce4f	✓	✗	✗
281123b4	✗	✗	✗
292dd178	✗	✗	✗
29700607	✗	✗	✗
2a5f8217	✗	✗	✗
2b01abd0	✓	✓	✗
2c0b0aff	✗	✗	✗
2c737e39	✓	✓	✗
2f0c5170	✗	✗	✗
310f3251	✓	✗	✗
3194b014	✗	✓	✗
319f2597	✓	✗	✗
31adaf00	✗	✗	✗
31d5ba1a	✓	✓	✗
32e9702f	✓	✓	✗
332efdb3	✓	✓	✓
3391f8c0	✗	✗	✗
33b52de3	✓	✗	✗
3490cc26	✗	✗	✗
34b99a2b	✗	✓	✗
351d6448	✗	✗	✗
358ba94e	✓	✓	✗
37d3e8b2	✗	✗	✗
3979b1a8	✓	✓	✓
3a301edc	✗	✗	✗
3b4c2228	✓	✓	✗
3d31c5b3	✓	✓	✗
3ed85e70	✗	✗	✗
3ee1011a	✓	✓	✗
3f23242b	✓	✗	✗
40f6cd08	✗	✗	✗
414297c0	✗	✗	✗
423a55dc	✓	✓	✗
42918530	✗	✗	✗
42a15761	✗	✗	✗
4364c1c4	✓	✗	✗
456873bc	✗	✗	✗
45737921	✓	✓	✓
45bbe264	✓	✓	✗
477d2879	✗	✗	✗
47996f11	✗	✗	✗
48131b3c	✓	✓	✗
4852f2fa	✓	✓	✗
48f8583b	✓	✓	✗
4aab4007	✗	✗	✗
4acc7107	✗	✗	✗
4b6b68e5	✗	✗	✗
4c177718	✗	✗	✗
4cd1b7b2	✓	✓	✗
4e45f183	✗	✗	✗
4e469f39	✗	✗	✗
4f537728	✓	✓	✗
4ff4c9da	✗	✗	✗
505fff84	✓	✓	✗
506d28a5	✓	✓	✗
50a16a69	✓	✓	✓
50aad11f	✗	✗	✗
50f325b5	✗	✗	✗
516b51b7	✗	✗	✗
5207a7b5	✗	✓	✗
5289ad53	✓	✓	✗
52fd389e	✗	✗	✗
54db823b	✗	✗	✗
55059096	✓	✗	✗
551d5bf1	✗	✗	✗
55783887	✗	✗	✗
575b1a71	✓	✓	✗
5783df64	✓	✓	✗
5833af48	✗	✗	✗
58743b76	✗	✗	✗
58e15b12	✗	✗	✗
59341089	✓	✓	✗
5a5a2103	✗	✗	✗
5af49b42	✓	✓	✗
5b526a93	✗	✗	✗
5b692c0f	✗	✗	✗
5b6cbef5	✓	✓	✗
5d2a5c43	✓	✓	✗
5ffb2104	✗	✗	✗
604001fa	✗	✗	✗
60a26a3e	✗	✗	✗
60c09cac	✓	✓	✗
626c0bcc	✗	✗	✗
62ab2642	✗	✗	✗
62b74c02	✓	✓	✓
639f5a19	✗	✗	✗
642248e4	✓	✓	✗
642d658d	✓	✗	✗
64a7c07e	✓	✓	✗
66e6c45b	✓	✓	✓
66f2d22f	✗	✗	✗
67636eac	✗	✓	✗
67b4a34d	✗	✗	✗
67c52801	✓	✗	✗
68b67ca3	✓	✓	✓
692cd3b6	✗	✗	✗
695367ec	✓	✓	✗
696d4842	✗	✗	✗
69889d6e	✗	✗	✗
6a11f6da	✓	✓	✗
6ad5bdfd	✗	✗	✗
6df30ad6	✓	✗	✗
6ea4a07e	✓	✓	✓
6f473927	✓	✓	✗
7039b2d7	✗	✓	✗
705a3229	✓	✓	✗
712bf12e	✗	✗	✗
72207abc	✗	✗	✗
72a961c9	✓	✓	✗
73182012	✓	✓	✗
73c3b0d8	✓	✗	✗
73ccf9c2	✗	✗	✗
759f3fd3	✓	✓	✗
762cd429	✗	✗	✗
770cc55f	✓	✓	✗
782b5218	✓	✗	✗
79369cc6	✗	✗	✗
7953d61e	✓	✓	✗
79fb03f4	✗	✗	✗
7bb29440	✗	✗	✗
7c8af763	✗	✗	✗
7c9b52a0	✗	✗	✗
7d18a6fb	✗	✓	✗
7d1f7ee8	✗	✗	✗
7d419a02	✗	✗	✗
7e02026e	✗	✓	✗
7ee1c6ea	✓	✓	✗
817e6c09	✗	✗	✗
81c0276b	✓	✓	✗
833dafe3	✓	✓	✗
845d6e51	✗	✗	✗
84db8fc4	✗	✓	✗
84f2aca1	✓	✗	✗
8597cfd7	✗	✗	✗
85b81ff1	✗	✗	✗
85fa5666	✓	✗	✗
8719f442	✗	✗	✗
88207623	✗	✗	✗
891232d6	✗	✗	✗
896d5239	✗	✗	✗
8a371977	✗	✗	✗
8b28cd80	✗	✗	✗
8ba14f53	✗	✗	✗
8cb8642d	✗	✗	✗
8dae5dfc	✓	✗	✗
8e2edd66	✓	✓	✗
8ee62060	✓	✓	✓
8fbca751	✗	✗	✗
90347967	✗	✗	✗
903d1b4a	✓	✗	✗
9110e3c5	✓	✓	✗
917bccba	✓	✗	✗
929ab4e9	✗	✗	✗
92e50de0	✗	✗	✗
9356391f	✓	✗	✗
93b4f4b3	✗	✗	✗
93c31fbe	✗	✗	✗
94133066	✗	✗	✗
94414823	✓	✓	✗
94be5b80	✗	✗	✗
95a58926	✓	✗	✗
963f59bc	✗	✗	✗
96a8c0cd	✗	✗	✗
97239e3d	✗	✗	✗
9772c176	✗	✗	✗
981571dc	✗	✗	✗
992798f6	✓	✗	✗
99306f82	✗	✗	✗
9a4bb226	✗	✓	✗
9b2a60aa	✗	✗	✗
9b365c51	✗	✗	✗
9b4c17c4	✓	✓	✗
9bebae7a	✗	✗	✗
9c1e755f	✗	✓	✗
9c56f360	✓	✓	✗
9caba7c3	✗	✗	✗
9ddd00f0	✗	✗	✗
9def23fe	✗	✗	✗
9f27f097	✓	✓	✗
a04b2602	✗	✗	✗
a096bf4d	✗	✗	✗
a3f84088	✗	✗	✗
a406ac07	✓	✓	✓
a57f2f04	✗	✗	✗
a59b95c0	✓	✓	✗
a680ac02	✗	✗	✗
a8610ef7	✗	✗	✗
a934301b	✓	✗	✗
aa18de87	✓	✓	✓
aa300dc3	✓	✓	✗
aa4ec2a5	✗	✗	✗
aab50785	✗	✓	✗
ac0c5833	✗	✗	✗
ac2e8ecf	✗	✗	✗
ac3e2b04	✗	✗	✗
ac605cbb	✗	✗	✗
ad7e01d0	✓	✓	✗
ae58858e	✓	✓	✗
aee291af	✗	✗	✗
af22c60d	✗	✗	✗
af24b4cc	✗	✓	✗
b0722778	✗	✗	✗
b0f4d537	✗	✗	✗
b15fca0b	✗	✗	✗
b1fc8b8e	✗	✓	✗
b20f7c8b	✗	✗	✗
b457fec5	✗	✗	✗
b4a43f3b	✓	✗	✗
b7999b51	✓	✗	✗
b7cb93ac	✗	✗	✗
b7f8a4d8	✗	✗	✗
b7fb29bc	✗	✗	✗
b942fd60	✗	✗	✗
b9630600	✗	✗	✗
ba9d41b8	✗	✗	✗
baf41dbf	✗	✗	✗
bb52a14b	✗	✗	✗
bbb1b8b6	✓	✗	✗
bc4146bd	✓	✓	✗
bcb3040b	✓	✓	✗
bd14c3bf	✗	✗	✗
be03b35f	✓	✗	✗
bf32578f	✗	✗	✗
bf699163	✗	✓	✗
bf89d739	✗	✗	✗
c074846d	✓	✓	✗
c1990cce	✓	✗	✗
c3202e5a	✗	✗	✗
c35c1b4c	✗	✗	✗
c48954c1	✓	✓	✗
c62e2108	✗	✗	✗
c64f1187	✗	✗	✗
c658a4bd	✗	✓	✗
c663677b	✗	✗	✓
c6e1b8da	✗	✗	✗
c7d4e6ad	✓	✓	✓
c87289bb	✗	✗	✗
c8b7cc0f	✓	✓	✓
c92b942c	✗	✗	✗
c97c0139	✗	✗	✗
ca8de6ea	✓	✓	✓
ca8f78db	✗	✗	✗
cad67732	✗	✗	✗
cb227835	✗	✗	✗
ccd554ac	✓	✓	✗
cd3c21df	✓	✓	✗
ce039d91	✗	✗	✗
ce8d95cc	✗	✓	✗
cf133acc	✗	✓	✗
cfb2ce5a	✗	✗	✗
d017b73f	✗	✗	✗
d19f7514	✓	✓	✗
d282b262	✗	✗	✗
d2acf2cb	✓	✓	✗
d304284e	✗	✗	✗
d37a1ef5	✓	✗	✗
d47aa2ff	✗	✗	✗
d492a647	✗	✗	✗
d4b1c2b1	✓	✓	✗
d4c90558	✗	✓	✗
d56f2372	✗	✗	✗
d5c634a2	✗	✗	✗
d931c21c	✗	✗	✗
d94c3b52	✗	✗	✗
da2b0fe3	✓	✓	✗
da515329	✗	✗	✗
dc2aa30b	✓	✗	✗
dc2e9a9d	✗	✗	✗
dd2401ed	✓	✓	✗
de493100	✗	✗	✗
df8cc377	✗	✗	✗
e0fb7511	✓	✗	✗
e133d23d	✓	✓	✗
e1baa8a4	✓	✓	✓
e1d2900e	✗	✗	✗
e2092e0c	✗	✗	✗
e21a174a	✓	✗	✗
e345f17b	✓	✓	✗
e4075551	✓	✗	✗
e41c6fd3	✗	✗	✗
e57337a4	✓	✓	✗
e5790162	✓	✗	✗
e5c44e8f	✗	✗	✗
e619ca6e	✗	✗	✗
e633a9e5	✓	✓	✓
e66aafb8	✗	✓	✗
e681b708	✗	✗	✗
e69241bd	✓	✓	✗
e6de6e8f	✗	✗	✗
e74e1818	✓	✓	✗
e760a62e	✗	✗	✗
e7639916	✓	✓	✗
e78887d1	✗	✗	✗
e7a25a18	✓	✓	✗
e7b06bea	✓	✓	✗
e7dd8335	✓	✓	✗
e872b94a	✓	✓	✗
e88171ec	✗	✗	✗
e95e3d8e	✓	✗	✗
e99362f0	✓	✓	✗
e9ac8c9e	✓	✓	✗
e9b4f6fc	✗	✓	✗
e9bb6954	✗	✗	✗
e9c9d9a1	✓	✓	✗
ea959feb	✗	✗	✗
ea9794b1	✓	✗	✗
ecaa0ec1	✓	✗	✗
ed74f2f2	✗	✗	✗
ed98d772	✓	✓	✗
ef26cbf6	✓	✓	✗
f0afb749	✗	✗	✗
f0df5ff0	✗	✗	✗
f21745ec	✗	✗	✗
f3b10344	✗	✗	✗
f3cdc58f	✗	✓	✗
f3e62deb	✓	✗	✗
f4081712	✗	✗	✗
f45f5ca7	✓	✓	✗
f5aa3634	✗	✗	✗
f5c89df1	✗	✗	✗
f823c43c	✗	✗	✗
f83cb3f6	✓	✗	✗
f8be4b64	✗	✗	✗
f9a67cb5	✗	✗	✗
f9d67f8b	✗	✗	✗
fafd9572	✗	✗	✗
fb791726	✓	✓	✗
fc754716	✓	✓	✓
fd096ab6	✗	✗	✗
fd4b2b02	✗	✗	✗
fe9372f3	✓	✗	✗
fea12743	✗	✗	✗
ff72ca3e	✗	✗	✗