SafeVLA-Bench Leaderboard

SR ↑ native task success SBU ↓ successes that still violate a spec VSI ↓ severity of the worst violation

24: policies evaluated
5: evaluation suites
22,500: episodes scored
8: safety specifications
2026-09-26: last updated

RoboCasa-365 · Atomic-Seen

#	Model	Safety ↑	SR ↑	SBU ↓	VSI ↓
🥇	RLDX-1-FT-RC365	53.1%	58.4%#4 by SR	24.0%	0.222
🥈	GigaWorld-Policy-0.1	52.0%	43.1%#7 by SR	20.0%	0.255
🥉	π_0.5	51.2%	42.3%#8 by SR	19.8%	0.195
4	PRTS	51.2%	64.1%#3 by SR	30.4%	0.241
5	GR00T-N1.6	51.0%	53.6%#5 by SR	24.6%	0.253
6	Xiaomi-Robotics-1	50.1%	81.9%#1 by SR	37.7%	0.233
7	WorldDreamer	48.9%	68.3%#2 by SR	31.1%	0.276
8	π₀	44.1%	32.4%#9 by SR	12.3%	0.318
9	GR00T-N1.5	40.7%	47.2%#6 by SR	26.3%	0.268

LIBERO · mean over four suites

#	Model	Train	Safety ↑	SR ↑	SBU ↓	VSI ↓
🥇	VLA-Adapter	SFT	82.0%	97.6%#3 by SR	17.0%	0.104
🥈	InternVLA-M1	SFT	81.5%	96.4%#10 by SR	16.8%	0.118
🥉	VLANeXt	SFT	81.5%	97.6%#4 by SR	17.1%	0.110
4	X-VLA	SFT	81.0%	97.2%#5 by SR	17.1%	0.113
5	MolmoAct2	SFT	80.9%	97.9%#1 by SR	17.5%	0.107
6	SimVLA	SFT	80.2%	96.9%#8 by SR	18.2%	0.123
7	OpenVLA-OFT-7B	SFT	80.1%	97.8%#2 by SR	18.1%	0.122
8	UniVLA	SFT	79.5%	94.8%#12 by SR	17.1%	0.127
9	GR00T-N1.6	SFT	78.4%	97.1%#6 by SR	20.0%	0.125
10	RIPT-VLA-7B	RL	78.2%	97.1%#7 by SR	20.0%	0.122
11	π_0.5	SFT	77.6%	96.6%#9 by SR	21.4%	0.147
12	π₀	SFT	77.5%	93.9%#14 by SR	19.2%	0.140
13	OpenVLA-7B	SFT	77.4%	69.4%#18 by SR	10.8%	0.174
14	Cosmos-Policy-2B	SFT	77.1%	95.2%#11 by SR	21.5%	0.147
15	π-RL-130	RL	76.4%	93.0%#15 by SR	20.0%	0.135
16	SmolVLA	SFT	72.9%	73.2%#17 by SR	11.9%	0.206
17	GR00T-N1.7	SFT	72.0%	94.2%#13 by SR	24.0%	0.169
18	π₀-FAST	SFT	71.8%	86.6%#16 by SR	15.0%	0.214

Ranked on unrounded Safety, so rows that tie on screen need not tie in the ordering. Every LIBERO row runs 200 rollouts per suite (10 tasks × 20 episodes); every RoboCasa-365 row runs 900 (18 task families × 50 episodes). The RoboCasa-365 policies come from two independently seeded groups (π₀, π_0.5, GR00T-N1.5 and RLDX-1-FT-RC365 share one seed sequence), so comparisons across the groups are unpaired.

What is scored

RoboCasa-365 atomic-seen covers 18 kitchen task families at 50 episodes each; LIBERO covers four tabletop suites. Policies run through their native inference wrappers with fixed seeds, so the success rate reported here is the one the policy's own harness produces — we add the safety layer on top rather than re-implementing the task.

The four numbers

SR is the benchmark's native success rate. Safety is the fraction of all episodes satisfying every applicable STL specification. SBU (Succ-But-Unsafe) is the fraction of episodes that both succeed and violate — the failures an SR-only evaluation cannot see. VSI is a bounded worst-violation depth, so a policy can have few unsafe successes and still score badly when the violations it does commit are severe.

Submitting a result

Open a pull request against SafeVLA-Bench-release adding one JSON file under submissions/ with your policy name, the benchmark and suite, episode count, and the per-episode verdict file produced by scripts/compute_stl_robustness.py. We re-score from the raw trajectories rather than taking reported numbers, so please include the per-step logs or a link to them. Entries currently on this board were all evaluated by us.

Scores come from the same aggregate the paper reads, regenerated by scripts/make_leaderboard.py. RoboCasa-365 verdicts were recomputed from the per-step force traces in September 2026; earlier numbers derived from the action log alone understated contact violations. See the paper for the specification library and threshold sensitivity analysis.