← robovista home

RoboVista · eval board

Multiple-choice spatial-reasoning questions over robot scenes — 460 scored (474 minus a 14-question blacklist). 30 models, run with a standard and (for most) a chain-of-thought prompt. updated 2026-10-15

RoboVista: Evaluating Vision-Language Models for Diverse Robot ApplicationsRSS 2026

Best overall

81.3%
gemini-4-argon · API

Best open-weight · new

55.2%
qwen3.5-397b-a17b · open-weight

Models evaluated

30 total
23 open-weight · 7 API
Contents
  1. Leaderboard — standard vs chain-of-thought
  2. Accuracy by application domain
  3. Scene understanding vs planning
  4. All runs

Leaderboard — standard vs chain-of-thought

Sorted by standard-prompt accuracy. CoT helps the Gemma 4 family (+3 to +4), the thinking-tuned variants and the Qwen3.6 dense model; it costs accuracy on most other open models.

standard prompt chain-of-thought Δ = CoT − standard

Accuracy by application domain

Standard-prompt runs. Domains follow the benchmark's taxonomy; Driving and Surgery are the smallest columns (n=20, n=32), so read those with wider error bars in mind.

Scene understanding vs planning

Standard-prompt runs, split by ability type (SC n=306, PL n=148 — planning here bundles decision-making, motion awareness and recovery). gemini-4-argon and gpt-6-astra score higher on planning than on scene questions and Gemini 3 Flash holds steady; nearly every other model drops.

scene understanding planning Δ = planning − scene

All runs

Exact figures for every completed pass.

modelweightsstandardcotΔSCPLreleased