Click a task to play its video.All task details →

Agentic RoboticsVariational Automation Benchmark (VABAR)

New in VABAR
Industrial
Built-in Skill Library
Deformable
6 task classes
Physics Feedback
Newton

Code (Nov 9)
https://arxiv.org/abs/2611.06881

Shuangyu Xie^3,*, Kaiyuan Chen³, Uksang Yoo³, Xinyue Zhu³, Meadow Shen³, Jaimyn Drake³, Eugen Solowjow¹, Lionel Gulich², Alexander Millane², Rafael Cathomen², Tuur Stuyck², Sangeeta Subramanian², Yichao Pan², Bruce McGowan², Michael Benedict², Linxi Fan², Yuke Zhu², Spencer Huang², Ken Goldberg³

¹Siemens²NVIDIA³UC Berkeley^*Equal contribution

1What is Variational Automation?

Variational Automation (VA) is a class of tasks where a robot persistently performs varying instances of a specific task, with known variation in object shape and pose: sorting tools into bins, hanging them on a pegboard, routing cables in a data centre. The workcell, robot and sensors are fixed and the range of objects and poses is bounded, so a system can reuse object models, calibrated sensors and modular skills while still adapting to every new instance.

Fixed Automation

A robot repeats identical instances with the same motions, as in spot welding or spray painting. Reliable and fast, but with no adaptivity.

Variational Automation

A robot repeats varying instances: objects vary in geometry and pose within a known, bounded workcell. This is what VABAR evaluates.

Generalist Robotics

A robot performs a broad variety of open-ended tasks, mostly with end-to-end policies. Highly flexible, but not yet at industrial reliability.

Eight top-down views of a tool tray. Top row: four widely separated singulated tools. Bottom row: four trays of overlapping tools. — **Figure 1.** Variational Automation on Tool Sorting with 2 difficulty levels. Top: Level 1, singulated tools with no distractors. Bottom: Level 2, where tools overlap and grasp selection must account for what is on top of what. Each column is an independently seeded instance.

Framework diagram. Left: task definition from initial configurations, goal configurations, an asset library and a robot definition. Top right: admissible task variation generation across object, arrangement and position, with Level 1 and Level 2 examples for each task. Bottom right: robot policy evaluation across the six tasks. — **Figure 2.** The VABAR evaluation framework. Left: tasks are specified by initial and goal configurations, an asset library, and robot definitions. Top right: admissible task variations span objects, arrangements and positions, with examples at two levels. Bottom right: robot policies are evaluated across the task suite.

2How coding agents do on VABAR

Agentic robotics systems write robot policies as code and improve them from what happens in simulation. We gave eight coding models the same skill library and the same execution feedback on five VABAR tasks. Zero-shot success says little about which agent will improve: models that start within a few points of each other end up 30 points apart after two rounds of feedback.

Overall, five tasks

Tool Hanging

Syringe Packing

Gear Meshing

USB-C Insertion

Cable Routing

zero-shot policy (@0)after two rounds of execution feedback (@2), logo = model providerhuman-authored policy

Figure 3. Coding agents on five VABAR tasks. Each agent writes a policy from the shared skill library, runs it in simulation, and edits its own code from execution feedback. Held-out per-object success (%), mean over five runs; rows sorted by final success; the best agent per task is ringed in blue (no ring where more than two agents tie). Opus 5 at xhigh effort. Tool Sorting was not part of this study.

3The skill library

VABAR extends the Modular Open Robot Skill Library with reusable procedures for functional-feature manipulation, constrained placement and deformable cable handling: metric feature localization, source–destination correspondence, in-hand registration and staged feature mating. Cable skills estimate and track ordered 3D centerlines and support anchoring, routing and terminal placement.

The 33-skill library in four groups: perception and state estimation, grasp proposal and execution, motion planning and execution, and verification. — **Figure 4.** The 33 skills span perception, grasping, motion planning and execution, and verification. Coloured cards marked “+” are 25 skill bundles added relative to the Graph-as-Policy library; white cards are eight retained counterparts.

4Variation at two levels

Each clip is a flip-book of initial conditions, not a trajectory: every frame is one independently seeded instance of the same task class, held for half a second. Level 1 and Level 2 run the same sampler under different bounds; what changes is the width of the admissible region, not the objective.

Varies: Tool type · clutter · bin occupancy

Level 1

Level 2

Varies: Tool type · fixture type · object and fixture poses

Level 1

Level 2

Varies: Gear count · gear size · initial pose

Level 1

Level 2

Varies: Syringe type · clutter · disposal geometry

Level 1

Level 2

Varies: Connector pose · port pose

Level 1

Level 2

Varies: Fixture layout · route · cable type

Level 1

Level 2

Figure 5. Seeded initial scenes per task and level. Instances are drawn under a controlled seed, so re-running a task specification reproduces the same scenes.

See task definitions, components and execution records →

Citation

@misc{xie2026vabar,
  title  = {VABAR: A Variational Automation Benchmark for Agentic Robotics},
  author = {Shuangyu Xie, Kaiyuan Chen, Uksang Yoo, Xinyue Zhu, Meadow Shen, Jaimyn Drake,
            Eugen Solowjow, Lionel Gulich, Alexander Millane, Rafael Cathomen, Tuur Stuyck,
            Sangeeta Subramanian, Yichao Pan, Bruce McGowan, Michael Benedict, Linxi Fan,
            Yuke Zhu, Spencer Huang, Ken Goldberg},
  year   = {2026},
  eprint = {2611.06881},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url    = {https://arxiv.org/abs/2611.06881}
}