On a simulated pegboard, five of eight coding models scored two percent or less at hanging tools, even after two opportunities to revise their programs. On a gear-meshing task, six of the same models achieved a perfect score. The contrast comes from VABAR, an industrial manipulation benchmark developed by researchers at UC Berkeley, NVIDIA and Siemens.
The researchers held the robot, sensors and workcell fixed while varying the objects or their starting positions within specified bounds. That setting resembles a station handling a family of parts. The equipment stays put, but a single prerecorded movement will miss a tool lying at a different angle.
Each coding agent received the same library of robot skills and could revise its program using feedback from simulated executions. The team compared results on previously held-out objects, reporting means over five runs. Its comparison covered five tasks drawn from a six-task suite. A human-authored policy achieved 87 percent on tool hanging, where the leading model reached 60 percent. On cable routing the ordering reversed, with the human policy scoring 47 percent and every model exceeding it.
What a workcell average hides
For a plant evaluating software for a particular station, those reversals matter more than a single overall ranking. A supplier might show excellent performance seating a gear while having little evidence about a hanging operation that requires the robot to leave a tool securely supported as the gripper withdraws.
The study also makes a practical distinction between code generation and commissioning. Programs can be inspected, rerun and compared in the same simulated cell. The published benchmark percentages describe those trials. A decision about putting a program on a line still turns on the physical tooling, parts and tolerances at that station.