What the AA-Robotics Benchmark Shows About Robot Arm Control

AA-Robotics v0.1: What the Leaderboard Shows

Yes. On the AA-Robotics benchmark, frontier language models can control a robot arm for simple tabletop tasks, but they are far too slow and expensive for production work. At The Bot Scout, we cover humanoid and general-purpose robots across several hundred pages.

In AA-Robotics v0.1, published by Artificial Analysis on October 14, 2026, the best model, GPT-6 Astra, scored 69.4 out of 100. The models took about 30 minutes per task, while a person took about 20 seconds.

The full ranking is GPT-6 Astra at 69.4, GPT-6.1 Sol at 64.6, Gemini 4 Argon at 55.3, Claude Opus 5.5 at 48.8 and Claude Fable 5.1 at 46.3. Each figure is the model's mean score across the eight tasks.

No model got every task right. The best grasp success rate was 45%, for GPT-6 Astra, and GPT-6.1 Sol, the second-ranked model overall, held its target in only 28% of grasps.

Artificial Analysis itself says frontier language models are far too slow and expensive for any kind of production usage, "nearly two orders of magnitude worse than human baselines." It also says it expects this to change quickly. For background on the wider field, see our explainer on what AI robots are.

The AA-Robotics Benchmark Setup and Eight Tasks

The AA-Robotics benchmark gives each language model control of a 6-axis robot arm, with 5 joints and a gripper, plus a simple objective. The arm is a Hiwonder NexArm. Each model reaches it through a simple Hypertext Transfer Protocol (HTTP) application programming interface (API) from an isolated container.

Artificial Analysis gives the models brief instructions and no further description of the arm. Four cameras show the scene: three mounted around the table and one on the arm's wrist. The model writes and runs code inside the container to plan each movement and send it to the arm.

The models ran inside coding-agent harnesses at the high effort setting. GPT models ran in Codex, Claude models ran in Claude Code, and Gemini 4 Argon ran in Antigravity.

Every attempt is zero-shot. That means the model got no task-specific examples and no training on this arm before it started.

The model ends each attempt when it judges the task complete, or when it gives up. A human judge then scores the end state against the task's pass criteria. A finished task earns 100, and a partial result earns points for the progress made.

Artificial Analysis presents the benchmark as the start of an era of embodied artificial intelligence (AI). The eight tasks test control and dexterity, spatial reasoning, and adaptability, and each task has its own pass condition.

Task-by-Task Scores for Each Model

Every model passed two of the tasks, and the scores split sharply on the other six. All five models scored 100 on pressing the red button and on placing the crocodile toy in the rectangle.

Full marks on the other tasks were spread across different models. GPT-6.1 Sol was the only model to score 100 on tidying the blocks into the box. Sol and Claude Fable 5.1 were the only two to score 100 on the shape matching board.

GPT-6 Astra and Gemini 4 Argon both poured the drink for 100, while GPT-6.1 Sol scored 0 on the same task. GPT-6 Astra led the tower task with 71, and Sol and Fable both scored 0 there.

Task Time Against a Human Baseline

The models took about half an hour per task, while the human average was 20 seconds. Artificial Analysis puts the gap at roughly 100 times slower.

Four of the five models averaged about 30 minutes per task. Gemini 4 Argon was the slowest at 52 minutes. It spent an average of 21 minutes observing and reasoning before its first move, nearly twice as long as any other model.

Artificial Analysis notes that Gemini 4 Argon's activity times are estimates, because Antigravity does not timestamp each step.

Speed varied widely inside single tasks too. GPT-6 Astra pressed the red button in 2.2 minutes. Claude Fable 5.1 spent about ten minutes understanding and planning before its first action.

On the crocodile task, GPT-6.1 Sol declared itself done after five minutes. Claude Opus 5.5 took 45 minutes on that task.

Models often keep refining an attempt after reaching a passing state, and the extra work can undo the pass. GPT-6.1 Sol built a passing tower after about 17 minutes, kept going, and then failed the task.

If you are judging any system that uses a language model to drive an arm, ask for the time per attempt next to the success rate. Models in AA-Robotics v0.1 took about half an hour per task.

Cost per Task on the AA-Robotics Benchmark

On the AA-Robotics benchmark, the cheapest model cost $1.93 per task on average and the most expensive cost $16.36. GPT-6.1 Sol was the cheapest, and GPT-6 Astra was the most expensive.

$16.36 divided by $1.93 is about 8.5, so Astra costs more than eight times as much as Sol per task. That extra spend bought a score only four points higher, 69 against 65.

Artificial Analysis prices human work at $37.81 per hour. That figure is the average hourly earnings of all employees on United States (US) private nonfarm payrolls, from preliminary US Bureau of Labor Statistics data for September 2026.

At $37.81 an hour, 20 seconds of work costs about $0.21, because an hour holds 180 blocks of 20 seconds. Sol's $1.93 is about nine times that. Artificial Analysis says the most cost-effective model costs about ten times as much as a person.

Seventy percent of the model spend goes on input context. Artificial Analysis puts this down to the novelty of the arm and scene, plus the volume of visual information the models must process.

If cost per task drives your choice, GPT-6.1 Sol delivers most of the top score for about an eighth of the price. If the single highest score matters more, GPT-6 Astra leads at $16.36 a task.

Artificial Analysis drove these models through coding-agent harnesses, so treat every figure here as a research-preview cost for one small arm. For what physical arms cost, see our robot arm price guide.

Grasp Misses, Spills and Other Model Mistakes

The models struggled with precise measurement and with holding objects. On Measure the box, which needs each dimension within 2 millimeters, no model scored above 17. GPT-6 Astra, Gemini 4 Argon and Claude Fable 5.1 each scored 0.

Grasps were attempted in six tasks: Place Crocodile, Pour Drink, Tidy Box, Match Shapes, Build Tower and Measure Box. A grasp counted as successful only when the arm held the intended object through its lift or carry.

No model succeeded on half of its grasps. GPT-6 Astra's 45% was the best, and GPT-6.1 Sol's 28% was the lowest.

Artificial Analysis groups the errors into what it calls a "mistake wall," made up of three patterns. The examples in the table come from its write-up, with distances in centimeters (cm).

Ask any system that drives an arm with a language model how it confirms a task is done. Artificial Analysis recorded models reporting success on tasks that were unfinished.

Takeaways for Robot Buyers and Builders

The AA-Robotics benchmark shows the direction general-purpose language models are moving on simple physical tasks. It does not tell you how any robot you can buy will perform.

AA-Robotics v0.1 is a research preview of eight tabletop tasks on one small arm, scored by a human judge. It covers no industrial robot or humanoid, and its 100-times time gap describes this benchmark only.

If you are buying, treat an "AI-controlled" robot claim as unproven until the maker gives you numbers. See our guide on whether humanoid robots are autonomous.

If you are building, the benchmark describes a small-scale setup. It pairs an arm behind a simple HTTP API with cameras around the table and on the wrist, and a model that writes code to move the arm. Start with robotics for beginners and our LeRobot guide if you plan to build one.

This verdict does not suit you if you need an arm doing paid work today. No model in AA-Robotics v0.1 comes near human speed or cost, so see our robot arms guide instead.

Our answer would change if a later AA-Robotics release showed a model finishing these tasks near the 20-second human time, at a cost near the $0.21 human figure. A model scoring well on Measure the box would also change it.