OpenAI
6 min read

Has GPT-6 Astra Learned to Run Robots? A 12-Benchmark Study Says Yes in Simulation, Not Yet on Real Hardware

A SenseNova-led report tests GPT-6-Astra as a robot brain. With demo-based in-context learning and up to 10x steps, it beats prior bests on most of 12 benchmarks. On eight real robots, cost, hour-long trials, and safety stops limit it.

Has GPT-6 Astra Learned to Run Robots? A 12-Benchmark Study Says Yes in Simulation, Not Yet on Real Hardware

A new technical report asks a blunt question in its title: Has GPT-6-Astra Achieved Embodied Intelligence? The authors' own answer, in the abstract, is that the model "has largely crossed the threshold of embodied competence, but not yet that of practical embodied intelligence." In simulation, OpenAI's general-purpose model, with no weight updates for these tasks, beats the best published robot policies on most of 12 benchmarks. On real robots, the same approach runs into cost, time, and safety limits that simulation hides.

The report was posted to arXiv on October 15, 2026, by a large joint team publishing under the SenseNova name (the author list includes Dahua Lin, Ziwei Liu, and Chen Change Loy). At the time of writing it is listed as the #2 paper of the day on Hugging Face Daily Papers, and the team has posted initial harness code, so far for the LIBERO benchmarks, as SenseNova-RoboRSI.

If you need background on the model itself, our GPT-6 Astra explainer covers its benchmarks, pricing, and access, and our Gemini 4 Argon vs GPT-6 Astra vs Claude Opus 5.5 comparison covers pricing and general benchmarks.

What the team tested

The study covers 12 simulated settings in five groups: tabletop arms (LIBERO-Long++, LIBERO-PRO, LIBERO-Ultra-Long, RoboDojo, RoboLab), dexterous hands (Bench2Dex, DexJoCo), a humanoid (SIMPLE), long household tasks (BEHAVIOR), and vision-language navigation (VLN-CE R2R and RxR, LHPR-VLN). The model weights stay fixed throughout. Two protocol choices drive the results.

In-context learning from training demos. Conventional robot policies train on each benchmark's demonstration set. So the authors let GPT-6-Astra study the same set before testing. The model reads the demos and writes a "knowledge package": task guidance (goals, stages and strategies), motion references (demonstration trajectories and grasp configurations), visual references (selected demonstration frames), Python helpers for kinematics and action generation, completion checks with recovery procedures, and metadata on the robot and the source data. The package is frozen before testing. Every test episode starts a fresh session with that package, and nothing carries over between episodes. The authors say they did not prescribe this structure: the model chose what to keep, and the same six components recurred across benchmarks.

A larger step budget. Each benchmark has an official step limit. The team reports results at that limit (1x) and with up to 10 times as many steps in the same episode. The reason: the model often notices a failed action, changes its plan, and tries again, and a strict step limit cuts that off.

The headline numbers

At the standard budget, GPT-6-Astra with in-context learning matches or beats the best published result on most settings, according to the report's Table 2. With the 10x budget, the gaps grow.

Bar chart of success rates on four benchmarks: prior best published result, GPT-6-Astra at the standard step budget, and GPT-6-Astra with up to 10x steps
  • LIBERO-Ultra-Long: best published 44.0%, GPT-6-Astra 82.2% at 1x and 98.9% at up to 10x.
  • DexJoCo: best published 40.3%, GPT-6-Astra 78.2% at 1x and 87.3% at up to 10x.
  • Bench2Dex: best published 25.5%, GPT-6-Astra 25.6% at 1x and 61.5% at up to 10x.
  • LHPR-VLN: best published 5.3% (a Claude Code agent), GPT-6-Astra 21.1% at 1x and 28.9% at up to 10x.
  • RoboDojo: best published 31.4%, GPT-6-Astra 81.9% at up to 10x.
  • BEHAVIOR: best published 14.8%, GPT-6-Astra 37.0% at the official budget (no 10x run, because that budget is already large).

The LIBERO family and SIMPLE are close to saturated. Bench2Dex shows the step budget at work most clearly: at 1x the model only ties the prior best, and the gain comes from extra attempts.

Two caveats matter. First, the baselines are reported at the standard budget, so the 10x numbers are not a like-for-like comparison. Second, in-context learning helps most when the training demos closely match the test tasks. On RoboLab, which has no training set, the model leads the listed baselines but reaches only 57.1% at 1x. In RoboDojo's Open group, which has no task-specific demos, it posts its lowest score of the five RoboDojo groups.

The cost

This is the part that matters most for anyone thinking about deployment. The report says about 98% of tokens are cached input, and uses a rule of thumb of about 1 USD per million tokens.

  • On LIBERO-Long++, where success is above 97% at 1x, each episode still uses about one million tokens and nearly an hour of inference and tool use.
  • On LIBERO-Ultra-Long, each episode uses roughly nine million tokens and nearly two hours.
  • On RoboDojo with the extended budget, a single episode takes more than three hours on average.

Step budgets do not count model thinking time. In simulation the world pauses while the model reasons. A real robot does not.

Pairing a slow planner with a fast controller

The team also tested GPT-6-Astra as a "System 2" planner on top of a "System 1" robot policy, π0.5, on RoboDojo:

  • π0.5 alone: 6.91% average success.
  • Open-loop, where GPT-6-Astra sets subgoals and π0.5 executes them directly: 5.83%.
  • Closed-loop, where GPT-6-Astra reviews every proposed action: up to 20 points better than open-loop, at 7.664 million tokens.
  • Reviewing chunks of four actions instead of one: 25.83% success at 2.796 million tokens, 63.5% fewer tokens.
  • Closed-loop plus in-context learning: 54.17%, the best result, at 19.335 million tokens.

Action format also mattered. In a separate test without π0.5, direct joint-angle commands scored 5.84 points lower than the baseline of Cartesian end-effector targets. The authors read this as a gap between how a multimodal model reasons and how robots are controlled, and suggest giving the model semantic commands while low-level controllers handle precise motion.

What happened on real robots

The real-robot tests used eight configurations, from a single Franka arm to a Unitree G1 humanoid. There were no demonstrations, only human-written context such as tool schemas and joint limits, and a wall-clock budget of about 60 minutes per trial with an operator on the emergency stop.

Panel of zero-shot real-robot results: dual-arm tasks 14 of 14, block stacking 6 of 6, cloth folding 2 of 4, plug insertion 0 of 11, navigation on a wheeled base 9 of 23, humanoid toy pick-and-drop 2 of 3

Simple grasping and placing worked well. On a two-arm AgileX setup, the model went 14 for 14 across 12 tasks, including T-shirt folding. Plug insertion failed on all three arms where it was tried (0 of 11 trials). Navigation and search on a wheeled base succeeded 9 of 23 times.

The report lists three limits that simulation does not model:

  • Safety stops. Contacts beyond controller limits usually end the trial, and the model cannot recover from hardware faults.
  • Mechanical load. On one dexterous hand, finger joints stalled during repeated grasps until a temperature guard the model had set for itself (62 °C) stopped the attempt. Torso overheating ended both flower-collection runs on a Galaxea R1 Pro.
  • Real time. On the G1, GPT-6-Astra spent about 90% of each run deliberating, and a single manipulation trial took over an hour.

The authors also note that the model shows active vision: it moves the camera on purpose to get the view it needs for the next action.

Why it matters

The result shows that a frontier model plus a good harness can match specialized robot policies on standard benchmarks without robot-specific training. It also shows that those benchmarks now measure less than they used to. A 10x step budget and a frozen pack of demo-derived notes are reasonable in simulation, but a real deployment cannot wait an hour per task or retry after a dropped object.

The team proposes that next-generation embodied benchmarks derive tasks from real-world needs, give no in-domain demos, and keep the world clock running while the agent thinks. For another long-running agent test, see our post on DeepMind's Minecraft run with Gemini 4 Argon agents. For engineers, the practical takeaway is in the harness section: review action chunks rather than single steps, and expose end-effector commands rather than joint angles.

Sources

OpenAIRoboticsEmbodied AIAgents