Subscribe
OpenAI
4 min read

OpenAI BioBasicsBench Tests AI Agents on Wet-Lab Data

OpenAI's BioBasicsBench tests AI agents on 100 routine wet-lab tasks. GPT-6 Astra earned 66.7% of credit at xhigh, versus 43.8% for GPT-5.6 Sol.

Olivetti-style cover reading BIOBASICS BENCH, WET-LAB TASKS and OpenAI tests AI agents on raw lab data, with GEL, PLATE and MICROSCOPE blocks in mustard, cream and red on teal.
Olivetti-style cover reading BIOBASICS BENCH, WET-LAB TASKS and OpenAI tests AI agents on raw lab data, with GEL, PLATE and MICROSCOPE blocks in mustard, cream and red on teal.

OpenAI has published BioBasicsBench, a benchmark that checks how well AI agents read the messy, raw data that scientists produce in a wet lab every day. In OpenAI's announcement, dated November 6, the top scorer, GPT-6 Astra, earned an average of 66.7% of the available credit at the 'xhigh' reasoning setting. GPT-5.6 Sol scored 43.8%.

The test targets routine lab analysis, not polished figures: reading a gel, checking a control plate, counting colonies, and knowing when the evidence does not support a conclusion.

Bar chart of BioBasicsBench mean reward at xhigh effort: GPT-6 Astra 66.7%, GPT-6.1 Sol 62.0%, GPT-5.6 Sol 43.8%, GPT-6 Luna 38.3%, GPT-5.5 35.4%, GPT-5.4 33.2%, GPT-5.2 31.2%

What BioBasicsBench is

OpenAI built the benchmark with Ligo, which it describes as "a biotech company focused on enzyme design." It has 100 tasks across five areas of routine wet-lab work:

  • protein methods
  • imaging and cytometry
  • quantification and assays
  • nucleic acid methods
  • molecular cloning

OpenAI says tasks range "from simple requests, such as reading the molecular weight of a DNA sample in a gel, to detailed assay analysis," but all involve work that wet-lab scientists "should be able to complete reliably."

The company's argument is that existing benchmarks are built around charts and images selected for scientific papers. Everyday wet-lab work involves failed experiments, ambiguous images and conflicting observations, and OpenAI says those benchmarks "may miss this ambiguity and the troubleshooting it requires."

Raw evidence, not polished figures

The inputs are the kind of files a scientist actually has on the bench. OpenAI lists gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work. By OpenAI's count, 31 tasks include gels and blots, 14 include microscopy, 12 include sequencing files and 7 include protocol videos. One task can include several kinds of data, so these counts overlap.

Scientists wrote the tasks from data from their own experiments, each paired with an expert-written grading rubric. Each task then went through three reviews: an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review.

Scoring gives partial credit. Each answer is checked against individual rubric criteria, so the benchmark can tell a correct conclusion apart from a partly correct analysis. Depending on the task, criteria cover the conclusion, numerical results, supporting reasoning, or "recognition that the evidence is insufficient."

The results

OpenAI tested seven models: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra. It ran 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 results.

At xhigh reasoning effort, which OpenAI says is the highest setting reported for all seven models, mean reward was:

  • GPT-6 Astra: 66.7%
  • GPT-6.1 Sol: 62.0%
  • GPT-5.6 Sol: 43.8%
  • GPT-6 Luna: 38.3%
  • GPT-5.5: 35.4%
  • GPT-5.4: 33.2%
  • GPT-5.2: 31.2%

OpenAI also reports output tokens per task. GPT-6 Astra used 4,704 tokens per task at xhigh, while GPT-5.2 used 27,620. OpenAI says Astra "substantially improves on GPT-5.6 Sol, even at low reasoning effort," and that the gains "suggest that stronger native multimodal capabilities may be key" for this kind of work. If you need a refresher on reading scores like these, see our guide to how to read AI benchmark scores.

What GPT-6 Astra got right and wrong on BioBasicsBench, per OpenAI's examples

Where Astra did better

OpenAI gives three examples of improvement over GPT-5.6 Sol.

  • Knowing the limits of the data. GPT-5.6 Sol called cells dead based on a dye signal, but the detector could also pick up light from a second dye. Astra recognized the ambiguity and called for better controls.
  • Reading fine detail in photos. In a cloning task, GPT-5.6 Sol relied on a negative control plate with no colonies. Astra noticed the plate was labeled "LB Kan50," for kanamycin, a different antibiotic from the ampicillin used in the experiment, and rejected it as a reliable control.
  • Keeping context in mind. Shown cells moving in and out of focus, GPT-5.6 Sol looked for a biological explanation. Astra correctly recognized a z-stack, a series of images taken at different depths.

Where models still fall short

OpenAI is direct that the scores are not a green light. "Higher benchmark scores do not yet mean models can make reliable judgments in the lab," the post says. It shows three Astra failures:

  • An assumption over the controls. In a colony PCR task, Astra assumed a larger PCR product meant successful cloning and picked lanes 4, 6, 7 and 8. The reference answer, based on the controls, picks lanes 5, 9 and 10.
  • Discounting contradicting data. Astra concluded vacuoles are generally darker than the rest of the cell, even though some looked lighter in the same image. Both appearances occur because different focal depths can reverse the contrast.
  • Miscounting colonies. Astra failed to separate close neighbors, missed visible colonies and marked empty agar, for a net overcount of about 12% against the benchmark reference on one plate.

Why it matters

OpenAI frames the stakes simply: misreading an image or overlooking a problem with the controls "can lead scientists to draw the wrong conclusion and spend time on experiments that won't help answer their question." BioBasicsBench measures that everyday part of the job rather than headline discoveries. OpenAI says progress here "could help scientists spend less time troubleshooting," and that tracking these skills will also help it understand "their implications for biosafety and biosecurity."

The results also show the limits plainly. Even the best model earned about two thirds of the available credit on tasks OpenAI calls routine. For comparison, see how Anthropic described Claude helping on a phage vault structure project, and our GPT-6 prompting guide for working with GPT-6 Astra and its sibling models.

Sources

OpenAIEvaluationAgentsNews