Google has published a framework for judging whether AI tools are safe and useful for health questions. A Google blog post by Dr Sarah Montgomery-Taylor, Clinical Evaluation Lead at Google for Health, announced it on November 4, 2026. The matching paper went onto arXiv on November 3.
The framework is a method, not a product. It tells a team what to test, which test to use at which stage, and how to read the results. Google says it built the framework and continues to use it to assess its own models in health contexts.
What Google announced
In the blog post, Google says it worked with health experts Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran on a whitepaper. The whitepaper describes an end-to-end evaluation approach for AI use cases that address people's health needs.
The post names three areas. The framework evaluates models across accuracy, safety and usefulness. It combines closed-question benchmarks, autoraters and expert human review to assess common and high-risk health scenarios.
Google says it hopes that sharing its methodology promotes transparency in how it approaches this area. It also says it wants to support the health ecosystem, from developers to researchers to clinicians, in building AI tools people can trust.
The paper is titled "Towards Multi-Method Evaluation of AI in Health: Integrating Benchmarks, Autoraters, and Expert Evaluation" (arXiv:2611.05127). It lists 32 authors. All but two are at Google LLC. Safran and McGlynn list outside affiliations, and the paper says they have no conflicts to disclose. The Google-affiliated authors declare that they are Google employees who may hold Alphabet stock.
The three areas: accuracy, safety, usefulness
The paper treats these as three separate questions, because a response can pass one and fail another.
- Accuracy covers factuality and clinical validity.
- Safety covers the risk from actions people take because of a model's output.
- Usefulness covers health literacy, comprehension and real-world actionability.
The authors give an example of why they split them. A response can be factually accurate yet unsafe, such as providing precise details that could facilitate self-harm. A response can also be accurate and safe but of little use if it is too complex or non-actionable for a patient or clinician to apply.
The paper also points to a gap in published work. It cites a systematic review of over 760 large language model evaluation papers and says that literature over-indexes on accuracy while under-measuring safety and real-world utility.
Three methods, each with a known weakness
The framework does not rely on one test. It uses three methods, and the paper states the limits of each one.
Method 1: closed-question benchmarks
These are tests with pre-defined answers, such as multiple-choice questions. The paper names MedQA, PubMedQA, AfriMed-QA and professional licensing exams as examples. They allow cheap, high-volume testing of core medical knowledge.
The paper says these tests lack real-world complexity. It also warns about data contamination, where test questions end up in training data. Its conclusion is that static benchmarks are a necessary starting point but remain insufficient for validating real-world performance. For a plain-English guide to reading such scores, see our explainer on what AI benchmarks are and how to read model scores.
Method 2: autoraters
An autorater is an LLM-as-a-judge: a separate AI system that grades free-text answers against a written rubric. The paper says it lets teams test many more scenarios than expert raters can review. For example, simulated users can start by withholding clinical details, to see whether the model asks the right clarifying questions before it gives guidance.
The paper lists weaknesses too. Autoraters carry the biases of the evaluator model, including verbosity and self-preference bias. The authors say expert-created ground truth is needed to check an autorater against human clinical judgment. They also say a judge from a distinct model family is preferable.
Method 3: expert human review
Clinicians and other domain experts rate responses using standardized guidance. The paper calls this the highest-quality validation. It adds red-teaming, where testers try to push past safety limits. One example in the paper is a product that should not give regulated medical advice. Testers might use emotional pleas, frame the request as a story, or claim to be a health professional.
Human review has its own problem. The paper says that in a given rating task it is common for two physicians to diverge in ratings by 30-40% or more. The authors note that this inconsistency predates AI.
Deciding what to test: Critical User Journeys
A health model can face a huge range of questions. The paper uses Critical User Journeys to choose what to test. A journey names the user, their goals and the steps they take. The paper's three examples show how the main risk changes.
- Checking symptoms (consumer). The primary focus is safety. A model that fails to flag red-flag symptoms may leave a person unaware that they need urgent care. An overly alarmist answer may cause unnecessary anxiety.
- Understanding a condition (consumer). The primary focus is usefulness. Information that was not written with consumer needs in mind may be ignored.
- Summarizing clinical notes (professional). The primary focus is accuracy. The paper gives an example of a clinician drafting a discharge summary, where hallucinations or omissions could mislead later decisions.
The paper says all three areas must be checked for every application. Each journey only changes which risk gets the most attention.
Three stages of an AI product's life
The framework maps the methods onto three phases: base model development, product development and post-deployment monitoring. The authors say the phases are a strategic map, not a rigid sequence.
For the first phase, the paper points to the Gemini 4 safety report as an operational example of a base-model evaluation cycle. For the last phase, it argues that pre-release testing is not enough. It calls for ongoing monitoring after launch to catch behavioral drift, and compares this to routine quality control for clinical equipment. It also says data silos may limit monitoring across organizations.
The authors also say clinical and development teams should co-design passing thresholds for each phase. They advise against universal or arbitrary numerical cutoffs, and say the thresholds should fit the clinical risk of the specific application.
Why it matters
The blog post opens by noting that many people have turned to AI with a health question or received care from a doctor who is using it. The paper adds that general-purpose models are being adopted quickly in healthcare and are hard to assess with traditional evaluation methods, which suit single tasks better than hundreds or thousands of them. It says its purpose is to support decisions on development, procurement and deployment by product teams, health systems and regulators.
The framework comes from Google and describes how Google tests its own models. The blog post and the paper abstract do not report test scores for any model, so a reader cannot use them to compare Google's health answers with another company's. What the paper gives is a set of questions. A buyer or regulator could ask a vendor which user journeys it tested, which methods it used, and how it checked its autoraters against human experts. That reading is our own, not a claim made in the paper.
The authors are also open about what is unfinished. They say deeper assessment of individual actionability remains an area for future work. They also say the healthcare ecosystem must still define shared responsibilities between technology developers and deploying health systems, and set up mechanisms for independent validation.
Related coverage on llms.blog
Google has been active in health AI this autumn. We covered its public health work with Singapore and its AI certificate for healthcare professionals. This framework addresses a different question: how any of these tools should be judged.
This post is general information about a published framework. It is not medical advice.
