Subscribe
Google
5 min read

Gemini Decisions API: What People Are Saying

Google's Gemini Decision 1 scores up to 1,000 questions in one pass. HN, X and Simon Willison react: a Jev copy, early tests, price gripes, trust doubts.

Olivetti-style cover reading Gemini Decisions API, What People Are Saying, with panels for 342 HN points, 105 HN comments and 1,000 questions per call
Olivetti-style cover reading Gemini Decisions API, What People Are Saying, with panels for 342 HN points, 105 HN comments and 1,000 questions per call

Google released the Gemini Decisions API on Wednesday, Nov 11. It runs on a new model, Gemini Decision 1, which does not write text. You send one input and up to 1,000 yes/no, multiple-choice or rubric questions, and it returns a probability and a confidence score for each one. The price is $0.10 per million input tokens, and output is free. Developers saw it at once as Google's answer to Jev, the decision model from TypeSafe AI. The Hacker News thread had 342 points and 105 comments by 9:30 PM PT on Nov 11. Here is what people are saying.

What Google launched

@sundarpichai announced it on X at 10:15 AM PT: "Instead of generating text token by token, it's built for closed-set choices scoring up to 1,000 questions across text, images, audio, video + PDFs in a single pass." @OfficialLoganK called it "our new frontier decision model in the Gemini API" and noted "You can try it for free in the API". His post had about 8,900 likes by 9:30 PM PT, the most of any post we found on the launch. Sundar Pichai's post had about 6,000.

The API is in Google AI Studio now. Support in Google Cloud is "coming soon," per Google. Our news report on the Gemini Decisions API has the full specs, rate limits and pricing.

Where the reactions landed: a Jev copy, early test results, price gripes and the Google graveyard worry

"Practically the same as Jev"

The first reaction on Hacker News came from iamdelirium: "Looks like Google announced their own Jev inspired API and model". Later in the thread, muzani put it in fewer words: "It's practically the same as Jev, but by Google".

Simon Willison made the same point on his blog. He wrote that the new API "is almost identical to Jev: it supports the same three types of questions (yes/no, choice, and rubric-scoring) and even reuses Jev's term "Noul" for a yes/no question."

The odd name got its own sub-thread. claiir asked: "Why's the probability type called "noul"?" ssivark answered that it is short for a Bernoulli random variable. iamdelirium said Google modeled it "on Jev's API instead of OpenAI's version."

On X, @kimmonismus posted head-to-head numbers: "In Google's own tests, it beats Jev 1.14 on 5 of 7 shared benchmarks (92.2% vs. 90.0% on JevBench) and is better calibrated." He added that it "costs $0.10 per 1M input tokens vs. $0.042 for Jev." That post had about 520 likes by 9:30 PM PT.

The tests people ran

Artificial Analysis. The benchmark group @ArtificialAnlys said Gemini Decision 1 scores 66 on its Decision Index v2.1 and leads decision models "in knowledge and reasoning benchmarks". The same post put it 7 points behind the leaders, Drex Max v1.1 and GLiDE v1.2, which both score 73. Two follow-up posts were less flattering. One said it costs $345 per 1M tasks across the index, against $103 for Drex Max v1.1. The other put its time per task at 211ms, faster than Drex Max v1.1 at 283ms but slower than Jev 1.14 at 92ms.

Simon Willison's video test. Willison passed four yes/no questions about his own video of a stomping robot in Yonkers: is there a dragon, is there a robot, is it set in France, is it set in the USA. The probabilities pointed the right way on all four, with about 0.92 for robot and USA and about 0.05 and 0.08 for dragon and France. He noted that "Jev and OpenAI both return 1.0 or 0.0 for yes/no questions, but Gemini insists on being a little less confident than that".

A Hacker News eval. Topfi wrote that their own quick evals showed "in many usage scenarios a 4x lower brier over Jev (2x - 6x depending on task)". A lower Brier score means better-calibrated probabilities. Topfi also saw p50 latency more than 100ms lower, "though all this cost twice as much across the entire eval end-to-end." The verdict: "Dare I say, this is a very impressive release by the Gemini team."

Cost in practice. LaurensBER, who has used decision models in internal workflows, wrote: "I was also shocked to see that after a week of heavily experimenting I did not even burn through 5 USD". That comment was about decision APIs in general, not only Google's.

Not everyone trusts the charts. When one commenter said Google's benchmarks show a better model than OpenAI's and Jev, apetresc replied: "Looking at anyone's benchmarks, they're all better than each other." wasabi991011 was "Surprised how close all these models are on the benchmarks".

Price and speed complaints

Several people said Google is charging more for the same idea. waximabbax said it is a "bit more expensive than other other Decision Models in the same lane" but said audio and video input "can be extremely useful". flockonus was blunter: "At 0.10/M this feels very overpriced vs Liquid's 0.02/M, which on my own tests (text input) is very capable."

holografix saw a familiar playbook: "a solid but not great "we do it too" product, priced to capitalise on risk and change aversion." SchemaLoad agreed that this works on big companies: "They don't need to be the best, they just need to not force a customer to go elsewhere."

Speed matters too. mikeocool recalled that Jev was faster than the first wave of rivals: "Getting a response in 250ms opens up a lot possibilities over 750ms."

Will Google keep it?

The "killed by Google" worry came up early. florkbork said Google was winding down its Jules coding agent that same day, and wrote: "It seems unlikely this will be around in 6 months at this rate."

Others pushed back. rgreasons wrote that the API "is likely to be used by enterprise customers in real products." antonvs said Google kills consumer products, but "With enterprise products, they have actual paying customers, so the situation is different."

Two practical complaints followed. vesterde asked when it would reach Vertex, Google Cloud's AI platform: "As far as I can tell it's just in AI Studio for now." BoorishBears posted a Google error message that blocked their input for "sensitive words" and wrote: "It's not the spirit of a competitive company."

What are decision models even for?

Not everyone is sold on the category, which already includes open models such as Baseten's joey-1. perching_aix wrote: "I still have no idea why this is useful. Models are atrocious without reasoning, for classification and otherwise". foopod replied that the value is "you don't need to train a classifier."

When kodoman asked how people use these models today, the thread filled with examples:

  • LaurensBER: they "work great as gates ("does this require a human review?") or filters ("is this suspicious?")".
  • te_chris built their own version of the /goal command from Claude and Codex for the Pi harness. Jev is called each turn to check if the agent is done.
  • richardw used one "to classify 10k ish repos" for hnrepos.com.
  • skapadia uses one for task classification, guardrail checks and search ranking.
  • bushbaba listed tool routing, such as "which MCP or Skill should i use for a given question".

sky2224, who works in manufacturing, gave a fair middle view: "I think right now decision models are in a "solution looking for a problem stage", but I think there's opportunity still."

What it means for Jev

pratzc07 asked the obvious question: "So is it over for Jev now that the big boys are in the game". brandall10 noted that TypeSafe raised "almost a billion in financing" after OpenAI's Decisions API came out. freeone3000 suggested: "Take the money and go home."

The mood across both sites: Google's version looks strong on calibration and multimodal input, it costs more than Jev, and developers want it in Google Cloud before they trust it in production.

Sources

GoogleGeminiReactionsAPIs