When you pay an API provider for a named language model, you get back text and a bill. You do not see which model wrote the text. A paper posted to arXiv on October 27, 2026, credited to TokenRhythm Technologies, describes how that gap can be abused: a provider could "cut costs by quietly routing a fraction of user requests to a cheaper substitute model while billing for the advertised one." The paper's answer is API-TAIL, an audit that tries to catch this from the returned text alone.
The paper does not report that any provider it tested was doing this. Its substitution tests are simulated, as explained below. What it offers is a method and a set of measurements of how well that method works.
Why this is hard to check
The paper names the obstacle directly: "honest providers do not serve a model in one canonical way." They pick their own temperature (how random the sampling is), cutoffs, repetition and presence penalties, and default system prompts, "usually without documenting these choices." Each of those changes the text a model returns.
So an auditor cannot compare an endpoint against one reference version of the model. The paper calls the set of honest setups a "composite null," and says audits built on a single reference face a dilemma: "They either raise false alarms on honest providers or, once calibrated to accept all legitimate deployments, lose their power."
How API-TAIL works
The method needs the claimed model's weights, so the auditor can run that model locally and score any text the API returns (Section 2). It does not use log-probabilities, seeds or other extras from the endpoint. The paper describes three steps.
- Score. Each returned response gets a typicality score: how much more surprising it is to the claimed model than a typical sample, measured in standard deviations.
- Align. The audit picks one temperature, from a grid of 0.30 to 1.80, at which the batch's scores are centered. The paper's idea is that honest serving choices "shift the whole batch together," while a substitute model "writes individual responses that stay atypical for the claimed model across temperatures."
- Count. It counts responses whose score stays above 3. The endpoint is flagged if that count passes a threshold set by running the same audit on the auditor's own outputs from the claimed model under 16 to 19 honest configurations per test panel, including penalties and a default system prompt.
If you have read about how models count text in tokens, the scoring step works at that level: the claimed model rates every token of the response, over its full vocabulary.
What the authors tested
The audit targets were Qwen3.8-27B (from the Qwen3.8 family) and Gemma-4-31B-it, both offered by many providers through OpenRouter. The authors used 26 "plain honest endpoints" (17 Qwen, 9 Gemma) and two request styles: P30, with fixed settings and answers of up to 30 tokens, and P-user, with the provider's default sampling and answers of up to 400 tokens. They collected 143,955 responses, including real API responses from six substitute models served by seven endpoints.
The honest audits used real API traffic. The substitution audits were simulated: each request was answered by a real substitute API response with probability 5%, 10% or 25%, and otherwise by a response of the claimed model generated locally.
The results, in the paper's numbers
At 200 requests and 10% substitution by each panel's main substitute (Table 1):
- API-TAIL's detection power, the chance of flagging a substituting endpoint, was 0.33 (Qwen P30), 0.69 (Qwen P-user), 0.89 (Gemma P30) and 1.00 (Gemma P-user).
- Its largest false-alarm rate on the plain honest endpoints was 0.035, 0.002, 0.003 and 0.069 on those four panels, under the paper's 0.10 limit.
- Two published audits, a rank-based test and a maximum-mean-discrepancy test, had power 0.00 on the P30 panels under the same calibration. Likelihood-count tests reached at most 0.13.
Across all 16 substitute and panel settings at 10% and 200 requests, power was at least 0.69 in 13, with a median of 0.99 (Table 2). Models from another family were detected even at 5% substitution: DeepSeek-V4.1-Flash with power 0.56 to 0.96, and Qwen3.8-27B (as a substitute for Gemma) with 0.54 and 0.96. Doubling the budget to 400 requests raised power on the main Qwen substitute from 0.33 to 0.62 (P30) and from 0.69 to 0.93 (P-user).
The authors put the cost of one 200-request audit at "less than 0.16 USD of API calls" plus two to eight GPU-minutes of local scoring on a single L20X GPU. The appendix gives 0.007 to 0.155 USD in API charges per audit. That is the cost of the requests the authors sent; the paper does not list the per-token prices behind it.
Limits the paper reports
- The closest substitute can slip through. The paper calls the previous release, Qwen3.6-27B, the closest substitute. Served from the developer's own platform, it "stays undetected on long default-sampling responses," even at 25% substitution, with power 0.07 at 200 requests (Table 2).
- Honest setups outside the pool get flagged. An endpoint that documents injecting its own system prompt was flagged in 79% (P30) and 99% (P-user) of audits. The paper says FP4-quantized endpoints "pass on short responses." On long responses, Table 5 shows three of the four Gemma FP4 endpoints with false-alarm rates of 0.135 to 0.360, above the 0.10 limit (the fourth was at 0.023), and a fifth that also injects a system prompt at 0.525; the three Qwen FP4 endpoints stayed at 0.000. Our quantization explainer covers what lower-precision weights are.
- The pool matters. Without the five penalty configurations, the false-alarm rate on the model developer's own Qwen endpoint rose to 0.269 on the short-response (P30) panel.
- Scope. The tests cover two claimed models, both run locally by the authors, and simulated substitution mixes.
Why it matters
The paper frames partial substitution as hard to see because it "is invisible in any single response." Its takeaway for anyone who buys model access by name is narrow but concrete. In its tests, a few hundred ordinary requests and a few GPU-minutes were enough to catch most 10% swaps to a different model family or a smaller sibling, while false alarms on the plain honest endpoints stayed under the paper's 0.10 limit. The same tests show the method can miss the previous release of the same model and can flag honest providers that add a system prompt or serve FP4 weights on long answers. The authors list code at the project page.
