Subscribe
AI
5 min read

What Dedicated LLM Inference Actually Costs Per Hour in Late 2026

Per-token API pricing gets constant scrutiny. The other market, where you rent a reserved slice of a serving platform to run a model the shared catalog does not carry, is less examined. It is also where the same nominal GPU rents for anywhere between $2.20 and $7.00 per hour, depending on who is metering it and what the meter counts. This is the late-2026 price list for dedicated LLM inference on the six managed platforms that publish rates: DeepInfra, Modal, Replicate, Together AI, Baseten, an

What Dedicated LLM Inference Actually Costs Per Hour in Late 2026

Per-token API pricing gets constant scrutiny. The other market, where you rent a reserved slice of a serving platform to run a model the shared catalog does not carry, is less examined. It is also where the same nominal GPU rents for anywhere between $2.20 and $7.00 per hour, depending on who is metering it and what the meter counts.

This is the late-2026 price list for dedicated LLM inference on the six managed platforms that publish rates: DeepInfra, Modal, Replicate, Together AI, Baseten, and Fireworks. The rates are list prices as published on their pricing pages, retrieved 21 November 2026, and they change often.

Six metering instruments in a row

Three meters, not one price

Platforms in this market bill on one of three units:

  • Per token (serverless). Shared infrastructure, pay only for what you generate. This is the same model as a frontier API, applied to open-weight models.
  • Per GPU-second. You deploy a model to dedicated capacity and the meter runs only while your compute is active. Modal, Replicate, and Fireworks bill this way.
  • Per GPU-hour, billed by the minute. A reserved replica bills for as long as it runs, busy or idle. Baseten and Together AI's dedicated endpoints bill this way.

The unit matters more than the hourly number, because idle time is handled differently in each scheme. An endpoint that bills "per hour" but can scale to zero is a different product from one whose replicas hold hardware until you stop them.

What an H100 costs, provider by provider

Using an Nvidia H100 80GB as the reference unit:

  • DeepInfra: $2.20 per GPU-hour, the cheapest published rate of the group. It also lists A100 80GB at $0.89, H200 at $2.69, and B200 at $3.69. Custom model deployments are billed in minute granularity.
  • Modal: $0.001097 per GPU-second, about $3.95 per hour, plus separately metered CPU and memory. H200 is about $4.54 per hour and B200 about $6.25. Modal bills by the second and does not charge for idle resources.
  • Together AI: two products, two rates. GPU Clusters (bare reserved capacity) list H100 at $3.99 per hour on-demand, $1.99 preemptible, and $3.19 at 91-to-180-day reserved terms. Dedicated Endpoints (managed single-model serving) list H100 at $5.49 per hour and B200 at $8.99.
  • Replicate: $0.001525 per GPU-second, about $5.49 per hour for H100; A100 80GB about $5.04.
  • Baseten: $0.10833 per minute, about $6.50 per hour for a dedicated H100. An H100 MIG slice (40GB) runs $3.75 per hour and an A100 $4.00.
  • Fireworks: on-demand deployments start at $7.00 per GPU-hour for H100 and H200, billed per second with autoscaling to zero.

That is a 3.2-fold spread, DeepInfra to Fireworks, for the same chip class. Run an always-on H100 replica for a 730-hour month and the list-price bill is roughly $1,606 at DeepInfra, $2,883 at Modal, $4,745 at Baseten, and $5,110 at Fireworks.

The fine print is the product

The hourly rate is the starting point. The billing semantics decide the invoice.

Idle time. Together's dedicated endpoint docs are explicit: a deployment bills for as long as it runs, per replica, regardless of tokens pushed through it. An inactivity timeout can stop a deployment after a configured number of quiet minutes and release the hardware, but idle shutdown is off unless you enable it, and minReplicas sets a continuous billing floor. Modal's page promises you "never pay for idle resources" by metering per second. Fireworks likewise bills per GPU-second with no extra charges for start-up times. Baseten does not charge replicas that have scaled to zero, but you are billed for minutes a replica spends deploying or scaling, not only serving. Replicate is the outlier with an explicit split: private models on dedicated hardware bill for setup time, idle time, and active time, while models labeled "fast booting fine-tunes" bill only the active window.

Discount tiers. Fireworks offers preemptible deployments at 50% off the on-demand rate, on GPUs borrowed from idle reserved capacity that can be reclaimed at any time; region-restricted deployments carry a 1.5x premium. Together's cluster ladder drops H100 from $3.99 to $1.99 by accepting reclaimable preemptible placement, or to $3.19 by signing a 91-to-180-day reservation. Baseten's MIG slice is the same discount mechanism expressed as hardware partitioning rather than contract terms.

What the credits cover. Modal's Starter plan includes $30 per month of compute, but since 1 September 2026 token usage on shared endpoints is excluded from that allotment and bills from the first request. The per-second meters are covered; the per-token one is not.

When dedicated beats per-token

Every provider in this market also sells serverless per-token access to popular open models, and their own documentation concedes the crossover. Together's pricing FAQ says dedicated inference suits sustained production traffic, while per-token billing "loses nothing during an idle window." The arithmetic is a utilization ratio: take the cheaper per-second rate and divide by the always-on rate for the same chip. Modal at $3.95 versus a Together Dedicated Endpoint at $5.49 gives 3.95/5.49 = 72%. Below roughly 72% average daily utilization the per-second meter wins; above it, the flat reserved rate does.

The case for leaving the per-token catalog is rarely pure price. Dedicated deployment is where you go when the model is not shared, because it is a fine-tuned checkpoint or a custom architecture; when you need latency isolation from other tenants; when a platform's serverless rate limits bind before your capacity does; or when you need a region restriction, which Fireworks sells as a paid product setting rather than an entitlement. For fine-tuned models specifically, the platforms split: Fireworks serves LoRA-tuned models at base-model serverless token prices on shared infrastructure, while most other cases route you to the per-hour meter anyway.

A short checklist

Before pointing a budget at any of these rates:

  1. Ask what the meter counts when traffic is zero. The answer ranges from nothing (Modal, Fireworks, scaled-to-zero setups) to the full hourly rate (Together endpoints with default settings).
  2. Ask what minReplicas or equivalent floors are set to. A two-replica redundancy setup at $6.50 per hour is a $9,490 monthly floor at list price before traffic.
  3. Ask whether the discount tier fits the workload's tolerance for interruption. Preemptible capacity at half price is worthless for a synchronous product request path.
  4. Divide your daily token volume by 86,400 seconds and compare per-token spend against one always-on replica's effective throughput. If the replica would sit idle most of the day, the serverless catalog is the cheaper contract.

The headline number is easy. The meter semantics are where dedicated inference bills actually live, and no two of the six platforms above agree on what an idle hour costs.

Sources

AILLMInferencePricing