Paying for a reply now or a batch result tomorrow

A coding agent is paid for with API tokens, a subscription, self-hosted hardware, or a mix of the three.

Each one needs its own accounting. Batch rates and prompt caching lower the token bill, and idle time raises the fixed cost of each active hour on hardware we own. The worked examples in the scenario table use published Claude Sonnet 4.6 rates and stated quantities, so the reader can reproduce the arithmetic without running a model.

An agent in your editor waits on every model reply before it takes its next step. A nightly run of 100 pull-request reviews can wait hours for all of its results. A lower batch rate suits the nightly run, and the same wait would stall the agent in the editor.

An interactive API request returns its response to the caller, who uses it before the next dependent step. A provider Batch API, such as the OpenAI Batch API or the Anthropic Message Batches API, accepts a job of independent requests, reports its state, and returns the results later. Continuous batching, which the vLLM README lists as a runtime feature, is a different thing: the server schedules arriving requests together, and the caller submits no batch job.

API work is metered per token. Anyone who can send requests on our key spends that money, and The gateway as an attack surface covers the budgets and rate limits that cap it. Non-urgent jobs can go in batches, and subscriptions sell access under stated limits. Self-hosting adds equipment and operating costs even when the equipment sits idle, and judging its useful output means measuring it on the hardware. So we account for subscriptions, API credits, and self-hosting separately, in five steps, and close with how to set up a cost-per-accepted-task comparison:

  1. Count the token classes an API bill charges for.
  2. Decide when a batch rate fits the work.
  3. Price a cache write separately from its later reads.
  4. Check the units a subscription uses.
  5. Work out how idle capacity raises the fixed cost of each active hour on owned hardware.
Three ways to ask. An interactive caller and a provider API are joined by two lanes: 1 send and wait, 2 the reply comes back on the same request. A batch submitter, the provider's batch queue and a mark named Results, later are joined by 3 hand in a job and 4 come back for the results. Live callers, the server's scheduler and a model server are joined by 5 live requests, grouped by the server, and 6 live replies. At the end a reply card rests at the interactive caller, a token tagged Results at the batch submitter, and two live request cards at the model server.
Provider batches require their own caller workflow, independent of the server's internal processing.
Figure text

Three lanes compare a caller waiting on a synchronous API, a submitter retrieving a provider batch later, and live requests passing through a serving scheduler. The provider owns the queue and the scheduler.

  • Wait now, collect later, or neither
  • Lit lamp: the caller is waiting
  • Unlit lamp: caller is not waiting
  • A provider batch is the caller's own workflow. Grouping inside a server is not a batch you submit.
  • 1 Send and wait
  • 2 The reply comes back on the same request
  • 3 Hand in a job
  • 4 Come back for the results
  • 5 Live requests, grouped by the server
  • 6 Live replies
  • Reply
  • Live request
  • Interactive caller
  • Provider API
  • Batch submitter
  • Provider's batch queue
  • Results, later
  • Live callers
  • Server's scheduler
  • Model server
  • Results
  • Different again: a server can group live requests on its own. The callers hand in no job.

Sources: [1] Business Pricing | OpenAI; [2] Batch API | OpenAI API; [3] Batch processing - Claude Platform Docs; [4] vLLM README at pinned revision.

Count the billed token classes

An API bill is counted in tokens, so the first thing to know is what a token is and which token classes the provider charges for.

A model's tokenizer defines each token. Word counts per token vary, and equal token counts can represent work of different quality. A provider's price schedule can bill these classes separately:

Reasoning tokens are billed as output tokens by OpenAI and Anthropic. They count as their own class only when a price sheet lists them separately, so they are never counted twice. models.dev, an open-source database of AI models with their specs, prices and capabilities, has cost-schema fields for several of these classes. OpenAI's pricing and Anthropic's pricing set the actual charges.

For rates quoted in USD per million tokens, the general calculation is:

API cost in USD = sum(tokens in class × USD per million tokens for class / 1,000,000) + applicable non-token charges

A non-token charge is a fee that is not counted in tokens. Web search is one: Anthropic lists $10 per 1,000 searches and OpenAI lists $10 per 1,000 calls, each plus the tokens of the search content. We count each billable retry once, with its token and non-token charges. The rate is set by:

When batch rates fit the work

Batch rates can lower per-token charges when the work can wait for results.

The OpenAI pricing page offers 50% off input and output rates for Batch API work processed asynchronously over a 24-hour window. Eligibility for the batch discount depends on the model and endpoint, so we check the ones we plan to use. The Batch API guide also defines batch-specific rate limits and expiration behavior. An expired batch can contain completed requests whose tokens are still charged.

Each result has to find its original task. Because OpenAI's batch output can arrive in a different order from its input, line position is unreliable, and we match each result by its request's custom_id. A workflow that treats the second output as the second input can attach an answer to the wrong task even though both requests succeeded. See the batch result retrieval docs for the ordering rule.

Anthropic's Message Batches guide likewise describes independent asynchronous requests and a 24-hour expiry. Its pricing is 50% of standard API prices. Expired or failed items need handling, because some requests may remain incomplete after the 24-hour window.

An agent loop is a poor fit for batch results, because each step depends on the previous reply. Batch fits only the independent work, such as the nightly review run.

Price a cache write separately from its later reads

The second way to lower token charges is prompt caching, which prices repeated prompt content differently from new content.

Provider prompt caching reuses eligible prompt content under the provider's cache rules. For Claude Sonnet 4.6, the pricing table lists standard input at $3, output at $15, cache writes at $3.75, and cache reads at $0.30 per million tokens. The write price is for the default cache, which lives five minutes and is refreshed at no cost each time it is used.

Each scenario prices 100 hypothetical requests, each with 10,000 input tokens and 2,000 output tokens. The cache scenario splits each input into an 8,000-token reusable prefix and 2,000 unique tokens. We assume one prefix write followed by 99 cache reads, each within five minutes of the previous use, so the cache never expires. A gap of more than five minutes between requests would expire the cache and force another write, so the $3.8676 total depends on that assumption.

Token-charge arithmetic for the three scenarios.

Each charge = tokens × USD per million tokens / 1,000,000. We use rates from Pricing - Claude Platform Docs. The cache partition is 200,000 unique input + 8,000 prefix-write + 792,000 read tokens = 1,000,000 input tokens.

The rates are Claude Sonnet 4.6 prices in USD per million tokens.

Sources: Pricing - Claude Platform Docs.

ScenarioToken classQuantity in tokensUSD per million tokensCalculated USD
Standard, no cacheInput Token class: input1,000,00033.00
Standard, no cacheOutput Token class: output200,000153.00
Batch, no cacheInput Token class: input1,000,0001.51.50
Batch, no cacheOutput Token class: output200,0007.51.50
One write, 99 hitsUnique input Token class: unique input200,00030.60
One write, 99 hitsFive-minute cache write Token class: five-minute cache write8,0003.750.03
One write, 99 hitsCache reads Token class: cache reads792,0000.30.2376
One write, 99 hitsOutput Token class: output200,000153.00
Standard, no cacheTOTALNot applicableNot applicable6.00
Batch, no cacheTOTALNot applicableNot applicable3.00
One write, 99 hitsTOTALNot applicableNot applicable3.8676

The scenario totals are $6.00, $3.00, and $3.8676 respectively.

For each row, we multiply tokens by the rate and divide by 1,000,000 (tokens × rate / 1,000,000). The totals cover token charges only, excluding request and tool charges, retries, and any regional, priority, or long-context surcharge.

Anthropic's pricing modifiers allow cache and batch discounts to stack, but we use standard, non-batch rates for the cache scenario. Batch cache hits are also best effort because requests run asynchronously and concurrently. For a real estimate, we replace the assumed cache counts with the billing-class counts from our own usage.

A console named The same 100 requests stands beside one tall meter named Token charges for 100 requests, whose top line is labelled $6.00: standard rates, no cache. At the end the meter is filled in stacked parts to $3.8676, with the output part labelled $3.00. A mark beside it is named $3.00: batch rates, half of standard. Three routes are labelled standard rates, batch rates and with caching.
  1. Standard, no cache: 100 hypothetical requests at standard rates: input $3.00 and output $3.00, a total of $6.00.
  2. Batch, no cache: batch pricing is 50% of standard prices: input $1.50 and output $1.50, a total of $3.00.
  3. One write, 99 hits: assuming one prefix write followed by 99 cache reads, at standard, non-batch rates: unique input $0.60, cache write $0.03, cache read $0.2376 and output $3.00, a total of $3.8676.
Token charges only, for the same 100 requests in each scenario. The blocks are to scale against the $6.00 standard total, so the cache write is a thin line.
A console named The same 100 requests stands beside three filled meters: Standard, no cache, $6.00; Batch, no cache, $3.00; One write, 99 reads, $3.8676. Each meter is stacked from input below and output above; the third also has thin parts for cached input. A panel named The cache assumption, with one row, No gap over five minutes, is linked to the third meter. At the end a card tagged Same work rests at the third meter.
We get token charges of $6.00 at standard rates, $3.00 at batch rates, and $3.8676 with one 8,000-token prefix write plus 99 reads before expiry.
Figure text

Three stacked bars show USD token charges of $6.00 for standard input and output, $3.00 for batch input and output, and $3.8676 for unique input, one prefix write, cache reads, and output. The rates are Claude Sonnet 4.6 prices in USD per million tokens.

  • One workload, three bills
  • Input at the input rate (lower)
  • Output (upper segment)
  • Cached input, re-read (thin parts)
  • Token charges only, for hypothetical requests at published rates. Small segments are to scale; the table has the exact numbers.
  • 1 Priced at standard rates
  • 2 The same work at batch rates
  • 3 The same work with a cached prefix
  • 4 Holds only while the cache stays alive
  • Batch, no cache: Output $1.50
  • $6.00
  • $3.00
  • $3.8676
  • The same 100 requests
  • Standard, no cache
  • One write, 99 reads
  • The cache assumption
  • Same work
  • No gap over five minutes
  • Output $3.00
  • The third bill assumes the stored prefix is reused often enough that it never expires.

Sources: [1] Pricing - Claude Platform Docs.

Check the units a subscription uses

A subscription does not bill per token; it bills for access under the plan's terms, so we check what units each plan counts in before comparing it with API charges.

Each plan sets its own units, and its included products may use different meters. The Claude Max support page specifies five-hour and weekly limits, plus an additional model-specific weekly limit. The Z.AI GLM Coding Plan page labels its allowances as weekly credits. Converting those units into a common token balance needs the provider's conversion rules.

Eligible Anthropic plans can also include a separate API-credit balance. The monthly API-credit rules cover Max, Ultra, and Team under stated eligibility and organization-linking conditions, and Free, Pro, and Enterprise are excluded from that offer. API-key users in the linked Console organization share the credits, which expire at the billing-cycle boundary and don't raise subscription usage limits. Those API credits exclude interactive Claude Code sessions.

We compare plans by their units or windows, whether usage is paid through a subscription login or an API key, and the available pricing details.

Plan names and prices change; the linked pages have the current ones.

Sources: Plans & pricing | Claude by Anthropic, What is the Max plan? | Claude Help Center, GLM Coding Plan - AI Coding Powered by GLM-5.3 & GLM-5.3-Flash & GLM-5.3-FlashX-Legal for Agents & IDEs, Monthly API credits for Max, Ultra, and Team plans | Claude Help Center.

Plan or access typeUnit or windowSubscription login or API keyPrice and other commercial detail
Claude MaxFive-hour and weekly limits; the plan states no fixed token allowance.Subscription-login use counts against the plan limits and is separate from API-key use of API credits.$100/month for web subscriptions; the mobile price may vary.
Claude UltraWeekly allowance; verified accounts have no five-hour limit, though not every account qualifiesClaude Code included; subscription-login use and API-key credit use are separate balances$200/month, tax excluded, per Plans & pricing | Claude by Anthropic
Z.AI GLM Coding PlanWeekly credits; the Lite tier lists 10,000 credits/week.The page says the plan supports 20+ agent tools, including Claude Code, and does not say how a tool signs in.The page shows multiple billing and promotional prices and gives no normalized price or token conversion.
Anthropic monthly API creditsA monthly balance; unused credits expire each billing cycle.Covers eligible API-key usage in the linked Console organization.The amount and eligibility depend on the plan. Once the credits run out, the organization's purchased-credit, reload, or invoicing arrangement applies.
OpenAI API alongside ChatGPT Plus, Business, Enterprise, or EduAPI usage is billed separately from these subscriptions.Funded through the API account.API charges follow the model's API rate schedule.

OpenAI bills API charges apart from subscription fees too. So before comparing a plan with API rates, we check which balance a client draws on: a subscription login uses the plan's limits, and an API key is billed to the API account.

How idle capacity affects fixed cost per active hour

Self-hosting replaces per-token charges with fixed equipment costs, and how much of the time the hardware is active decides what each active hour costs.

Self-hosting brings costs that keep running while the equipment sits idle. Let F be a fixed monthly expenditure in USD, H the available hours in that month, and u the fraction of those hours treated as active. For F > 0, H > 0, and 0 < u <= 1, fixed cost allocated per active hour is F / (H × u). At full utilization it is F / H. Dividing the first by the second gives the dimensionless ratio y = 1/u.

The ratio rises as the active fraction falls.

Values of y = 1/u at five utilization levels.

y = 1/u for 0 < u ≤ 1; undefined at u = 0. The ratio is normalized to full utilization.

The curve is y = 1/u. The chart clips above 10.

Active utilization fractionActive utilizationFixed cost per active hour relative to full utilization
0.1010%10
0.2525%4
0.5050%2
0.7575%1.333333… Fixed-cost allocation ratio y: 4/3 ≈ 1.3333
1.00100%1

At 25% utilization the allocation is four times the full-utilization figure, and F doesn't change. At zero utilization, cost per active hour is undefined because there are no active hours to divide by.

Before applying the ratio to a real system, we pick one definition of "active" and keep it. GPU busy, a request in flight, and a developer logged in each give a different u for the same month.

This formula calculates part of the ownership cost, including active and idle energy:

period_cost = F + H*e*(u*P_active + (1-u)*P_idle)

One accounting period and currency apply throughout:

Cost per active hour is period_cost/(H*u), in USD/active hour. Cost per accepted task is period_cost/N_accepted, in USD/accepted task, defined only when N_accepted > 0. At zero utilization, the period cost can still include fixed expenditure and idle energy. Changing utilization shifts only the mix of active and idle power, and F stays fixed. Usage-dependent maintenance and opportunity costs are not in the formula.

A meter named Share of hours in use stands empty and reads No hours in use. Beside it a rack unit named Owned hardware carries a block, the fixed monthly bill. A second meter named Fixed cost per active hour, compared with full use, reads No figure. A dashed lane leads from it to a mark named No active hours: no figure. The scale is a ratio against full use, not dollars.
Lower utilization spreads the unchanged monthly bill over fewer active hours, raising the fixed cost per active hour.
Figure text

Two meters stand for the relation y = 1/u: the share of hours in use, u, between 0 and 1, and the fixed cost per active hour, y, which is undefined at zero. The cost meter is dimensionless and normalized to the full-utilization level.

  • Idle hours raise the hourly cost
  • Meter level: a ratio, not dollars
  • No active hours: no figure
  • Lower use spreads the unchanged monthly bill over fewer active hours. The scale is a ratio against full use, not dollars.
  • 1 The share of the month's hours in use
  • 2 The same bill, divided over the hours in use
  • 3 Never in use, nothing to divide by
  • Fixed monthly bill
  • No hours in use
  • No figure
  • Share of hours in use
  • Owned hardware: Available hours in the month
  • Fixed cost per active hour: Compared with full use
  • Never in use: the bill is still owed, but there is no active hour to divide it by.

Check the cache arithmetic yourself

Listing each quantity beside its billing class makes double-counting visible, so we lay out the cache case in full.

For the cache case, the input partition is 200,000 + 8,000 + 792,000 = 1,000,000 tokens. The read quantity comes from 99 × 8,000, and the unique quantity from 100 × 2,000. Pricing all four classes gives (200,000 × 3 + 8,000 × 3.75 + 792,000 × 0.30 + 200,000 × 15) / 1,000,000 = $3.8676. Charging another million standard input tokens on top would double-count the partitioned input.

A short Python script does the same arithmetic, with the rates and quantities from the scenario table as inputs at the top:

"""Token charges for 100 requests in three scenarios, from this post's figures."""
# USD per million tokens: the Claude Sonnet 4.6 rates this post quotes, not live prices.
INPUT, OUTPUT, CACHE_WRITE, CACHE_READ = 3.00, 15.00, 3.75, 0.30
BATCH_FACTOR = 0.5  # batch pricing is 50% of standard prices
REQUESTS = 100
PREFIX, UNIQUE, OUT = 8_000, 2_000, 2_000  # tokens per request
WRITES = 1  # prefix writes; every other request reads the cached prefix


def usd(tokens, rate):
    return tokens * rate / 1_000_000


standard = usd(REQUESTS * (PREFIX + UNIQUE), INPUT) + usd(REQUESTS * OUT, OUTPUT)
print(f"standard, no cache: ${standard:.2f}")
print(f"batch, no cache:    ${standard * BATCH_FACTOR:.2f}")
rows = {
    "unique input": (REQUESTS * UNIQUE, INPUT),
    "cache write": (WRITES * PREFIX, CACHE_WRITE),
    "cache read": ((REQUESTS - WRITES) * PREFIX, CACHE_READ),
    "output": (REQUESTS * OUT, OUTPUT),
}
for name, (tokens, rate) in rows.items():
    print(f"  {name:<12} {tokens:>9,} tokens  ${usd(tokens, rate):.4f}")
total = sum(usd(tokens, rate) for tokens, rate in rows.values())
print(f"cache, prefix writes {WRITES}, reads {REQUESTS - WRITES}: ${total:.4f}")

It prints the three scenario totals and the four rows of the cache case:

$ python3 session_cost.py
standard, no cache: $6.00
batch, no cache:    $3.00
  unique input   200,000 tokens  $0.6000
  cache write      8,000 tokens  $0.0300
  cache read     792,000 tokens  $0.2376
  output         200,000 tokens  $3.0000
cache, prefix writes 1, reads 99: $3.8676

To price your own usage, replace the rates and the token counts with the ones on your bill. Raise WRITES to count each time a gap of more than five minutes expires the cache and forces another write.

Compare cost per accepted task

Cost per token or per active hour only compares fairly once we also count what the spend produced.

To compare cost per accepted task, we:

  1. Define acceptance first.
  2. Measure costs and outcomes on the same task set and serving configuration.
  3. Divide included cost by the number of accepted tasks, only when that count is above zero.

Rates and inputs

Every rate and plan term above changes over time, so we check the provider terms for the model and account we plan to use before reusing these numbers.

The ones to re-check are:

What an attacker targets here

Pay-per-token billing turns access into spend, so an attacker targets the keys and routes that can run up the bill.