Paying for a reply now or a batch result tomorrow
A coding agent is paid for with API tokens, a subscription, self-hosted hardware, or a mix of the three.
Each one needs its own accounting. Batch rates and prompt caching lower the token bill, and idle time raises the fixed cost of each active hour on hardware we own. The worked examples in the scenario table use published Claude Sonnet 4.6 rates and stated quantities, so the reader can reproduce the arithmetic without running a model.
An agent in your editor waits on every model reply before it takes its next step. A nightly run of 100 pull-request reviews can wait hours for all of its results. A lower batch rate suits the nightly run, and the same wait would stall the agent in the editor.
An interactive API request returns its response to the caller, who uses it before the next dependent step. A provider Batch API, such as the OpenAI Batch API or the Anthropic Message Batches API, accepts a job of independent requests, reports its state, and returns the results later. Continuous batching, which the vLLM README lists as a runtime feature, is a different thing: the server schedules arriving requests together, and the caller submits no batch job.
API work is metered per token. Anyone who can send requests on our key spends that money, and The gateway as an attack surface covers the budgets and rate limits that cap it. Non-urgent jobs can go in batches, and subscriptions sell access under stated limits. Self-hosting adds equipment and operating costs even when the equipment sits idle, and judging its useful output means measuring it on the hardware. So we account for subscriptions, API credits, and self-hosting separately, in five steps, and close with how to set up a cost-per-accepted-task comparison:
- Count the token classes an API bill charges for.
- Decide when a batch rate fits the work.
- Price a cache write separately from its later reads.
- Check the units a subscription uses.
- Work out how idle capacity raises the fixed cost of each active hour on owned hardware.

Figure text
Three lanes compare a caller waiting on a synchronous API, a submitter retrieving a provider batch later, and live requests passing through a serving scheduler. The provider owns the queue and the scheduler.
- Wait now, collect later, or neither
- Lit lamp: the caller is waiting
- Unlit lamp: caller is not waiting
- A provider batch is the caller's own workflow. Grouping inside a server is not a batch you submit.
- 1 Send and wait
- 2 The reply comes back on the same request
- 3 Hand in a job
- 4 Come back for the results
- 5 Live requests, grouped by the server
- 6 Live replies
- Reply
- Live request
- Interactive caller
- Provider API
- Batch submitter
- Provider's batch queue
- Results, later
- Live callers
- Server's scheduler
- Model server
- Results
- Different again: a server can group live requests on its own. The callers hand in no job.
Sources: [1] Business Pricing | OpenAI; [2] Batch API | OpenAI API; [3] Batch processing - Claude Platform Docs; [4] vLLM README at pinned revision.
Count the billed token classes
An API bill is counted in tokens, so the first thing to know is what a token is and which token classes the provider charges for.
A model's tokenizer defines each token. Word counts per token vary, and equal token counts can represent work of different quality. A provider's price schedule can bill these classes separately:
- input
- output
- cache reads
- cache writes
Reasoning tokens are billed as output tokens by OpenAI and Anthropic. They count as their own class only when a price sheet lists them separately, so they are never counted twice. models.dev, an open-source database of AI models with their specs, prices and capabilities, has cost-schema fields for several of these classes. OpenAI's pricing and Anthropic's pricing set the actual charges.
For rates quoted in USD per million tokens, the general calculation is:
API cost in USD = sum(tokens in class × USD per million tokens for class / 1,000,000) + applicable non-token charges
A non-token charge is a fee that is not counted in tokens. Web search is one: Anthropic lists $10 per 1,000 searches and OpenAI lists $10 per 1,000 calls, each plus the tokens of the search content. We count each billable retry once, with its token and non-token charges. The rate is set by:
- the model
- the provider
- the service mode, which Anthropic names standard, batch or priority (OpenAI renamed its priority processing Fast mode on July 30, 2026)
- any context or regional conditions
When batch rates fit the work
Batch rates can lower per-token charges when the work can wait for results.
The OpenAI pricing page offers 50% off input and output rates for Batch API work processed asynchronously over a 24-hour window. Eligibility for the batch discount depends on the model and endpoint, so we check the ones we plan to use. The Batch API guide also defines batch-specific rate limits and expiration behavior. An expired batch can contain completed requests whose tokens are still charged.
Each result has to find its original task.
Because OpenAI's batch output can arrive in a different order from its input, line position is unreliable, and we match each result by its request's custom_id.
A workflow that treats the second output as the second input can attach an answer to the wrong task even though both requests succeeded.
See the batch result retrieval docs for the ordering rule.
Anthropic's Message Batches guide likewise describes independent asynchronous requests and a 24-hour expiry. Its pricing is 50% of standard API prices. Expired or failed items need handling, because some requests may remain incomplete after the 24-hour window.
An agent loop is a poor fit for batch results, because each step depends on the previous reply. Batch fits only the independent work, such as the nightly review run.
Price a cache write separately from its later reads
The second way to lower token charges is prompt caching, which prices repeated prompt content differently from new content.
Provider prompt caching reuses eligible prompt content under the provider's cache rules. For Claude Sonnet 4.6, the pricing table lists standard input at $3, output at $15, cache writes at $3.75, and cache reads at $0.30 per million tokens. The write price is for the default cache, which lives five minutes and is refreshed at no cost each time it is used.
Each scenario prices 100 hypothetical requests, each with 10,000 input tokens and 2,000 output tokens. The cache scenario splits each input into an 8,000-token reusable prefix and 2,000 unique tokens. We assume one prefix write followed by 99 cache reads, each within five minutes of the previous use, so the cache never expires. A gap of more than five minutes between requests would expire the cache and force another write, so the $3.8676 total depends on that assumption.
Token-charge arithmetic for the three scenarios.
Each charge = tokens × USD per million tokens / 1,000,000. We use rates from Pricing - Claude Platform Docs. The cache partition is 200,000 unique input + 8,000 prefix-write + 792,000 read tokens = 1,000,000 input tokens.
The rates are Claude Sonnet 4.6 prices in USD per million tokens.
Sources: Pricing - Claude Platform Docs.
| Scenario | Token class | Quantity in tokens | USD per million tokens | Calculated USD |
|---|---|---|---|---|
| Standard, no cache | Input Token class: input | 1,000,000 | 3 | 3.00 |
| Standard, no cache | Output Token class: output | 200,000 | 15 | 3.00 |
| Batch, no cache | Input Token class: input | 1,000,000 | 1.5 | 1.50 |
| Batch, no cache | Output Token class: output | 200,000 | 7.5 | 1.50 |
| One write, 99 hits | Unique input Token class: unique input | 200,000 | 3 | 0.60 |
| One write, 99 hits | Five-minute cache write Token class: five-minute cache write | 8,000 | 3.75 | 0.03 |
| One write, 99 hits | Cache reads Token class: cache reads | 792,000 | 0.3 | 0.2376 |
| One write, 99 hits | Output Token class: output | 200,000 | 15 | 3.00 |
| Standard, no cache | TOTAL | Not applicable | Not applicable | 6.00 |
| Batch, no cache | TOTAL | Not applicable | Not applicable | 3.00 |
| One write, 99 hits | TOTAL | Not applicable | Not applicable | 3.8676 |
The scenario totals are $6.00, $3.00, and $3.8676 respectively.
For each row, we multiply tokens by the rate and divide by 1,000,000 (tokens × rate / 1,000,000).
The totals cover token charges only, excluding request and tool charges, retries, and any regional, priority, or long-context surcharge.
Anthropic's pricing modifiers allow cache and batch discounts to stack, but we use standard, non-batch rates for the cache scenario. Batch cache hits are also best effort because requests run asynchronously and concurrently. For a real estimate, we replace the assumed cache counts with the billing-class counts from our own usage.

- Standard, no cache: 100 hypothetical requests at standard rates: input $3.00 and output $3.00, a total of $6.00.
- Batch, no cache: batch pricing is 50% of standard prices: input $1.50 and output $1.50, a total of $3.00.
- One write, 99 hits: assuming one prefix write followed by 99 cache reads, at standard, non-batch rates: unique input $0.60, cache write $0.03, cache read $0.2376 and output $3.00, a total of $3.8676.

Figure text
Three stacked bars show USD token charges of $6.00 for standard input and output, $3.00 for batch input and output, and $3.8676 for unique input, one prefix write, cache reads, and output. The rates are Claude Sonnet 4.6 prices in USD per million tokens.
- One workload, three bills
- Input at the input rate (lower)
- Output (upper segment)
- Cached input, re-read (thin parts)
- Token charges only, for hypothetical requests at published rates. Small segments are to scale; the table has the exact numbers.
- 1 Priced at standard rates
- 2 The same work at batch rates
- 3 The same work with a cached prefix
- 4 Holds only while the cache stays alive
- Batch, no cache: Output $1.50
- $6.00
- $3.00
- $3.8676
- The same 100 requests
- Standard, no cache
- One write, 99 reads
- The cache assumption
- Same work
- No gap over five minutes
- Output $3.00
- The third bill assumes the stored prefix is reused often enough that it never expires.
Sources: [1] Pricing - Claude Platform Docs.
Check the units a subscription uses
A subscription does not bill per token; it bills for access under the plan's terms, so we check what units each plan counts in before comparing it with API charges.
Each plan sets its own units, and its included products may use different meters. The Claude Max support page specifies five-hour and weekly limits, plus an additional model-specific weekly limit. The Z.AI GLM Coding Plan page labels its allowances as weekly credits. Converting those units into a common token balance needs the provider's conversion rules.
Eligible Anthropic plans can also include a separate API-credit balance. The monthly API-credit rules cover Max, Ultra, and Team under stated eligibility and organization-linking conditions, and Free, Pro, and Enterprise are excluded from that offer. API-key users in the linked Console organization share the credits, which expire at the billing-cycle boundary and don't raise subscription usage limits. Those API credits exclude interactive Claude Code sessions.
We compare plans by their units or windows, whether usage is paid through a subscription login or an API key, and the available pricing details.
Plan names and prices change; the linked pages have the current ones.
Sources: Plans & pricing | Claude by Anthropic, What is the Max plan? | Claude Help Center, GLM Coding Plan - AI Coding Powered by GLM-5.3 & GLM-5.3-Flash & GLM-5.3-FlashX-Legal for Agents & IDEs, Monthly API credits for Max, Ultra, and Team plans | Claude Help Center.
| Plan or access type | Unit or window | Subscription login or API key | Price and other commercial detail |
|---|---|---|---|
| Claude Max | Five-hour and weekly limits; the plan states no fixed token allowance. | Subscription-login use counts against the plan limits and is separate from API-key use of API credits. | $100/month for web subscriptions; the mobile price may vary. |
| Claude Ultra | Weekly allowance; verified accounts have no five-hour limit, though not every account qualifies | Claude Code included; subscription-login use and API-key credit use are separate balances | $200/month, tax excluded, per Plans & pricing | Claude by Anthropic |
| Z.AI GLM Coding Plan | Weekly credits; the Lite tier lists 10,000 credits/week. | The page says the plan supports 20+ agent tools, including Claude Code, and does not say how a tool signs in. | The page shows multiple billing and promotional prices and gives no normalized price or token conversion. |
| Anthropic monthly API credits | A monthly balance; unused credits expire each billing cycle. | Covers eligible API-key usage in the linked Console organization. | The amount and eligibility depend on the plan. Once the credits run out, the organization's purchased-credit, reload, or invoicing arrangement applies. |
| OpenAI API alongside ChatGPT Plus, Business, Enterprise, or Edu | API usage is billed separately from these subscriptions. | Funded through the API account. | API charges follow the model's API rate schedule. |
OpenAI bills API charges apart from subscription fees too. So before comparing a plan with API rates, we check which balance a client draws on: a subscription login uses the plan's limits, and an API key is billed to the API account.
How idle capacity affects fixed cost per active hour
Self-hosting replaces per-token charges with fixed equipment costs, and how much of the time the hardware is active decides what each active hour costs.
Self-hosting brings costs that keep running while the equipment sits idle.
Let F be a fixed monthly expenditure in USD, H the available hours in that month, and u the fraction of those hours treated as active.
For F > 0, H > 0, and 0 < u <= 1, fixed cost allocated per active hour is F / (H × u).
At full utilization it is F / H.
Dividing the first by the second gives the dimensionless ratio y = 1/u.
The ratio rises as the active fraction falls.
Values of y = 1/u at five utilization levels.
y = 1/u for 0 < u ≤ 1; undefined at u = 0. The ratio is normalized to full utilization.
The curve is y = 1/u. The chart clips above 10.
| Active utilization fraction | Active utilization | Fixed cost per active hour relative to full utilization |
|---|---|---|
| 0.10 | 10% | 10 |
| 0.25 | 25% | 4 |
| 0.50 | 50% | 2 |
| 0.75 | 75% | 1.333333… Fixed-cost allocation ratio y: 4/3 ≈ 1.3333 |
| 1.00 | 100% | 1 |
At 25% utilization the allocation is four times the full-utilization figure, and F doesn't change.
At zero utilization, cost per active hour is undefined because there are no active hours to divide by.
Before applying the ratio to a real system, we pick one definition of "active" and keep it.
GPU busy, a request in flight, and a developer logged in each give a different u for the same month.
This formula calculates part of the ownership cost, including active and idle energy:
period_cost = F + H*e*(u*P_active + (1-u)*P_idle)
One accounting period and currency apply throughout:
Fis whatever fixed costs we choose to include for the period, in USD. We include capital amortization, fixed maintenance (labor hours times a USD/hour rate), and other fixed overhead, and we list each one.His the available powered hours in that same period, withH > 0, anduis the active fraction, with0 < u <= 1.P_activeandP_idleare measured active and idle power in kW, andeis the electricity price in USD/kWh, so the energy term comes out in USD for the period.N_acceptedis the measured number of tasks meeting a specified quality criterion in that period and serving configuration.
Cost per active hour is period_cost/(H*u), in USD/active hour.
Cost per accepted task is period_cost/N_accepted, in USD/accepted task, defined only when N_accepted > 0.
At zero utilization, the period cost can still include fixed expenditure and idle energy.
Changing utilization shifts only the mix of active and idle power, and F stays fixed.
Usage-dependent maintenance and opportunity costs are not in the formula.

Figure text
Two meters stand for the relation y = 1/u: the share of hours in use, u, between 0 and 1, and the fixed cost per active hour, y, which is undefined at zero. The cost meter is dimensionless and normalized to the full-utilization level.
- Idle hours raise the hourly cost
- Meter level: a ratio, not dollars
- No active hours: no figure
- Lower use spreads the unchanged monthly bill over fewer active hours. The scale is a ratio against full use, not dollars.
- 1 The share of the month's hours in use
- 2 The same bill, divided over the hours in use
- 3 Never in use, nothing to divide by
- Fixed monthly bill
- No hours in use
- No figure
- Share of hours in use
- Owned hardware: Available hours in the month
- Fixed cost per active hour: Compared with full use
- Never in use: the bill is still owed, but there is no active hour to divide it by.
Check the cache arithmetic yourself
Listing each quantity beside its billing class makes double-counting visible, so we lay out the cache case in full.
For the cache case, the input partition is 200,000 + 8,000 + 792,000 = 1,000,000 tokens.
The read quantity comes from 99 × 8,000, and the unique quantity from 100 × 2,000.
Pricing all four classes gives (200,000 × 3 + 8,000 × 3.75 + 792,000 × 0.30 + 200,000 × 15) / 1,000,000 = $3.8676.
Charging another million standard input tokens on top would double-count the partitioned input.
A short Python script does the same arithmetic, with the rates and quantities from the scenario table as inputs at the top:
"""Token charges for 100 requests in three scenarios, from this post's figures."""
# USD per million tokens: the Claude Sonnet 4.6 rates this post quotes, not live prices.
INPUT, OUTPUT, CACHE_WRITE, CACHE_READ = 3.00, 15.00, 3.75, 0.30
BATCH_FACTOR = 0.5 # batch pricing is 50% of standard prices
REQUESTS = 100
PREFIX, UNIQUE, OUT = 8_000, 2_000, 2_000 # tokens per request
WRITES = 1 # prefix writes; every other request reads the cached prefix
def usd(tokens, rate):
return tokens * rate / 1_000_000
standard = usd(REQUESTS * (PREFIX + UNIQUE), INPUT) + usd(REQUESTS * OUT, OUTPUT)
print(f"standard, no cache: ${standard:.2f}")
print(f"batch, no cache: ${standard * BATCH_FACTOR:.2f}")
rows = {
"unique input": (REQUESTS * UNIQUE, INPUT),
"cache write": (WRITES * PREFIX, CACHE_WRITE),
"cache read": ((REQUESTS - WRITES) * PREFIX, CACHE_READ),
"output": (REQUESTS * OUT, OUTPUT),
}
for name, (tokens, rate) in rows.items():
print(f" {name:<12} {tokens:>9,} tokens ${usd(tokens, rate):.4f}")
total = sum(usd(tokens, rate) for tokens, rate in rows.values())
print(f"cache, prefix writes {WRITES}, reads {REQUESTS - WRITES}: ${total:.4f}")
It prints the three scenario totals and the four rows of the cache case:
$ python3 session_cost.py
standard, no cache: $6.00
batch, no cache: $3.00
unique input 200,000 tokens $0.6000
cache write 8,000 tokens $0.0300
cache read 792,000 tokens $0.2376
output 200,000 tokens $3.0000
cache, prefix writes 1, reads 99: $3.8676
To price your own usage, replace the rates and the token counts with the ones on your bill.
Raise WRITES to count each time a gap of more than five minutes expires the cache and forces another write.
Compare cost per accepted task
Cost per token or per active hour only compares fairly once we also count what the spend produced.
To compare cost per accepted task, we:
- Define acceptance first.
- Measure costs and outcomes on the same task set and serving configuration.
- Divide included cost by the number of accepted tasks, only when that count is above zero.
Rates and inputs
Every rate and plan term above changes over time, so we check the provider terms for the model and account we plan to use before reusing these numbers.
The ones to re-check are:
- rates
- batch eligibility
- cache rules
- plan limits
- credit terms
What an attacker targets here
Pay-per-token billing turns access into spend, so an attacker targets the keys and routes that can run up the bill.
- Volume: OWASP's LLM10:2025 calls a high volume of billed operations denial of wallet.
We give each virtual key a
max_budget, because LiteLLM's globalmax_budgetdefaults to 0, which means no cap, as The gateway as an attack surface notes. - Size: sponge examples raised energy consumption 10 to 200 times on the models Shumailov and colleagues tested, so a request count does not bound the work each request causes.
We budget in money per key rather than in requests, knowing that a tag's
tpm_limitis charged only after the call completes.