An LLM gateway between coding agents and providers holds the keys, reads every prompt and decides where each request goes, which makes it a target of its own. Our gateway post traced one request through LiteLLM to show what the adapter changes. Here we read the same position from the attacker's side and end with a checklist for anyone running one.

The gateway usually holds the provider keys

A gateway usually holds the credentials its providers accept, and whoever controls the gateway then controls those credentials. LiteLLM's configuration shows the layout:

LiteLLM's incident post of October 21, 2026 spells out the consequence. Its rotation list asks customers to rotate every provider key stored in the LiteLLM proxy database, plus the master and virtual keys and the DATABASE_URL password. One compromised gateway means every key stored in it.

Kong's AI Proxy plugin also keeps the provider credential at the gateway, in the plugin's config.auth.header_value. Kong's reference marks that field as encrypted and referenceable, so it can point to a vault instead of holding the value, and by default a request cannot override it.

Our own trace used two different keys: the client called the gateway with x-api-key: $LITELLM_KEY, while a recorder on the host added the upstream key, so the provider key never entered the gateway's container.

Sources: LiteLLM documentation, Config.yaml, Virtual Keys, Model Management and Rotating the Master Key; LiteLLM, security incident of October 21, 2026; Kong, AI Proxy configuration reference; our trace in When a coding agent needs a proxy or gateway.

The gateway is a web app with its own CVEs

A gateway is an HTTP server with a database, an admin role and outbound requests, and LiteLLM's 2026 advisories hit each of those. The GitHub advisories give these versions:

The first is the reason to keep a gateway off the internet, and the second is a missing role check, which let any authenticated user change what only an admin should. The SQL injection advisory lists a workaround while an upgrade waits: set disable_error_logs: true under general_settings.

The package itself has been the attack twice in 2026. On March 24, litellm==1.82.7 and litellm==1.82.8 on PyPI carried a payload that harvested environment variables, SSH keys, cloud credentials, Kubernetes tokens and database passwords. LiteLLM's update says they were live from 10:39 UTC for about 40 minutes before PyPI quarantined them, and its townhall post says the team had worked with PyPI to delete them by 16:00 UTC. The same update says customers running the official LiteLLM Proxy Docker image were not affected that time.

On October 21, a malicious litellm==1.107.0 was live on PyPI from 11:39 to 13:23 UTC, and the Docker images v1.107.0, latest, main-latest and main-stable were compromised from 11:46 to 14:20 UTC. LiteLLM says that version contained compiled malware that steals credentials and loads remote-code execution tools. LiteLLM found an unexpected user account in an internal deployment that was exposed to the public internet, and its interim belief is that the attacker gained access to that deployment and took GitHub CLI OAuth tokens from its logs. The post warns that some early assumptions proved inaccurate or incomplete, so we treat that chain as LiteLLM's interim belief.

The trace in our gateway post ran LiteLLM 1.106.4, which is not one of the compromised versions.

Sources: GitHub, GHSA-r75f-5x8p-qvmc, GHSA-53mr-6c8q-9789 and GHSA-hx8v-g79f-8w5f; NVD, CVE-2026-42208; LiteLLM, security hardening of April 2026, security update of March 2026, townhall updates and security incident of October 21, 2026.

It reads and can log every prompt

A gateway that converts a request has to read it, and whatever it reads it can also write to a log. With --detailed_debug, our trace printed the prompt text in full while it masked the upstream Authorization header. LiteLLM's debugging page shows the same behavior, and adds litellm_request_debug=True in a request body to emit the raw provider request and response for that one request.

Debug logs are not the only copy. LiteLLM documents three controls for content:

Stored content is worth stealing because it can hold secrets. LiteLLM's October post reports that it observed, in specific situations, Claude Code embedding a user's GitHub CLI OAuth token in a response body, so a gateway that logs response bodies can end up holding that token.

Sources: LiteLLM documentation, Debugging, Logging, UI Spend Log Settings and Security FAQ; LiteLLM, security incident of October 21, 2026.

What the reply tells the client

Leaks also run back to whoever holds a client key. In our trace, the id LiteLLM returned was resp_ followed by base64, and decoding it gave this, with the hash and the upstream id shortened:

litellm:custom_llm_provider:openai;model_id:<hash>;response_id:resp_...

Our run showed three things:

Source: our trace in When a coding agent needs a proxy or gateway.

Unbounded consumption and denial of wallet

Every request through a gateway is billed to someone, so a stolen client key or an open route turns into spend. OWASP's LLM10:2025, Unbounded Consumption, covers an application that lets users run excessive and uncontrolled inferences, with denial of service, economic losses, model theft and degraded service as the results. It names denial of wallet as the case where a high volume of operations exploits the cost-per-use model of cloud AI services, and it recommends rate limiting and user quotas per source.

LiteLLM's controls for this sit on keys, tags and the whole instance:

A tag's token limit has a gap worth knowing: its tpm_limit is charged with a request's actual usage after the call completes, so the request that crosses the limit is admitted and only the next one is rejected. One very large request therefore passes in full.

Request count is also the wrong measure on its own. Shumailov and colleagues built inputs they call sponge examples, designed to maximise energy consumption and latency, and raised energy consumption by a factor of 10 to 200 on the vision and language models they tested, so counting requests does not bound the work each one causes.

Sources: OWASP, LLM10:2025 Unbounded Consumption; LiteLLM documentation, Budgets, Rate Limits, tag budgets and config settings; Shumailov et al., Sponge Examples: Energy-Latency Attacks on Neural Networks, 6th IEEE EuroS&P.

Model extraction through query access

A gateway key is also query access to every model behind it, and query access is enough to copy behavior. MITRE ATLAS technique AML.T0024, Exfiltration via AI Inference API, describes adversaries taking information out of a model through the responses its inference API returns, including responses used to replicate the model's behavior. Its procedure example AML.CS0056 records that DeepSeek, Moonshot AI and MiniMax obtained Claude-generated responses through API access, including access through commercial proxy services. OWASP's LLM10:2025 makes the same point about crafted queries that collect enough outputs to build a partial copy or shadow model.

ATLAS mitigation AML.M0004 is the gateway's job: limit the volume of API queries in a given period, which caps how much an attacker can learn.

Sources: MITRE ATLAS, AML.T0024 (the page serves its content with an HTTP 404 status); OWASP, LLM10:2025 Unbounded Consumption.

The route decides which provider gets the data

The client asks for a model name, and the gateway's configuration decides which provider receives the prompt and whose data terms apply. LiteLLM offers three ways to change that answer and one to hide a route:

In our trace the reply named only the alias, so a switch of provider is invisible to the client unless it decodes the id. Calling that invisible is our reading of the trace; LiteLLM's documentation describes these mechanisms as features and does not call them a risk. An attacker who can edit the configuration, as the /config/update advisory allowed, could use the same mechanisms to send prompts to a backend of their choosing. The operator's record is in the spend logs, which store the attempted fallbacks and the original model group.

Sources: LiteLLM documentation, Config.yaml, Routing, Proxy Load Balancing and Fallbacks.

Guardrails are inline checkpoints

Because the gateway sits on the path, it is also where input and output checks can run. LiteLLM's guardrails run in one of these modes:

The mode decides whether anything is stopped: in logging_only mode nothing is blocked, and a request the input scan flags still reaches the model.

Source: LiteLLM documentation, Guardrails Quick Start.

An illustrative config

Several of the controls above fit in one file, shown here as an illustrative LiteLLM YAML config we wrote from the documentation and did not deploy:

# Illustrative only: built from LiteLLM's documented settings, not deployed.
model_list:
  - model_name: gpt-route
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY   # provider key read from the environment

litellm_settings:
  turn_off_message_logging: True           # no prompt or reply content to logging providers

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY  # admin key, never handed to clients
  store_prompts_in_spend_logs: false       # spend logs keep metadata only, such as tokens, cost and model
  disable_error_logs: true                 # the CVE-2026-42208 workaround

Clients then get virtual keys with a budget instead of the master key:

curl -s "$LITELLM_URL/key/generate" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"max_budget": 10}'

Set LITELLM_SALT_KEY before adding any model, because LiteLLM warns that credentials encrypted with one value cannot be decrypted after it changes.

Sources: LiteLLM documentation, Config.yaml, Logging, UI Spend Log Settings, Virtual Keys, Budgets, Rate Limits and Model Management; GitHub, GHSA-r75f-5x8p-qvmc.

A checklist for running a gateway

Each item answers one of the attacks above:

  1. Patch past every fixed version above, 1.83.9 or later for the three advisories, skip the compromised releases, and recheck the advisories before each upgrade.
  2. Never expose the gateway to the internet, since LiteLLM's interim account of the October incident starts from an exposed deployment and the SQL injection needs no credentials.
  3. Keep the master key, the salt key, the provider keys and each client's virtual key separate, and hand clients only virtual keys.
  4. Turn off content logging with turn_off_message_logging and store_prompts_in_spend_logs: false, and keep --detailed_debug for test runs.
  5. Give every virtual key a budget and every source a rate limit, and remember that a token limit lands after the request.
  6. Pin the package or image version and verify it before upgrading, because both 2026 compromises were published as litellm releases on PyPI.
  7. Review the logs for secrets, unexpected accounts and unexpected fallbacks; the spend logs record attempted fallbacks.
A client and the holder of a stolen client key, one gateway, one provider. A vault named Provider keys is cabled to the gateway, and a dashed outline named Log stands beside it. A bar named Stolen key's budget is filled past its limit line; the part above the line is red and labelled Over budget. At the end a card tagged Request + virtual key rests at the provider, a barrier at the gateway reads Rejected, and the request on the stolen key is back at its holder.
  1. The gateway holds the provider keys: a gateway usually holds the credentials its providers accept, and whoever controls the gateway then controls those credentials.
  2. It can log what it reads: a gateway that converts a request has to read it, and whatever it reads it can also write to a log.
  3. A stolen client key turns into spend: every request through a gateway is billed to someone, so a stolen client key or an open route turns into spend.
  4. Clients get only virtual keys: keep the master key, the salt key, the provider keys and the virtual key of each client separate, and hand clients only virtual keys.
  5. A budget on the key: a virtual key takes a max_budget and a budget_duration, and requests are rejected once the key exceeds its budget.
The number of clients and providers drawn carries no meaning. The budget stops the spend only after the key has exceeded it. The dashed outline of the log marks something the gateway can write, not an attacker.

Where to go next

The trace these findings build on is in When a coding agent needs a proxy or gateway. For the credentials a Claude Code route runs under once a gateway is in the path, read Evaluating a Claude Code to GPT route through LiteLLM.