An LLM gateway between coding agents and providers holds the keys, reads every prompt and decides where each request goes, which makes it a target of its own. Our gateway post traced one request through LiteLLM to show what the adapter changes. Here we read the same position from the attacker's side and end with a checklist for anyone running one.
The gateway usually holds the provider keys
A gateway usually holds the credentials its providers accept, and whoever controls the gateway then controls those credentials. LiteLLM's configuration shows the layout:
- each deployment in
model_listcarries its provider key inlitellm_params, for exampleapi_key: "os.environ/AZURE_API_KEY_EU"; - the
master_keyis a separate credential that LiteLLM calls the Proxy Admin key, and it can create other keys through/key/generate; - those generated keys are virtual keys, which clients can send instead of the master key;
- provider keys added through the UI are stored in the database, encrypted with
LITELLM_SALT_KEY, or with the master key when no salt key is set.
LiteLLM's incident post of October 21, 2026 spells out the consequence.
Its rotation list asks customers to rotate every provider key stored in the LiteLLM proxy database, plus the master and virtual keys and the DATABASE_URL password.
One compromised gateway means every key stored in it.
Kong's AI Proxy plugin also keeps the provider credential at the gateway, in the plugin's config.auth.header_value.
Kong's reference marks that field as encrypted and referenceable, so it can point to a vault instead of holding the value, and by default a request cannot override it.
Our own trace used two different keys: the client called the gateway with x-api-key: $LITELLM_KEY, while a recorder on the host added the upstream key, so the provider key never entered the gateway's container.
Sources: LiteLLM documentation, Config.yaml, Virtual Keys, Model Management and Rotating the Master Key; LiteLLM, security incident of October 21, 2026; Kong, AI Proxy configuration reference; our trace in When a coding agent needs a proxy or gateway.
The gateway is a web app with its own CVEs
A gateway is an HTTP server with a database, an admin role and outbound requests, and LiteLLM's 2026 advisories hit each of those. The GitHub advisories give these versions:
- CVE-2026-42208, critical, CVSS v4 9.3: an SQL injection in API key verification, reachable without authentication through a crafted
Authorizationheader on any LLM API route, such asPOST /chat/completions; affected from 1.81.16 up to but not including 1.83.7, patched in 1.83.7; NVD lists it in CISA's Known Exploited Vulnerabilities catalog, added May 8, 2026; - CVE-2026-35029, high:
/config/updatedid not check that the caller had the admin role, so any authenticated user could change the proxy's configuration and environment, register a pass-through handler pointing at their own Python code for remote code execution, read server files and overwrite the UI username and password; affected before 1.83.0, patched in 1.83.0; - CVE-2026-59823, moderate: a server-side request forgery in which an authenticated caller smuggles an
api_baseinside theuser_configrequest body and sends the proxy's outbound request to a host of their choosing; affected up to and including 1.83.8, patched in 1.83.9.
The first is the reason to keep a gateway off the internet, and the second is a missing role check, which let any authenticated user change what only an admin should.
The SQL injection advisory lists a workaround while an upgrade waits: set disable_error_logs: true under general_settings.
The package itself has been the attack twice in 2026.
On March 24, litellm==1.82.7 and litellm==1.82.8 on PyPI carried a payload that harvested environment variables, SSH keys, cloud credentials, Kubernetes tokens and database passwords.
LiteLLM's update says they were live from 10:39 UTC for about 40 minutes before PyPI quarantined them, and its townhall post says the team had worked with PyPI to delete them by 16:00 UTC.
The same update says customers running the official LiteLLM Proxy Docker image were not affected that time.
On October 21, a malicious litellm==1.107.0 was live on PyPI from 11:39 to 13:23 UTC, and the Docker images v1.107.0, latest, main-latest and main-stable were compromised from 11:46 to 14:20 UTC.
LiteLLM says that version contained compiled malware that steals credentials and loads remote-code execution tools.
LiteLLM found an unexpected user account in an internal deployment that was exposed to the public internet, and its interim belief is that the attacker gained access to that deployment and took GitHub CLI OAuth tokens from its logs.
The post warns that some early assumptions proved inaccurate or incomplete, so we treat that chain as LiteLLM's interim belief.
The trace in our gateway post ran LiteLLM 1.106.4, which is not one of the compromised versions.
Sources: GitHub, GHSA-r75f-5x8p-qvmc, GHSA-53mr-6c8q-9789 and GHSA-hx8v-g79f-8w5f; NVD, CVE-2026-42208; LiteLLM, security hardening of April 2026, security update of March 2026, townhall updates and security incident of October 21, 2026.
It reads and can log every prompt
A gateway that converts a request has to read it, and whatever it reads it can also write to a log.
With --detailed_debug, our trace printed the prompt text in full while it masked the upstream Authorization header.
LiteLLM's debugging page shows the same behavior, and adds litellm_request_debug=True in a request body to emit the raw provider request and response for that one request.
Debug logs are not the only copy. LiteLLM documents three controls for content:
turn_off_message_loggingkeeps messages and responses out of the logging provider while spend and other metadata are still tracked;store_prompts_in_spend_logsdecides whether new spend logs keep the request and response content for the Logs UI, or only metadata such as tokens, cost and model, and it does not change what other logging callbacks receive;disable_spend_logsanddisable_error_logsappear in LiteLLM's security checklist as an optional step when logs contain sensitive data.
Stored content is worth stealing because it can hold secrets. LiteLLM's October post reports that it observed, in specific situations, Claude Code embedding a user's GitHub CLI OAuth token in a response body, so a gateway that logs response bodies can end up holding that token.
Sources: LiteLLM documentation, Debugging, Logging, UI Spend Log Settings and Security FAQ; LiteLLM, security incident of October 21, 2026.
What the reply tells the client
Leaks also run back to whoever holds a client key.
In our trace, the id LiteLLM returned was resp_ followed by base64, and decoding it gave this, with the hash and the upstream id shortened:
litellm:custom_llm_provider:openai;model_id:<hash>;response_id:resp_...
Our run showed three things:
- the decoded id names the provider and carries the upstream response id, so any client can read LiteLLM's provider label and the upstream response id;
- the upstream reply reported
"store": truealthough LiteLLM's request did not setstore, a setting the client never chose; - LiteLLM replaced the upstream model name,
gpt-6-luna, with the alias the client asked for,gpt-route, so the visiblemodelfield hid the real model.
Source: our trace in When a coding agent needs a proxy or gateway.
Unbounded consumption and denial of wallet
Every request through a gateway is billed to someone, so a stolen client key or an open route turns into spend. OWASP's LLM10:2025, Unbounded Consumption, covers an application that lets users run excessive and uncontrolled inferences, with denial of service, economic losses, model theft and degraded service as the results. It names denial of wallet as the case where a high volume of operations exploits the cost-per-use model of cloud AI services, and it recommends rate limiting and user quotas per source.
LiteLLM's controls for this sit on keys, tags and the whole instance:
- a virtual key takes a
max_budgetand abudget_duration, and requests are rejected once the key exceeds its budget; - a per-model budget returns a
budget_exceedederror by default; - a tag past its limit is rejected with HTTP 429;
- a global
max_budgetcaps spend across all providers, and its default of 0 means no cap.
A tag's token limit has a gap worth knowing: its tpm_limit is charged with a request's actual usage after the call completes, so the request that crosses the limit is admitted and only the next one is rejected.
One very large request therefore passes in full.
Request count is also the wrong measure on its own. Shumailov and colleagues built inputs they call sponge examples, designed to maximise energy consumption and latency, and raised energy consumption by a factor of 10 to 200 on the vision and language models they tested, so counting requests does not bound the work each one causes.
Sources: OWASP, LLM10:2025 Unbounded Consumption; LiteLLM documentation, Budgets, Rate Limits, tag budgets and config settings; Shumailov et al., Sponge Examples: Energy-Latency Attacks on Neural Networks, 6th IEEE EuroS&P.
Model extraction through query access
A gateway key is also query access to every model behind it, and query access is enough to copy behavior. MITRE ATLAS technique AML.T0024, Exfiltration via AI Inference API, describes adversaries taking information out of a model through the responses its inference API returns, including responses used to replicate the model's behavior. Its procedure example AML.CS0056 records that DeepSeek, Moonshot AI and MiniMax obtained Claude-generated responses through API access, including access through commercial proxy services. OWASP's LLM10:2025 makes the same point about crafted queries that collect enough outputs to build a partial copy or shadow model.
ATLAS mitigation AML.M0004 is the gateway's job: limit the volume of API queries in a given period, which caps how much an attacker can learn.
Sources: MITRE ATLAS, AML.T0024 (the page serves its content with an HTTP 404 status); OWASP, LLM10:2025 Unbounded Consumption.
The route decides which provider gets the data
The client asks for a model name, and the gateway's configuration decides which provider receives the prompt and whose data terms apply. LiteLLM offers three ways to change that answer and one to hide a route:
- several deployments can share one
model_name, which LiteLLM load-balances, and its routing example putsopenai/gpt-6.1-solandazure/gpt-6.1-solunder one name; model_group_aliassends every request for one name to another model group;- a fallback moves a failed call, after its retries, to another model group, and LiteLLM's example falls back from a
gpt-6.1-solgroup to ananthropic-claudegroup; hidden: truekeeps a route out of/v1/modelsand the model info endpoints.
In our trace the reply named only the alias, so a switch of provider is invisible to the client unless it decodes the id.
Calling that invisible is our reading of the trace; LiteLLM's documentation describes these mechanisms as features and does not call them a risk.
An attacker who can edit the configuration, as the /config/update advisory allowed, could use the same mechanisms to send prompts to a backend of their choosing.
The operator's record is in the spend logs, which store the attempted fallbacks and the original model group.
Sources: LiteLLM documentation, Config.yaml, Routing, Proxy Load Balancing and Fallbacks.
Guardrails are inline checkpoints
Because the gateway sits on the path, it is also where input and output checks can run. LiteLLM's guardrails run in one of these modes:
pre_callchecks the input before the model call;during_callchecks the input in parallel with the call and holds the response until the check completes;post_callchecks input and output after the call.
The mode decides whether anything is stopped: in logging_only mode nothing is blocked, and a request the input scan flags still reaches the model.
Source: LiteLLM documentation, Guardrails Quick Start.
An illustrative config
Several of the controls above fit in one file, shown here as an illustrative LiteLLM YAML config we wrote from the documentation and did not deploy:
# Illustrative only: built from LiteLLM's documented settings, not deployed.
model_list:
- model_name: gpt-route
litellm_params:
model: openai/gpt-6-luna
api_key: os.environ/OPENAI_API_KEY # provider key read from the environment
litellm_settings:
turn_off_message_logging: True # no prompt or reply content to logging providers
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY # admin key, never handed to clients
store_prompts_in_spend_logs: false # spend logs keep metadata only, such as tokens, cost and model
disable_error_logs: true # the CVE-2026-42208 workaround
Clients then get virtual keys with a budget instead of the master key:
curl -s "$LITELLM_URL/key/generate" \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"max_budget": 10}'
Set LITELLM_SALT_KEY before adding any model, because LiteLLM warns that credentials encrypted with one value cannot be decrypted after it changes.
Sources: LiteLLM documentation, Config.yaml, Logging, UI Spend Log Settings, Virtual Keys, Budgets, Rate Limits and Model Management; GitHub, GHSA-r75f-5x8p-qvmc.
A checklist for running a gateway
Each item answers one of the attacks above:
- Patch past every fixed version above, 1.83.9 or later for the three advisories, skip the compromised releases, and recheck the advisories before each upgrade.
- Never expose the gateway to the internet, since LiteLLM's interim account of the October incident starts from an exposed deployment and the SQL injection needs no credentials.
- Keep the master key, the salt key, the provider keys and each client's virtual key separate, and hand clients only virtual keys.
- Turn off content logging with
turn_off_message_loggingandstore_prompts_in_spend_logs: false, and keep--detailed_debugfor test runs. - Give every virtual key a budget and every source a rate limit, and remember that a token limit lands after the request.
- Pin the package or image version and verify it before upgrading, because both 2026 compromises were published as litellm releases on PyPI.
- Review the logs for secrets, unexpected accounts and unexpected fallbacks; the spend logs record attempted fallbacks.

- The gateway holds the provider keys: a gateway usually holds the credentials its providers accept, and whoever controls the gateway then controls those credentials.
- It can log what it reads: a gateway that converts a request has to read it, and whatever it reads it can also write to a log.
- A stolen client key turns into spend: every request through a gateway is billed to someone, so a stolen client key or an open route turns into spend.
- Clients get only virtual keys: keep the master key, the salt key, the provider keys and the virtual key of each client separate, and hand clients only virtual keys.
- A budget on the key: a virtual key takes a
max_budgetand abudget_duration, and requests are rejected once the key exceeds its budget.
Where to go next
The trace these findings build on is in When a coding agent needs a proxy or gateway. For the credentials a Claude Code route runs under once a gateway is in the path, read Evaluating a Claude Code to GPT route through LiteLLM.