Short definitions of the terms this series uses, each with one source you can check.
When an article in the series explains a term in depth, its entry links there too.
APIs and clients
- API
- An application programming interface: the requests a program accepts and the responses it returns. A coding agent reaches a model through one. Source: MDN Web Docs. Our article: Which APIs do coding agents support?.
- Model
- In this series, a large language model: a trained network that reads tokens and predicts the next ones. It answers a coding agent with text or tool calls. Source: Claude Platform Docs glossary.
- Token
- The unit a model reads and writes, such as a word, part of a word, a character or a byte. Context limits and most API prices are counted in tokens. Source: Claude Platform Docs glossary.
- Context window
- How much text, counted in tokens, a model can look back on while it writes a reply. Source: Claude Platform Docs glossary.
- Tool call
- A structured request from the model to run a named tool with arguments. The client runs the tool and sends the result back in its next request. Source: Claude Platform Docs: tool use. Our article: How a coding agent connects its tools to a model.
- MCP
- Model Context Protocol: an open protocol that standardizes how applications give models context and tools. Source: Model Context Protocol docs. Our article: MCP servers your coding agent should not trust.
- Prompt injection
- Input that changes a model's behavior or output in ways its developer did not intend. Indirect prompt injection arrives through external content the model reads, such as a website or a file. Source: OWASP LLM01:2025 Prompt Injection. Our article: Tool-call trust in coding agents.
- Messages API
- Anthropic's API for Claude: a request carries alternating user and assistant messages, and tool use travels as content blocks. Source: Claude API reference: Messages.
- Chat Completions
- OpenAI's original chat API: a request carries a list of role-tagged messages and the reply carries one or more choices. Many other servers accept this shape. Source: OpenAI API reference: Chat Completions.
- Responses API
- OpenAI's newer API: a request carries input items and the reply returns output items, including tool calls, in one response object. Source: OpenAI API reference: Responses.
- OpenAI-compatible
- A server that accepts OpenAI's request and response shapes, so a client written for OpenAI can point at it instead. Source: vLLM docs: OpenAI-compatible server.
Hosting and routing
- Endpoint
- The URL a client sends API requests to. A hosted model sits behind a provider's endpoint, often with a deployment or model name in the path or body. Source: Microsoft Learn: Foundry endpoints. Our article: Finding a hosted model's endpoint and deployment.
- Inference
- Running a trained model to produce output, as opposed to training it. Serving a coding agent is inference. Source: NVIDIA glossary: AI inference.
- Gateway
- A shared service in front of model providers that adds central keys, routing, budgets and logs for many users. Source: LiteLLM docs: proxy server. Our article: When a coding agent needs a proxy or gateway.
- Proxy
- A server between a client and the real destination that forwards requests. Here it often translates one API shape into another. Source: MDN Web Docs. Our article: When a coding agent needs a proxy or gateway.
- LiteLLM
- An open-source library and proxy server that calls many model providers through one OpenAI-style interface. Source: LiteLLM docs. Our article: Evaluating a Claude Code to GPT route through LiteLLM.
- Amazon Bedrock
- AWS's managed service for calling foundation models, including Claude, from an AWS account. Source: AWS docs: What is Amazon Bedrock. Our article: Finding a hosted model's endpoint and deployment.
- Microsoft Foundry
- Microsoft's Azure platform for deploying models and calling them through endpoints in an Azure subscription. Source: Microsoft Learn: Foundry. Our article: Finding a hosted model's endpoint and deployment.
- OpenRouter
- A hosted service that gives one API key and an OpenAI-style API access to models from many providers. Source: OpenRouter docs.
- Container
- An isolated process with its own filesystem, started from an image. Running an agent in one limits what it can reach on the host. Source: Docker docs: What is a container. Our article: Running coding agents on the host and in containers.
- vLLM
- An open-source inference engine that serves models over an OpenAI-compatible API and can split one model across several GPUs. Source: vLLM docs.
Models and memory
- DGX Spark
- NVIDIA's desktop AI system built on the GB10 Grace Blackwell Superchip, with 128 GB of unified memory and ConnectX-7 networking. Source: NVIDIA DGX Spark. Our article: Planning memory for inference on one DGX Spark.
- Unified memory
- One memory pool shared by the CPU and GPU. On a DGX Spark, model weights, the KV cache and everything else draw from the same 128 GB. Source: NVIDIA DGX Spark docs. Our article: Planning memory for inference on one DGX Spark.
- Weights
- The learned numbers that make up a model. Their count and precision set most of the memory a model needs. Source: Hugging Face docs: quantization.
- Quantization
- Storing weights at lower precision, such as 8 or 4 bits instead of 16, to cut memory and speed up inference at some cost in accuracy. Source: Hugging Face docs: quantization.
- KV cache
- The attention keys and values a model keeps for every token already in context, so it does not recompute them. It grows with context length and with the number of requests served at once. Source: Kwon et al., PagedAttention (arXiv 2309.06180). Our article: What the KV cache is and how much memory it takes.
- Tensor
- A multi-dimensional matrix whose elements share one data type. A model's weights and the values it computes are stored as tensors. Source: PyTorch docs: tensors. Our article: Tensors and tensor parallelism.
- Pickle
- Python's built-in format for saving and restoring Python objects. Loading a pickle can import and call functions named in the file, so a pickle from an untrusted source can run code. PyTorch checkpoints use it by default. Source: Python docs: pickle. Our article: Model files are code.
- Safetensors
- A file format that stores tensors and a string-only metadata map: a size field, a JSON header with each tensor's type, shape and offsets, then the raw bytes. It was made to replace pickle for model weights, so loading a file does not run code. Source: Hugging Face docs: Safetensors. Our article: Model files are code.
Splitting a model across machines
- Tensor parallelism
- Splitting each layer's weight tensors across several GPUs so they compute every step together, exchanging partial results inside each layer. Source: Shoeybi et al., Megatron-LM (arXiv 1909.08053). Our article: Tensors and tensor parallelism.
- TP2
- Tensor parallelism across two devices. In this series, two DGX Sparks joined by a direct cable. Source: vLLM docs: parallelism and scaling. Our article: Planning a two-node DGX Spark TP2 setup.
- TP4
- Tensor parallelism across four devices. In this series, four DGX Sparks. Source: vLLM docs: parallelism and scaling. Our article: DGX Spark cabling for three nodes and four-device TP4.
- NCCL
- NVIDIA Collective Communications Library: it moves data between GPUs, inside one machine or across the network, for multi-GPU work. Source: NVIDIA NCCL user guide.
- RDMA
- Remote direct memory access: one machine reads or writes another machine's memory over the network without involving the remote CPU. Source: Wikipedia: Remote direct memory access.
- RoCE
- RDMA over Converged Ethernet: RDMA carried on Ethernet links. Source: NVIDIA networking docs: RoCE.
- ConnectX-7
- NVIDIA's network adapter family. The DGX Spark uses one for its high-speed ports. Source: NVIDIA ConnectX-7 user manual. Our article: DGX Spark cabling for three nodes and four-device TP4.