A 200 Gb/s link rate between two DGX Sparks does not tell us by itself whether serving gets faster. Two DGX Sparks are cabled together, and we want them to serve a model as one instance. Whether that works depends on the second device's limits, how ranks and collective operations use the cable, and what the pinned community Qwen3.5-122B-FP8 recipe declares, and each of those has to be measured on its own. By the end, each question is paired with the measurement that answers it and the unit that measurement reports, and we match the recipe's dependencies before comparing measurements.
The cable gives us a physical connection between the devices. The serving runtime still has to distribute the model's computation across both devices and move the data it needs between them.
A second machine might improve response time.
In road terms, the physical links are the roads, the runtime ranks are the delivery addresses, and the communication algorithm is the delivery plan. The model implementation determines how its tensors are partitioned, and a switch can carry an algorithm called a ring even when the cables have a different physical layout.
We take this in five steps:
- Decide what the second device is for.
- Define which device holds which rank.
- Pin the recipe's versions and settings.
- Read network measurements in the units they report.
- Check what a fast link and a correct AllReduce prove.
Uses for the second device
Before assigning ranks, we decide which problem the second device is meant to solve.
NVIDIA's multi-Spark playbook covers connecting two to four systems for workloads that need more compute or memory than one device has, and its cabling and configuration pages cover general setup.
Three layouts put a second device to work:
- Tensor parallelism (TP) splits parts of one model's computation across a group of participants.
- Pipeline parallelism (PP) puts sequential model stages on different participants or groups.
- Independent replicas are separate complete serving instances, and each one may itself span several devices.
Which one we want depends on whether the problem is fitting one instance, serving more simultaneous requests, or hitting a latency target.
vLLM's scaling guide describes TP and PP for a single model replica, and its data-parallel deployment guide covers replicated weights for independent request batches. Some data-parallel and expert-parallel combinations for mixture-of-experts models synchronize across groups.
"Two copies" describes one possible data-parallel design.

Figure text
Two Spark devices, each with its own memory, joined by one high-speed link.
- Two machines, two memories, one cable
- Solid line: the one cable
- Two bars: two separate memories
- Two machines, two separate memories, one cable. The setup decides the rest, and none has been chosen here.
- Memory of Spark A: Separate memory budget
- Memory of Spark B: Separate memory budget
- Spark A: Separate device
- Spark B: Separate device
- Runtime setup: Decides where the computation runs
- Split one model's work; Put stages in sequence; Run separate complete copies
- One high-speed cable
- Nothing is chosen yet. Which option fits depends on the problem being solved.
Sources: [1] ConnectX-7 Networking - DGX Spark User Guide; [2] Connect Multiple DGX Sparks for Distributed Workloads | DGX Spark (connect multiple sparks).
Defining rank assignments
Once we know what we want the second device for, the runtime still needs to know which device plays which part in the distributed group.
A rank is a participant's identity within a distributed group. For an example TP2 placement, we assign rank 0 to one CUDA device on Spark A and rank 1 to one CUDA device on Spark B. We chose this mapping; two cables or two hostnames leave the rank assignment unspecified.
The NCCL Tests documentation gives a formula for the total rank count, MPI processes times threads per process times CUDA devices per thread. That formula counts the test's participants. A model server has its own launch and placement configuration, and its logs should show the mapping it actually used.
NCCL provides collective operations among those participants.
In AllReduce, every rank contributes values to a reduction and every rank gets the result.
In a toy sum where rank 0 contributes [1, 4] and rank 1 contributes [2, 3], both receive [3, 7] after element-wise addition.
![Spark A, rank 0 by our choice, and Spark B, rank 1 by our choice, are joined by one cable on the floor. Raised lanes, apart from the cable, lead from each machine to a mark between them labelled Add, position by position, and back. At the end each machine holds a block reading [3, 7].](/posts/images/dgx-spark-tp2-recipes-networking-f03-tp2-allreduce-end-frame-mobile.webp)
- Two devices, one cable: two Sparks, each with its own memory, joined by one direct cable.
- TP2 placement: tensor parallelism splits parts of one model's computation; we assign rank 0 to Spark A and rank 1 to Spark B.
- Every rank contributes: in AllReduce, every rank contributes values to a reduction; in the toy sum rank 0 contributes
[1, 4]and rank 1 contributes[2, 3]. - Every rank gets the result: both receive
[3, 7]after element-wise addition.
A line between ranks in a TP diagram can mean required communication without showing the actual packets on the physical network, so we check what each line means. The model implementation decides which collective operations it uses and where, and a model's layers may use collective operations besides AllReduce. Tensors and tensor parallelism runs a short script in which two halves of a weight matrix produce partial sums, which is the case that needs an AllReduce.

Figure text
A bidirectional cable connects two Spark nodes in the physical layer. In a separate logical layer, AllReduce takes input from each of two illustrative ranks and returns the same reduced result to every participant.
- The cable and the group are separate
- One member's numbers
- The shared result
- Dashed outline: not specified here
- The cable is connectivity. Rank placement and the route the software takes are separate questions, answered by the launch settings and the logs.
- 1 r0 puts in its numbers; r1 puts in its numbers
- 2 r0 gets the sum; r1 gets the sum
- Rank r0
- AllReduce: SUM
- Rank r1
- Which Spark, which rank?
- Spark A
- Spark B
- s = x₀ + x₁
- 1. Physical layer
- Port 0
- Nothing here says which machine is which member. That is set when the server starts, and its logs show it.
Sources: [1] ConnectX-7 Networking - DGX Spark User Guide; [2] Collective Operations - NCCL 2.32.3 documentation; [3] nccl tests; [4] Connect Multiple DGX Sparks for Distributed Workloads | DGX Spark (cable devices).
The recipe's versions and settings
Ranks and collectives are the runtime's concern; a recipe fixes which runtime, model and settings we actually launch, and that is what makes two runs comparable.
The community repository eugr/spark-vllm-docker has a Qwen3.5-122B-FP8 recipe at commit e2f445ef4ef3ed430a7776617270ddbc42cbfa32.
The file declares Qwen/Qwen3.5-122B-A10B-FP8, cluster-only operation, and a default tensor-parallel size of two.
An outside maintainer publishes the recipe, and it has no NVIDIA support statement.
The recipe's active modification list includes a Qwen3.5 chat-template fix, even though a nearby comment says no modifications are required. We use the active field when recording this recipe's dependencies. Its command template also picks a distributed execution backend and model-specific parsers, so swapping only the model identifier changes the combination we're evaluating.
We match the recipe's dependencies before comparing measurements, using the values pinned at that commit and their sources.
A community-maintained recipe.
Sources: pinned-tp2-recipe pinned-recipe-guide recipe-license.
| Recipe field | Pinned / declared value | Immutable source |
|---|---|---|
| Recipe identity | Qwen3.5-122B-FP8; recipe_version 1 | e2f445ef4ef3ed430a7776617270ddbc42cbfa32 pinned-tp2-recipe |
| Model | Qwen/Qwen3.5-122B-A10B-FP8 | pinned-tp2-recipe |
| Cluster / TP default | cluster_only: true; tensor_parallel: 2 | pinned-tp2-recipe pinned-recipe-guide |
| Container | vllm-node | pinned-tp2-recipe |
| Active modification | The active mods field lists mods/fix-qwen3.5-chat-template; the nearby “No mods required” comment conflicts with it. | pinned-tp2-recipe |
| Runtime command declarations | The command declares a Ray executor, instanttensor load format, qwen3_coder tool parser, qwen3 reasoning parser and unsloth.jinja template. | pinned-tp2-recipe |
| Resource / context defaults | gpu_memory_utilization: 0.8; max_model_len: 262144; max_num_batched_tokens: 8192. | pinned-tp2-recipe pinned-recipe-guide |
| License | MIT at the recipe commit. | recipe-license |
| Network | Listen defaults: host: 0.0.0.0, port: 8000 | Recipe default |
The pinned recipe guide explains that command values can come from defaults or command-line overrides, so the same commit can launch with different effective values.
vllm-node names the image, while a registry digest identifies the built bytes.
The repository license at the same commit is MIT.
To read the pinned values ourselves, we clone the repository, check out the commit, and search the recipe for its cluster, tensor-parallel and modification lines.
git clone https://github.com/eugr/spark-vllm-docker.git
cd spark-vllm-docker
git checkout e2f445ef4ef3ed430a7776617270ddbc42cbfa32
grep -n -E 'cluster_only|tensor_parallel|mods' recipes/qwen3.5-122b-fp8.yaml
In the file at that commit, the pattern matches six lines: cluster_only: true, the "No mods required" comment, the mods: key with its active mods/fix-qwen3.5-chat-template entry, the tensor_parallel: 2 default, and the -tp {tensor_parallel} slot in the command template that the default fills.
The repository README at the same commit gives the clone command, and the recipe guide's CLI reference lists --tp as the override for tensor parallelism.
Units for network measurements
With the recipe pinned, we read each link measurement in the unit it reports and ask what that measurement covers.
NVIDIA's ConnectX-7 guide describes two QSFP ports per Spark, each up to 200 Gb/s, with the cable affecting the negotiated link speed. The two-device cabling instructions call for a supported QSFP112 DAC (a direct-attach cable) in Ethernet mode and one direct cable. For NVIDIA's manual setup helper in the Configure Manually tab, we connect port 0 to port 0 or port 1 to port 1 to meet its interface requirements.
The reported link rate tells us the link's operating mode, and we measure useful workload bandwidth separately. NVIDIA's cluster troubleshooting table notes that a link can report 200 Gb/s while transfers stay slow, because traffic is going through a switch CPU or the link has errors.
Remote direct memory access (RDMA) moves data between device memories through the network adapter. Each measurement in the stack adds something:
- An RDMA measurement tests one transport path and one transfer pattern.
- A collective benchmark adds coordination across ranks.
- Model serving adds the model's computation and scheduling.
- The coding agent adds its client behavior and tool calls.
We pair each question with the measurement that answers it and the unit we expect that measurement to report.
Choose a link, microbenchmark or application result that can answer the question.
Write down the versions, scope, and workload next to each number.
Sources: nccl tests Connect Multiple DGX Sparks for Distributed Workloads | DGX Spark (troubleshooting) Serve LLMs with vLLM on One Device | DGX Spark (vllm).
| Question | What answers it | Measured as |
|---|---|---|
| Do the devices have the intended physical links? | Port identity, observed endpoints, cable compatibility, cable arrangement and negotiated link state | Link state and Gb/s |
| Does the configured network carry traffic? | Addressing, reachability, path, and error counters | Reachability and transfer result |
| Does RDMA work on the chosen path? | Versioned RDMA test with interfaces and message sizes, plus the transfer result under its recorded interface, transport and test conditions. Connect Multiple DGX Sparks for Distributed Workloads | DGX Spark (troubleshooting) | Transfer bandwidth and latency |
| Is a collective correct, and how fast is it? | NCCL test, rank mapping, sizes, correctness result and recorded participants / conditions for the tested collective. nccl tests | Collective bandwidth, latency, and errors |
| Does the model serve the target workload? | Results for the pinned model/runtime, effective serving settings and model requests. Serve LLMs with vLLM on One Device | DGX Spark (vllm) | Tokens/s, latency, and memory readings |
| Does the coding-agent task work? | Client version, agent request, bounded tool result and task evaluation under recorded conditions. | Task outcome and coding quality |
What a fast link and a correct AllReduce prove
A link-rate reading and an AllReduce result are the two easiest results to obtain.
A link reports 200 Gb/s and a two-rank AllReduce returns the expected sum. Suppose we test whether the pinned Qwen recipe serves a coding agent twice as fast as a single device. For that comparison, we check four things:
- the recipe's effective model and runtime configuration
- server behavior
- the client's tool loop
- a matched performance baseline
If the model exceeds the single-device baseline's capacity, we report the capacity difference and omit that point from the speedup curve.
What to do next
Pin the recipe at commit e2f445ef and match its dependencies before comparing speed. The recipe's listen default is every interface, so before the pair serves anyone else, bind its APIs to localhost or put authentication in front of them, for the reasons in Model servers are web apps.
What an attacker targets here
Two nodes serving one model mean two listeners and two places the weights are loaded, and an attacker needs only one of either.
- In ShellTorch, TorchServe's default
allowed_urlsaccepted every HTTP, HTTPS and local file URL, so an attacker could make the server download a malicious model and run it. We restrict what each server may fetch, as Model servers are web apps recommends. - Both nodes load the weights, and the Hugging Face Hub fetches the latest revision unless told otherwise. We pin the full commit hash on both nodes so they load the same files, as Model files are code explains.