Field note · llama.cpp
Dropping --kv-unified silently turns a 262144-token server into a 65536-token one
llama-server divides the context size by the slot count unless --kv-unified is set: n_ctx_seq = n_ctx / n_seq_max. There is no error and no warning. The same command line that asks for --ctx-size 262144 --parallel 4 serves 65536 per sequence, and the only evidence is one line in the boot log. Long prompts then fail or shift in ways that look like a model problem rather than a configuration one.
- Hardware
- NVIDIA DGX Spark · GB10 (SM121) · aarch64 · 128 GB unified
- Software
- llama.cpp master · unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS · llama-server, 4 slots
The silent division
src/llama-context.cpp:
if (cparams.kv_unified) { n_ctx_seq = n_ctx; } // 262144 per slot
else { n_ctx_seq = n_ctx / n_seq_max; } // 65536 per slot--ctx-size is the size of the whole pool, not the size available to a request. Without --kv-unified, four slots each get a quarter of it. Nothing warns you. The flag you did not pass is the flag that would have kept the number you did pass.
The failure this produces is indirect, which is what makes it expensive: prompts that fit comfortably in the context you asked for start truncating or shifting, and the first suspicion lands on the model or the client rather than on a divide you never saw.
Confirming it
There is exactly one line of evidence, in the boot log:
init: initializing, n_slots = 4, n_ctx_slot = 262144, kv_unified = 'true'n_ctx_slot is the number that matters. If it is n_ctx / n_slots, unified mode is off. Grep for it on every start; it costs nothing and it is the only place the truth is printed.
The caveat that comes with the fix
--kv-unified is not free capacity. Four slots at 262144 is a scheduling ceiling, not a guarantee — unified mode allocates one shared pool, so four simultaneous conversations all sitting near full context would oversubscribe it. What you get is the ability for any single request to use the whole context, not the ability for all four to do so at once.
For a prefill-bound workload that is the right trade anyway, and on this host the slot count went to 1 for an unrelated and larger reason: four slots scattered requests across slots by LRU and collapsed prefix-cache reuse from 86.7% to 1.2%. That story is in I turned off concurrency and the server got faster.
Related: the binary outlives the checkout
A neighbouring silent-configuration trap on the same server, worth the same defensive habit. build/ survives a git checkout, so switching branches and forgetting to rebuild leaves a binary from the previous branch — and the only runtime evidence is an unknown-architecture abort that never names the branch you needed. Probe the binary, not the source tree:
LIB_HITS=$(strings "$LIB" 2>/dev/null | grep -c "qwen4exp" || true)
(( LIB_HITS == 0 )) && { echo "stock master build — wrong binary"; exit 1; }Use grep -c, not grep -q. Under set -o pipefail, grep -q exits on its first match, which SIGPIPEs strings (exit 141), and pipefail then reports a successful match as a failed pipeline — rejecting a perfectly good binary. -c consumes all of its input, so there is no early exit to signal.
