A per-request speculative.type of "none" still drafted 198 tokens

The symptom

Six configurations swept on a single server load — varying speculative.n_max and p_min, plus a "speculative.type": "none" arm as the baseline — came back within 2% of each other. That looked like a clean, if disappointing, result: speculative decoding making no measurable difference.

The none arm reported 198 drafted tokens.

The cause

The give-away was in the echoed settings the server returns:

speculative.types = none,draft-mtp

Plural. The per-request value appended to the launch-time list rather than replacing it. Every row in that sweep was the same configuration, and the arm labelled "off" was running MTP like all the others. The 2% spread was run-to-run noise across six identical arms.

The fix

Restart the server for the baseline. MTP=0 ./start.sh (or simply omitting --spec-type) is the only arm that is genuinely off, and it must be its own process.

Correctly measured, with a real restart between arms — UD-IQ4_XS, q8_0 KV, temp 0, 300 tokens, single stream:

workload	bare	MTP (n-max 3, p-min 0.75)	acceptance
code	27.75 tok/s	43.30 (+56%)	91.6% (197/215)
prose	28.28 tok/s	33.47 (+18%)	86.5% (154/178)

A 56% effect, invisible in the original sweep.

What to actually check

The failure here is not really about a flag. It is that the sweep had no arm capable of producing a different answer, and every symptom of that was consistent with a true negative result. Six rows agreeing looks like a robust finding. It is the same shape as a fixture that cannot exhibit the behaviour under test.

Two habits that would have caught it in seconds:

• Read the echoed settings on every arm, not just the first. The server told me exactly what it was running and I did not look until the numbers were already written up.
• Assert the baseline is inert. spec_decode_* counters at /metrics should be zero on the off arm. Non-zero drafted tokens with speculation "disabled" is a contradiction the test should fail on, not something a human has to notice.

Had I not checked the echo, I would have published a well-formatted table showing speculative decoding making no difference whatsoever. That is the third time this year I have shipped an instrument that agreed with me for structural reasons rather than empirical ones — the longer version of that argument is in my guardrail checked the config, not the corpus, and the full graft write-up is in I grafted a speculative decoding head into a 90 GB model file.

A related proxy failure on the same server

Same root cause, different surface. A script here checked whether an upstream multi-slot fix had landed by testing for its commit hash. The pull request was squash-merged, so none of the original SHAs exist in master, and the checker announced "not fixed, keep PARALLEL_SLOTS=1" against a master that plainly contained the fix.

An earlier version of the same script had tested whether a GGML_ASSERT had disappeared — but the assert was correct and deliberately kept, so that check would have reported "not fixed" forever.

Both failures share one cause: testing for a proxy instead of for the thing itself. Grepping the code for the changed identifier survives rebases, squashes, cherry-picks and reverts alike:

grep -c LLM_ARCH_QWEN4EXP src/llama-context.cpp        # expect 1
grep -c llama_mul_mat_hadamard src/models/qwen4exp.cpp # expect 4