qwen4exp is not qwen3next: the August 26 llama.cpp loading failure

125B at 22–27 tok/s, with no drafter at all

Qwen3.8-Flash-Next is a 125B-parameter, 6B-activated MoE — the model card also lists a 51B n-gram embedding table and a 4B MTP head layered on top. On this host, with an f16 KV cache and no speculative decoding of any kind, it decodes at 22.47 tok/s cold and 26.90 tok/s warm (250 tokens, same prompt both times). That's close to what this host's Qwen3.8-27B gets on vLLM INT4 with a DFlash2 drafter attached (~25.5 tok/s). A 125B model matching a drafted 27B model's throughput with no drafter at all is the reason the rest of this note is worth the trouble — 6B active parameters, plus an embedding table that costs memory rather than compute, doing exactly what the architecture promises.

Model load took 85–90 seconds for the 93.6 GB, three-shard file. (Prefill numbers of 35–77 tok/s came out of the same run, but from prompts of only 17–53 tokens — too short to mean anything. Treat them as noise, not a benchmark. It is not, though, evidence that prefill is a non-issue at real context lengths: later measurement on a ~25.6K-token prompt showed marginal prefill throughput more than halving by the end of it, and with prefix-cache reuse at only ~1.2%, a long-prompt call spent close to 45% of its wall-clock before emitting a single token. Decode is the number in the headline above; prefill is the cost that actually degrades on long prompts.)

Getting to that number at all means solving an architecture-detection problem first.

qwen4exp is not qwen3next

The GGUF is upfront about what it is, straight out of its own header:

general.architecture              = 'qwen4exp'
general.description               = 'A Preview of the Qwen4 Architecture'
general.size_label                = '512x56B'

Not qwen3next. It shares a version number with Qwen3.8-27B and almost nothing else — the 27B is arch qwen35, dense-hybrid, with its own embedded MTP head and a vision projector, and none of that configuration transfers here.

llama.cpp master has never heard of qwen4exp:

$ grep -rc "qwen4exp" src/*.cpp src/*.h
(zero hits)

Point stock llama-server at these weights and it dies on unknown-architecture. It doesn't tell you a branch exists, let alone which one.

PR #27742 is mandatory

The model card links PR #27742 as the only way to run these weights outside Unsloth's own desktop app. Open and mergeable as I write this, at +2560/-9 across 21 files:

git fetch origin pull/27742/head:pr-27742
git checkout -b qwen4exp-27742 master
git merge pr-27742        # clean
./build.sh                # CUDA_ARCH=121a-real

It touches no common/arg.cpp and no tools/server/, so it adds no new CLI flags — everything downstream is stock llama-server.

The trap: build/ outlives git checkout

build/ isn't part of the source tree git checkout operates on, so checking out the right branch and forgetting to rebuild leaves a binary compiled from whichever branch was checked out before — and the only runtime symptom is the same unknown-architecture abort you'd get from stock master. Source tree and binary can disagree silently, and nothing about the abort tells you which one is lying.

The fix is to stop trusting the source tree and probe the binary instead:

LIB_HITS=$(strings "$LIB" 2>/dev/null | grep -c "qwen4exp" || true)
(( LIB_HITS == 0 )) && { echo "stock master build — wrong binary"; exit 1; }

grep -c, not grep -q — and the difference isn't stylistic. This preflight runs under set -o pipefail. grep -q returns as soon as it finds one match: it doesn't drain the rest of strings' output, it just exits. strings is still writing to that pipe when grep -q closes its end, so the next write lands on a closed pipe and strings is killed by SIGPIPE, exiting 141. grep -q itself exited 0 — it found exactly what it was looking for — but under pipefail, the pipeline's reported status is whichever command actually exited non-zero, and that's strings' 141, not grep's 0. The pipeline reports failure for a search that actually succeeded, and the preflight rejects a perfectly good, correctly patched binary. grep -c counts every match instead of stopping at the first one, so strings always finishes writing and always exits cleanly — the pipeline's status ends up reflecting the real answer instead of an artifact of how grep happened to stop reading.

Once it's running

Two limitations showed up in my August 26 setup: a quantized KV cache crashed this architecture — see A quantized KV cache crashes qwen4exp on llama.cpp — and the advertised 4B MTP head was not available for speculative decoding in that build — see the original MTP investigation.

By August 28, I had working speculative decoding using a third-party head, the graph from draft PR #27836, and a local GGUF graft. The follow-up essay documents that setup and its caveats.

When #27742 merges, this whole branch dance goes away: checkout master, pull, rebuild, drop qwen4exp-27742. None of the flags above change — they're all upstream's, not something this workaround added.