Question the measurement
My LLM judge was flipping a coin a third of the time
What happened when I tested the judge I had trusted for months.
The notebook
Writing
Small thoughts, things I learned the hard way, and ideas that needed more room.
Follow via RSS — every new thought, field note, and essay in one feed.
A few places to start
Follow a fix
Every FP8 GEMM traps on DGX Spark (SM121)
An exact failure, the hardware behind it, and the two-line fix.
Look at the craft
A page transition that breathes
The engineering behind a quieter page transition on this site.
Browse the notebook
5 pieces about Agents & tooling
Field note
Dropping --kv-unified silently turns a 262144-token server into a 65536-token one
llama-server divides the context size by the slot count unless --kv-unified is set: n_ctx_seq = n_ctx / n_seq_max. There is no error and no warning. The same command line that asks for --ctx-size 262144 --parallel 4 serves 65536 per sequence, and the only evidence is one line in the boot log. Long prompts then fail or shift in ways that look like a model problem rather than a configuration one.
Field note
A pre-flight probe reported an exhausted thinking budget as an unconstrained grammar
An engine-swap safety check asked a reasoning model for a JSON verdict inside a 40-token budget. Reasoning consumed the whole budget, content came back empty, and the probe scored that as a schema that does not bind. The grammar was fine. A three-arm test isolates budget from grammar to show why.
Field note
A reasoning parser with no start/end strings silently disables thinking-budget enforcement
Muse-Glimmer delimits reasoning by channel markers rather than a token pair, so vLLM cannot resolve reasoning token ids, so reasoning_config.enabled stays false, so the budget cap never engages — on a model whose chat template reasons by default.
Field note
Muse-Glimmer returns empty reasoning_content while the parser demonstrably extracts it
Every completion reports reasoning_content: "" and reasoning_tokens: 0, even when a hundred tokens of reasoning were generated. The parser returns the reasoning when called directly. The defect is somewhere between the two, and I have not found it.
Field note
Structured output stalls forever once you register 28 tools
vLLM builds a Lark grammar enumerating every registered tool as an alternative. Past roughly 28 tools the guidance parser gives up, and the model generates tokens that never reach the client.