Michael Hospedales

Software engineer / Miami, FL

The notebook

Writing

Small thoughts, things I learned the hard way, and ideas that needed more room.

Follow via RSS — every new thought, field note, and essay in one feed.

A few places to start

Browse the notebook

Clear filters

9 pieces about Linux & hardware

Field note

pgrep -f matched my own shell, and nohup did not survive the timeout

Two process-management traps that both look like a crashed inference server. pgrep -f matches the command line of the process doing the grepping, so a stop script can kill its own shell and a wait loop can "confirm" a server stopped when it did not. And nohup guards against SIGHUP, not the SIGTERM a timing-out parent sends to its process group — so a 90-second model load inside a 2-minute command budget is killed at exactly two minutes and looks identical to a load failure. Have the child write its own PID, and use setsid.

Field note

DefaultOOMScoreAdjust=200 in the systemd user manager kills terminal-launched servers first

A production llama-server process was killed by the kernel with nothing in its own log: no assert, no stack trace, just silence, noticed only because /metrics stopped responding. The killed process carried oom_score_adj:200. The cause was not the coding-agent session that launched it — it was a systemd user-manager default, DefaultOOMScoreAdjust=200, which scores every terminal-launched process to die before system services do, even ones many times smaller. That score cannot be lowered after launch by an unprivileged process, so the fix is to run the server as a systemd system service instead, which defaults to OOMScoreAdjust=0.

Field note

Detecting which inference engine owns a port: vLLM vs. llama.cpp

Detection that infers the engine from the port number breaks the moment either engine moves. Moving llama.cpp onto the port normally used by vLLM mislabeled it, and downstream code silently wrote null. /props is a clean discriminator: llama-server serves it, and the vLLM OpenAI-compatible server has no such route and returns 404. A second gotcha in the same detour: docker ps --filter publish= does not match a container using host networking.

Field note

RSS is the wrong instrument for memory on a DGX Spark

The llama-server process shows about 29 GiB of RSS while about 87 GiB of weights for Qwen3.8-Flash-Next sit in CUDA buffers that RSS never counts, and nvidia-smi reports [N/A] for memory on GB10 because there is no separate GPU pool to report. MemAvailable before and after start is the honest measurement, and that holds for every model on a DGX Spark, not just this one. The idle headroom measured at launch was not production headroom either: the same configuration was later killed by the kernel under real load, with nothing in its own log, and working defaults moved to a smaller context size and ubatch afterward.

Field note

Every FP8 GEMM traps on DGX Spark (SM121)

vLLM guards its FP8 CUTLASS kernels on __CUDA_ARCH__ == 1200 and executes a deliberate trap instruction on anything else. DGX Spark is arch 1210, so every FP8 GEMM crashes. Two lines fix it.