SLM Daily #10: one skipped check nearly doubled a model's memory use — and broke its answers
1,952 MiB versus 4,160 MiB — that's the KV cache a real, currently-shipping model used before and after a one-line ordering fix landed in llama.cpp on August 11. The bug didn't just waste memory: while it was live, affected builds produced what the fix's own author calls "nonsensical output generation." It's the only phone-relevant change worth flagging since our last check. Everything else this edition is routine: zero catalog additions, six candidates checked and rejected.
The bug: a check ran before the number it depends on was read
The current latest tag is b10362 (August 11), up from b10331 when Daily #9 last checked. The fix that matters is PR #26848, "fix SWA not being enabled for EXAONE 4.5," merged the same day by ggerganov. The model-loading code checked hparams.n_layer() == 64 to decide whether sliding-window attention (SWA) should turn on — but n_layer() returns n_layer_all - n_layer_nextn, and at the point that check ran, n_layer_nextn hadn't been read from the GGUF yet, so it still held its default of zero. Any EXAONE 4.5 GGUF with a multi-token-prediction (MTP) head — block_count 65, nextn 1, exactly what LG's official release ships — evaluated n_layer() as 65 at that moment, missed the == 64 check, and silently skipped the entire SWA code path in favor of dense attention over the full context.
The visible symptom wasn't a crash. llama_model_n_swa() still correctly reported a 4096-token window; the internal swa_type just silently stayed LLAMA_SWA_TYPE_NONE. Dense attention over the full context that SWA was supposed to cap is exactly why the KV cache measured 4160 MiB instead of the expected ~1952 MiB — a real 2.1x, not a rounding difference — and it's also the direct cause of the broken output: EXAONE 4.5 was designed and trained expecting a bounded attention window, and giving it unbounded dense attention instead measurably changed what it produced. EXAONE 4.0 was never affected — it has no MTP head, so its block_count is 64 and the check passed correctly.
We diffed src/llama-arch.h against our last check: no new LLM_ARCH_* tag beyond LLM_ARCH_MUSE_GLIMMER, added this week to support Meta's new release (more on that below). LFM2, LFM2MoE, the Qwen3 family, and Gemma3/3n/4 are all unchanged and still fine.
What we checked and rejected for the catalog
- Muse Glimmer 30B (Meta, released August 10, Apache-2.0) — llama.cpp shipped native architecture support within days, which is a genuinely fast turnaround. Doesn't change the verdict: even the smallest official 4-bit quants run 17–29.6 GB, nowhere near our 2.5 GB phone cap.
- NVIDIA Nemotron-3.5-Lightning-30B-A3B — a mixture-of-experts model with only ~3B active parameters per token, but the full GGUF still has to store all the experts. File size lands in the ~30 GB class at Q4 regardless of how few parameters fire per token.
- SpatialAxiom-9B and SpatialAxiom-35B-A3B — both over our cap at every quant we found.
- EXAONE 4.5, 1.2B tier — small enough to fit easily, but LG's license explicitly prohibits redistribution and commercial use without a separate agreement. Rejected on license, not size.
- webAI-Official/TwIL-LM3 (3B) — fails on two independent grounds: a non-commercial license, and the model card's own admission that it "lacks instruction-following alignment work," meaning it isn't a chat/instruct model at all.
One note, not a rejection: LFM2.5-2.6B (Liquid AI) is real and genuinely dropped in-window, but it's already been in our catalog since we added it in July — not new to us, just newly re-surfaced in trending lists.
Also published today
Two other pieces. Our deep-dive looks at what the EU's new AI-content transparency law — enforceable since August 2 — actually requires of a text-generating AI system, and why nothing in the open-weight GGUF ecosystem, including our own engine, has any mechanism to satisfy it — read it here. And an Edge in the Wild spotlight on an independent builder who got an 80-billion-parameter model running in 4.3 GB of RAM on an unmodified Mac by streaming its weights from disk instead of loading them — read it here.
The catalog now
Still 22 models, unchanged since Daily #9. Same standing note: we cap ourselves at 8 generalist entries, and 14 of the 22 are pre-cap specialist models (medical, legal, finance and similar) that we haven't unilaterally pulled from a catalog people are actively downloading from.