SLM Daily #9: a Metal bug quietly returned the wrong answer on Apple's own GPU
A Metal shader bug that could silently corrupt a model's own normalization math — on any hidden or head dimension that isn't a multiple of the GPU's SIMD width — got fixed in llama.cpp on August 7, one day after our last check. It's the only phone-relevant change in 33 build tags over two days, and it lands directly on the Metal backend our own xcframework build uses. Nothing else this edition needed our attention: zero catalog additions, seven candidates checked and rejected.
The bug: wrong math, not a crash
Tag b10331 is the current latest, up from b10298 when we last checked. The one fix worth flagging is PR #26708, merged August 7: ggml_metal_op_norm sized its threadgroup as min(nth, ne00_t), which could leave the thread count not a multiple of Metal's SIMD width. The cross-simdgroup reduction that follows then silently dropped part of the partial-sum tail — producing a wrong mean and variance for the whole row on any model whose hidden or head dimension didn't happen to divide evenly. Not a crash, not an error message. A quietly wrong number, on Apple GPUs specifically, for however long that build was live. Since our engine is a Metal-backed xcframework fetched fresh from "latest" at every build (see our July 13 post on exactly this non-determinism), whether any specific install was affected depends entirely on when it happened to build — which nothing in the app currently records.
Everything else in the window is server/WebUI tooling (Docker-based sandboxing, chat-form slash commands, a model-scheduler LRU policy) or SYCL backend catch-up work for Intel Arc GPUs, including new ops for DeepSeek V4 — an architecture that was already merged well before this window, just getting broader backend coverage now. We diffed src/llama-arch.h between b10298 and b10331 directly: no new LLM_ARCH_* tag landed. LFM2, LFM2MoE, the full Qwen3 family, Gemma3/3n/4, and DeepSeek V4 are all unchanged and all still supported.
What we checked and rejected for the catalog
- LFM2.5-1.2B-Thinking (Liquid AI) — real, reasoning-tuned, but the base release dates to January 20, 2026. Not new, just newly noticed.
- Qwen3.6-27B / 35B-A3B — both far over our size cap at any quant; no sub-4B Qwen3.6 variant exists yet.
- Nemotron 3 Nano Omni (30B-A3B) — Nvidia's new omni-modal model has GGUF quants, but the file itself runs tens of GB even with only 3B active parameters. Wrong shape for a phone.
- Inkling Small (Thinking Machines) — already on our rejected list; still a 276B-total MoE, not a phone candidate at any quant.
- Granite 4.0 Nano (IBM) — genuinely small and Apache-2.0, but the release is from October 2025. Nearly ten months old, not "new."
- Tiny Aya (Cohere Labs) — 3.35B, GGUF quants exist near our size cap, but the license is CC-BY-NC (non-commercial), which our catalog rules exclude outright.
- Mistral Small 4 — released March 2026, and "Small" for Mistral still runs well above our 2.5 GB cap at any quant we could find.
The newest actual uploads on Hugging Face's GGUF "created" sort this week were, almost without exception, anonymous community finetunes and merges — not reputable-quantizer originals of a new base model, and not something we'd verify well enough to put in front of users.
Also this edition
Two other pieces published today. Our deep-dive checks the "diffusion LLMs will make on-device AI fast" pitch against the only real phone-NPU benchmark we could find, and the accuracy cost of the shortcut that makes it work at all — read it here. And an Edge in the Wild spotlight on a builder who got PrismML's Bonsai 27B — the model we said in July had no GGUF path — running on a Jetson Orin NX at 6.76 tok/s, via a route that still doesn't touch mainline llama.cpp — read it here.
The catalog now
Still 22 models, unchanged since SLM Daily #8. Same standing caveat: we hold ourselves to a cap of 8 generalist entries, and 14 of the 22 are pre-cap specialist models (medical, legal, finance and similar) we haven't unilaterally pulled from a catalog people are actively downloading from. Still flagging it rather than letting it go quiet.