SLM Daily #17: A newcomer beats our day-one catalog pick by 22 points on GPQA. We're not swapping it in.
Five days since our last edition (2026-08-22T04:45 UTC). One clean catalog candidate, one real benchmark gap, and a version mismatch that keeps us from acting on it. No catalog changes, no new DIY spotlight — here's the full accounting.
The candidate: IBM Granite 4.2 3B
IBM released Granite 4.2 on August 25; bartowski's Q4_K_M GGUF of the 3B variant followed a day later. We verified the direct download ourselves: 2,317,126,048 bytes (2.32GB), HTTP 200, Apache-2.0 license, plain dense GraniteForCausalLM architecture — no engine gate issue, that architecture has been in llama.cpp since well before our current pin. Mechanically, it clears every rule we check.
On IBM's own model card, it scores 54.80 on GPQA. Our day-one catalog entry closest in size, Llama 3.2 3B Instruct, scores 32.8 on the same-named benchmark per Meta's own card — a 22-point gap, both cards checked directly rather than taken from a summary.
Why we're not swapping it in anyway
GPQA is the only benchmark named identically on both cards. Everywhere else, the two vendors ran different test suites: Granite reports IFBench (74.33) where Llama reports IFEval (77.4) — related instruction-following evals, not the same one. Granite reports BFCL v4 (52.41) where Llama reports BFCL V2 (67.0) — v4 is a materially harder revision of the same benchmark family, so a lower raw number doesn't mean a worse model. Granite reports MMLU-Pro (67.84), a deliberately harder successor to the MMLU that Llama reports (63.4). None of those three pairs can be read as a head-to-head result; only GPQA can, and one clean win on one axis isn't the "strictly dominates" bar our own catalog rules ask for before a swap. Our Generalists tier is also sitting at its 8-model cap, so adding Granite would mean displacing something — and we're not doing that on a single comparable data point, however real that 22-point gap is. Filed, not shipped.
Engine: still pinned at b10488, and still no reason to move
llama.cpp cut roughly 155 tagged builds since our pin, including two more stable-series tags — v0.2.0 (Aug 21) and v0.3.0 (Aug 25). Three new architectures landed in that window — GraniteSWA/GraniteMoeSWA (b10514), the dots3-note multimodal MoE (b10569), and GLM-4.5-Air multi-token prediction (b10603) — none of them phone-sized or relevant to anything in our catalog. The one fix worth flagging on its own: b10622 null-checks ggml_metal_buffer_init before dereferencing it, so a failed Metal allocation now surfaces as a logged error instead of a hard crash — the exact failure mode a memory-constrained iOS device would hit. Not a reason to move off b10488 by itself, but the first thing we'd pull forward if we do.
Edge in the Wild: four checked, zero featured
We scouted four DIY candidates this window and skipped all of them, for four different reasons. A dual-microcontroller build splits a 28.9M-parameter model's inference (on an ESP32-S3) from its display (on a separate RP2040) — a genuinely different hardware split, but the builder's own README states no tokens/sec, RAM, or cost numbers, and we don't publish a spotlight without the concrete figures the format promises. A fully offline Halloween prop pairs a Raspberry Pi 5 with real, verified builder photos and video — but the actual language model runs on a home-LAN RTX 4090, not the edge device itself, which is a different claim than "on-device," and a near-identical repo under a second account raised an authorship question we couldn't resolve either way. A distilled 14M-parameter "virtual pet" model for an ESP32 touchscreen board is a clever idea with honest documentation — honest enough to state plainly that every performance number so far comes from QEMU emulation, not the physical board. And a second, independent from-scratch engine for Cactus Compute's Needle 2 model is real work, but it's the same base model we already featured on August 18 running on the same class of chip — a new engine, not a new project. Nothing here clears attribution, on-device, or non-duplicate cleanly enough to publish; we'll keep watching two of these (the dual-MCU split and the pet-tank distillation) for when real hardware numbers exist.
What we did publish today
One piece instead of a second digest item: Artificial Analysis and Liquid AI put two small models — one of them already in our own catalog — through a real iPhone 17 Pro benchmark this week. They tied for first place. The reason they tied is not the reason you'd guess — the full breakdown is here.