What is an SLM? Small language models, explained
Everyone knows the acronym LLM by now. The models behind ChatGPT, Claude and Gemini are large language models — hundreds of billions of parameters, running in data centers, reachable only through an internet connection and someone else's servers.
An SLM — a small language model — is the same technology at a radically different scale: roughly 1 to 3 billion parameters, compressed to a few hundred megabytes or a couple of gigabytes. Small enough to fit in your phone's memory. Fast enough to answer on the chip in your pocket.
Small doesn't mean dumb
The surprising story of the past two years is how much capability survived the shrinking. Modern SLMs — Llama 3.2 1B, Qwen2.5 1.5B, Gemma 2 2B, Phi-3 Mini, Apple's ~3B on-device model — handle the tasks most people actually use AI for:
- Answering everyday questions and explaining concepts
- Drafting and rewriting messages, emails and notes
- Summarizing text, brainstorming ideas, translating
- Casual conversation with real context memory
Three things made this possible: better training data (quality beats quantity at small scale), distillation from larger teacher models, and aggressive quantization — storing weights in 4 bits instead of 16 with minimal quality loss.
Why run a model on the device at all?
Privacy, by physics rather than by promise. A cloud provider says "we don't store your conversations." An on-device model can't send them anywhere in the first place — turn on airplane mode and it keeps working. For anything sensitive — health, money, work, feelings — that's a different kind of guarantee.
Latency and availability. No round trip to a server means answers start instantly, on a plane, in the subway, on a hike.
Cost. Inference on your own chip is free. No subscription meter running while you think.
Where SLMs still lose
Honesty matters: a 1–3B model will not out-reason a frontier model. Long chains of logic, obscure facts, complex code — the big models are still clearly better. The right mental model is a very capable assistant, not an oracle. For a large share of everyday tasks, that's exactly enough.
The platform shift is already here
Apple ships a ~3B model inside iOS and macOS and lets apps call it directly through the Foundation Models framework. Google ships Gemini Nano inside Android via AICore. The operating systems themselves have decided: small models belong on the device.
The model comes to your data — instead of your data going to the model.
That's the idea privateSLM is built on: use the system model when your device has one, or download an open SLM and run it locally. Either way, your conversations stay yours.