Small Language Model

A compact neural language model with fewer parameters for lower compute, faster inference and easier deployment on limited hardware.

Also known as: SLM, small LLM, compact model, edge model

In plain English

A small language model is like a pared-down smart assistant that understands and writes text but with simpler abilities. It can run on phones or local servers and usually costs much less to operate than large cloud models.

In practice

When buying AI tools, organisations choose small language models to reduce cloud costs, cut latency and keep data on-device for privacy. They offer faster inference and cheaper scaling but typically trade off some accuracy and breadth of capability compared with larger models.

Under the hood

Small language models use the same Transformer-style architectures but with substantially fewer parameters and often shorter context windows, reducing FLOPs, memory and latency. They are commonly produced by techniques like distillation, pruning and aggressive quantization to enable efficient on-device or low-cost inference while retaining task-relevant behaviour.

Example

"Let's prototype a small language model on-device so we can reduce inference latency and avoid sending user text to the cloud."

Related terms

Parameters · Inference · Model Distillation

•