Quantisation

Reducing numeric precision of a model’s weights or activations to lower size, memory use and speed up inference with minor accuracy trade-offs.

Also known as: Quantization, INT8, Post-training quantisation, Quantisation-aware training

In plain English

Quantisation is like saving a photo in a smaller file so it opens faster and takes less space. For AI models, it means using simpler numbers to store the model so it can run on phones or cheaper servers. The model usually still works but may lose a little precision.

In practice

Quantisation matters when deploying or buying AI tools because it reduces model size, memory footprint and latency, enabling edge or cost-effective cloud inference. You often choose post-training quantisation for quick savings or quantisation-aware training when accuracy is critical. Check hardware support (for example INT8 kernels) and validate task-specific accuracy before production rollout.

Under the hood

Quantisation maps floating-point weights and activations (e.g. float32) to lower-bit representations (e.g. int8, int4) using scale and zero-point parameters; options include per-tensor or per-channel schemes. Typical workflows are post-training quantisation (PTQ) with calibration or quantisation-aware training (QAT) that simulates low-precision during training to preserve accuracy. Performance gains depend on hardware support (CPU/GPU/accelerator integer kernels) and the chosen quantisation granularity and range calibration.

Example

"Let's quantise the model to INT8 and run benchmark tests on the edge device to check latency and accuracy before deployment."

•