Full Precision vs Quantized LLMs
Quantization compresses LLM weights to use less memory and run faster — but how much quality do you lose?
Full Precision vs Quantized
The same model, two different representations
Standard Weights
Each weight stored as a full 32-bit floating-point number, preserving maximum precision and accuracy.
Compressed Weights
Weights are mapped to fewer bits, dramatically reducing memory usage while maintaining acceptable quality.
Weight Representation: Same Model, Different Sizes
Model Size by Quantization Level
Based on a 7-billion parameter model
Quantization Methods
Different approaches to compressing model weights
FP16 / BF16
Half-precision or brain-float 16. Simple reduction from FP32 with minimal quality loss. Widely supported.
Near LosslessINT8 Quantization
Maps weights to 256 discrete levels. 4× compression from FP32 with minimal quality degradation.
Minimal LossINT4 Quantization
Maps weights to 16 discrete levels. 8× compression. Some quality loss but still highly usable.
Moderate LossGPTQ / AWQ
Advanced methods that calibrate quantization per-layer using a small dataset. Better quality at same bitwidth.
Smart LossQuality vs Compression Trade-off
Try It: Watch Precision Disappear
The curve is a slice of a weight tensor. Drag the slider to round it onto fewer and fewer levels, and watch where the error actually shows up.