LLM Quantization Testing: Impact of fp8 and fp4 on Quality and Price
The story so far
OpenRouter tested in 240 scoring tasks found that fp8, fp4, and fp16 showed no precision differences, with all six endpoint scores ranging from 98.3% to 99.2%. Low-precision quantization is generally cheaper, but in 7 models, fp8 was more expensive than fp16, and in 7 models, 4-bit was more expensive than 8-bit. fp4 performed the worst in 40 tasks without workflow multiplication, but the gap might be due to chance.
AI-written from the reports · Updated 8 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- OpenRouter BlogLLM Quantization: What fp8 and fp4 Do to Quality and Price
Testing on OpenRouter, fp8, fp4, and fp16 showed no precision differences in 240 scoring tasks, with all six endpoint scores ranging from 98.3% to 99.2%. Lower precision is generally cheaper, but there were 7 models where fp8 was more expensive than fp16, and 7 models where 4-bit was more expensive than 8-bit. The fp4 endpoint performed the worst among 40 workflows without multiplication tasks, but the difference might be due to chance.
Heat over time
- Now
- 3
- Peak
- 512 Nov, 06:00
- 24h change
- -49%
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.