Skip to content
Live
Trending storyPast story

LLM Quantization Testing: Impact of fp8 and fp4 on Quality and Price

1 reports1 outlets8 days agoUpdate

The story so far

AI overview

OpenRouter tested in 240 scoring tasks found that fp8, fp4, and fp16 showed no precision differences, with all six endpoint scores ranging from 98.3% to 99.2%. Low-precision quantization is generally cheaper, but in 7 models, fp8 was more expensive than fp16, and in 7 models, 4-bit was more expensive than 8-bit. fp4 performed the worst in 40 tasks without workflow multiplication, but the gap might be due to chance.

AI-written from the reports · Updated 8 days ago

Coverage timeline

Follow the coverage to see every side of the story.

11 Nov
  1. OpenRouter Blog
    LLM Quantization: What fp8 and fp4 Do to Quality and Price

    Testing on OpenRouter, fp8, fp4, and fp16 showed no precision differences in 240 scoring tasks, with all six endpoint scores ranging from 98.3% to 99.2%. Lower precision is generally cheaper, but there were 7 models where fp8 was more expensive than fp16, and 7 models where 4-bit was more expensive than 8-bit. The fp4 endpoint performed the worst among 40 workflows without multiplication tasks, but the difference might be due to chance.

Heat over time

Now
3
Peak
512 Nov, 06:00
24h change
-49%

Hover or tap the chart to see each hour; use the arrow keys on a keyboard.