Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models

Brief overview

T²Quant framework predicts accuracy sensitivity through minimal unlabeled calibration prompts, allocating activation precision within a fixed bit-width budget, improving existing quantization methods by 6.7 percentage points across 3 diffusion language models and 6 benchmark tests without increasing average accuracy budget.

Article

Excited to share our new paper: Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models! 🚀

We reveal that quantization sensitivity is both task and time-dependent, with different downstream tasks exhibiting distinct precision requirements across denoising steps.

Building on this finding, we introduce T²Quant, a mixed-precision framework that:

🔍 Predicts precision sensitivity using only a few unlabeled calibration prompts.
⚙️ Allocates activation precision across denoising steps under a fixed bit-width budget.
📈 Improves existing quantization methods by up to 6.7 percentage points, without increasing the average precision budget.

Evaluated across 3 diffusion language models and 6 benchmarks.

📄 Paper: https://arxiv.org/abs/2610.31198

We welcome feedback, questions, and discussions!