Google achieves stable reinforcement learning with per-voxel consistency on TPUs
The story so far
Google implemented the Zero-TIM (Zero Training-Inference Mismatch) scheme in the open-source JAX library Tunix, making the sampler and trainer produce identical logprob per digit for the same action and token history under the same weights, thus achieving true on-policy stability in TPU reinforcement learning training. The scheme has been validated on a multi-node scale of 64 TPU v5p (DP8×TP8). Google claims that the related kernels and collective communication patterns can serve as reusable blueprints for teams to implement bit-wise consistent on-policy RL on distributed JAX/TPU stacks. Currently, the scheme remains a prototype in Tunix, and Google states it is expanding support for linear attention and mixed expert (MoE) architectures.
AI-written from the reports · Updated 9 hours ago
Coverage timeline
Follow the coverage to see every side of the story.
- Google Developers BlogFeaturedGoogle achieves stable on-policy RL with bit-wise consistent training and inference on TPU
Google implemented Zero-TIM in Tunix to ensure that samplers and trainers provide the same logprob for the same token at the same weight, and validated it on a multi-host scale of 64 TPU v5p (DP8×TP8).
Heat over time
- Now
- 5
- Peak
- 617 Nov, 01:00
- 24h change
- –
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.