QbitAI·· 19 days agoFeatured78
Rockchip Matrix, in collaboration with CUHK and CMU, has open-sourced EdgeRock-4B, reducing KV Cache by 99.1%.
Brief overview
Rockchip Technology's Rockchip Matrix, in collaboration with CUHK and CMU, has released and open-sourced EdgeRock-4B, with a fixed KV Cache of 72MiB under 256K context, reducing the source model Qwen3.5-4B by 99.1%.
Why it matters
The original provides a comparison of throughput under fixed memory, which can be used to judge the trend of GPU memory costs for long-context reasoning.
Source: QbitAI · qbitai.com