NVIDIA breaks through AI memory bottlenecks: HBM only increased by about 3%, with the maximum improvement in agent throughput per dollar up to 30 times.

NVIDIA proposed 'Full-Stack Co-Design' to break through AI memory bottlenecks. NVFP4 compresses model weights and KV cache, NVL72 distributes weights across 72 GPUs, Vera CPU provides buffering, Dynamo software migrates context, and RAG indices remain on the CPU and storage side.