Skip to content
Live
NVIDIA Technical Blog· Zhihan Jiang·· 9 hours ago65

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0

Brief overview

NVIDIA Vera Rubin NVL72 performed best in MLPerf Endpoints v1.0 testing, supporting agent inference for 8k context lengths, with a 3.5x speedup and a price of $3.5 per million tokens.

Article

Two rows of black data center racks with gold-colored server components against a black background.Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool...Two rows of black data center racks with gold-colored server components against a black background.

Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool calls, and responses. Inference systems must support users at scale while remaining responsive. MLPerf Endpoints v1.0 evaluates this balance between serving capacity and responsiveness using coding and enterprise workflows through API…

Source

Source: NVIDIA Technical Blog · developer.nvidia.com