Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1
Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1
NVIDIA TensorRT schedules interdependent decisions such as graph-IR conversion, fusion, implementation selection, memory layout, and quantization when deploying trained neural networks. The Global Performance Tuner (GPT) enables TensorRT performance optimization beyond a single build path.
Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For...
Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For example, when NVIDIA TensorRT builds an engine, it makes interdependent decisions about graph-IR transformations, fusion, implementation selection, memory placement, and quantization. This process is called scheduling.
Source: NVIDIA Technical Blog · developer.nvidia.com