Skip to content
Live
Trending storyPast story

Deepgram optimizes speech AI performance

2 reports2 outlets26 days agoUpdate

The story so far

AI overview

Deepgram deploys the Nova-3 speech recognition model to PyTorch via AWS Trainium2, running 8 replicas on a single trn2.3xlarge instance, processing 11-70% more audio per dollar when handling offline batch workloads, and using the NKI library and Neuron Explorer/NAD toolchain for kernel-level tuning to enhance inference efficiency. Amazon Trainium significantly improves its competitiveness in the training field with major enhancements to its native PyTorch training experience. Trainium3 supports the standard PyTorch API, allowing most PyTorch code to be migrated to .to("neuron") without XLA annotation, but the BF16 training experience is not yet fully mature. SemiAnalysis engineering team testing shows its performance is close to the CUDA ecosystem, but there is still room for improvement.

AI-written from the reports · Updated 22 days ago

Developments

2 developments

  1. 24 Oct, 03:06 · 1 reports
    Amazon’s Trainium Takes a Strike at the Training King
    SemiAnalysis:Amazon’s Trainium Takes a Strike at the Training King
  2. 20 Oct, 00:23 · 1 reports
    Deepgram optimized voice AI at scale with AWS Trainium
    AWS Machine Learning Blog:How Deepgram optimized voice AI at scale with AWS Trainium

Coverage timeline

Follow the coverage to see every side of the story.

24 Oct
  1. SemiAnalysis
    Amazon’s Trainium Takes a Strike at the Training King

    Amazon Trainium has significantly improved its training experience with native PyTorch, enhancing its competitiveness in the training field. Trainium3 supports the standard PyTorch API, allowing most PyTorch code to be migrated to .to('neuron') without XLA annotations, but the BF16 training experience is still not fully refined. SemiAnalysis's engineering team testing shows its performance is close to the CUDA ecosystem, but there is still room for improvement.

20 Oct
  1. AWS Machine Learning Blog
    How Deepgram optimized voice AI at scale with AWS Trainium

    Deepgram deploys the Nova-3 speech recognition model to PyTorch via AWS Trainium2, running 8 replicas on a single trn2.3xlarge instance, processing offline batch workloads with 11-70% more audio per dollar, and using the NKI library and Neuron Explorer/NAD toolchain for kernel-level tuning to improve inference efficiency.