Exa has enhanced the throughput of AWS AI chips.
The story so far
Exa achieves a 50% boost in embedding throughput by optimizing embedding infrastructure on AWS Inferentia2 and Trainium2 Chips, processing billions of documents for indexing at a lower cost. With Neuron SDK and AWS Trainium team support, through weight layout optimization, automatic precision reduction calculation, and NKI kernel technology, the throughput is increased by about 1.5 times while maintaining a cosine similarity accuracy rate of 0.999.
AI-written from the reports · Updated 17 minutes ago
Coverage timeline
Follow the coverage to see every side of the story.
- AWS Machine Learning BlogExa boosts embedding throughput by 50% to index billions of documents on AWS AI chips
Exa achieves a 50% boost in embedding throughput by optimizing embedding infrastructure on AWS Inferentia2 and Trainium2 Chips, processing billions of documents for indexing at a lower cost. With Neuron SDK and AWS Trainium team support, through weight layout optimization, automatic precision reduction calculation, and NKI kernel technology, the throughput is increased by about 1.5 times while maintaining a cosine similarity accuracy rate of 0.999.
Heat over time
Not enough data yet to draw a trend.