Skip to content
Live
Trending storyDeveloping

AWS Launches Self-Service LLM Gateway

2 reports1 outlets19 hours agoUpdate

The story so far

AI overview

Amazon released a self-service LLM gateway using Amazon Bedrock and LiteLLM, allowing developers to quickly access base models through application inference profiles, achieving single API access, user-level cost control, and model performance observability. Subsequently, Amazon used NVIDIA NeMo Switchyard on Amazon Bedrock to route model requests for coding agents such as Claude Code and OpenAI Codex CLI, with developers paying per task rather than per single call.

AI-written from the reports · Updated 19 hours ago

Latest14 Nov, 06:30
Using NVIDIA NeMo Switchyard on Amazon Bedrock to Route Model Requests for Coding Agents like Claude Code and OpenAI Codex CLI

Coverage timeline

Follow the coverage to see every side of the story.

14 Nov
  1. AWS Machine Learning Blog
    Using NVIDIA NeMo Switchyard on Amazon Bedrock to Route Model Requests for Coding Agents like Claude Code and OpenAI Codex CLI

    Developers using coding agents like Claude Code and OpenAI Codex CLI need to be billed per task rather than per single call, so model routing should be based on the cost per task. NVIDIA NeMo Switchyard is an open-source library that helps AI agents select models to process each request, supporting Stage and Random routing strategies.

13 Nov
  1. AWS Machine Learning Blog
    Build an LLM gateway using Amazon Bedrock and LiteLLM

    Developers can quickly access base models through Amazon Bedrock application inference profiles and LiteLLM, achieving single API access, user-level cost control, and model performance observability.

Heat over time

Now
6
Peak
1014 Nov, 07:00
24h change
+13%

Hover or tap the chart to see each hour; use the arrow keys on a keyboard.