Introducing Private Model Gateways for Mastra Agents

BlogModels & Routing

Self-hosted models that keep data private and costs fixed.

You can now route agent requests to self-hosted open-weight models using a custom model gateway. Give the gateway a custom ID, then use it as the model prefix in your agent's config: private/ollama/gpt-oss:20b.

Using a hosted provider service can complicate HIPAA or GDPR compliance (unless data retention agreements are in place). Self-hosting removes the third party, preventing sensitive data from reaching models outside of your network.

Costs may also be reduced as you're only paying for infrastructure, not tokens, and you control which models your agents can use.

To hide external providers from Studio's model picker, set AUTO_BLOCK_EXTERNAL_PROVIDERS=true in your .env so users can only select models from your own gateways.

Get started

Install the latest @mastra/core and an AI SDK provider for your gateway's API:

npm install @mastra/core@latest @ai-sdk/[email protected]

note

Requires @mastra/[email protected] or later, added in PR #18153.

Run a local model

This example uses Ollama and gpt-oss:20b. Running ollama serve starts the server, with an OpenAI-compatible API at http://localhost:11434/v1:

brew install ollama
ollama pull gpt-oss:20b
ollama serve

Create a gateway

Create a gateway that sends every model call to the local Ollama server:

• id: The model ID prefix, e.g. private in private/ollama/gpt-oss:20b.
• name: The gateway's display name.
• fetchProviders(): Returns the providers and models the gateway offers, listed in Studio's model picker.
• buildUrl(): Returns the base URL for model calls.
• getApiKey(): Returns the API key to send with each call (not required for local models).
• resolveLanguageModel(): Creates the language model for each call.

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import type { MastraModelGatewayInterface } from "@mastra/core/llm";
 
export const privateGateway: MastraModelGatewayInterface = {
  id: "private",
  name: "Private Gateway",
  fetchProviders: async () => {
    return {
      ollama: {
        name: "Ollama",
        models: ["gpt-oss:20b"],
        apiKeyEnvVar: [],
        gateway: "private",
        url: "http://localhost:11434/v1"
      }
    };
  },
  buildUrl: () => {
    return "http://localhost:11434/v1";
  },
  getApiKey: async () => {
    return "";
  },
  resolveLanguageModel: ({ modelId, providerId, apiKey }) => {
    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL: "http://localhost:11434/v1"
    }).chatModel(modelId);
  }
};

Register the gateway

Add the gateway to your Mastra instance:

export const mastra = new Mastra({
  // ...
  gateways: { privateGateway }
});

Use the model

Reference the model by [gatewayId]/[provider]/[model]:

export const researchAgent = new Agent({
  id: "research-agent",
  name: "Research Agent",
  instructions: /* ... */,
  model: "private/ollama/gpt-oss:20b"
});

Hide external providers in Studio

Add the variable to your .env:

AUTO_BLOCK_EXTERNAL_PROVIDERS=true

For more information and full configuration options, see:

• Custom model gateways
• When to use custom gateways
• Showing only custom gateways in Studio
• Gateway-owned authentication
• Model pricing

Paul Scanlon sits between Developer Education and Product Marketing at Mastra. Previously, he was a Technical Product Marketing Manager at Neon and worked in Developer Relations at Gatsby, where he created educational content and developer experiences.

All articles by Paul Scanlon →

BlogAug 18, 2026

AI SDK v7 support in Mastra

Image generation, reasoning control, multimodal prompts, stream handlers, and UI helpers.

ArticleAug 16, 2026

LLM vs generative AI: key differences and how to choose

LLM vs generative AI explained: how they differ in scope, architecture, and use cases, plus how to choose the right approach for your app.