Agentuity Documentation

Agentuity's AI Gateway routes LLM requests through managed infrastructure, giving you observability and cost tracking across all model providers.

How It Works

When you make LLM requests from your agents, they're automatically routed through the AI Gateway:

Your Agent → AI Gateway → Provider API (OpenAI, Anthropic, etc.)

The AI Gateway provides:

Consolidated billing across all LLM providers
Automatic observability with token tracking and latency metrics
Request logging visible in the Agentuity console
No configuration required when using your SDK key

Using the AI Gateway

The AI Gateway works automatically, whether you use provider SDKs directly (Anthropic, OpenAI, Groq), the Vercel AI SDK, or frameworks like Mastra and LangGraph. No configuration needed.

Provider SDKs

Use provider SDKs directly and get AI Gateway routing automatically:

import { createAgent } from '@agentuity/runtime';
import Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import Groq from 'groq-sdk';
import { s } from '@agentuity/schema';
 
// Direct SDK clients — all route through the AI Gateway
const anthropic = new Anthropic();
const openai = new OpenAI();
const groq = new Groq();
 
const agent = createAgent('AnthropicChat', {
  schema: {
    input: s.object({ prompt: s.string() }),
    output: s.object({ response: s.string() }),
  },
  handler: async (ctx, input) => {
    const result = await anthropic.messages.create({
      model: 'claude-sonnet-4-5',
      max_tokens: 1024,
      messages: [{ role: 'user', content: input.prompt }],
    });
 
    const text = result.content[0]?.type === 'text'
      ? result.content[0].text
      : '';
 
    return { response: text };
  },
});
 
export default agent;

AI SDK Providers

The Vercel AI SDK providers also route through Agentuity's AI Gateway:

import { createAgent } from '@agentuity/runtime';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { s } from '@agentuity/schema';
 
const agent = createAgent('TextGenerator', {
  schema: {
    input: s.object({ prompt: s.string() }),
    output: s.object({ response: s.string() }),
  },
  handler: async (ctx, input) => {
    const { text } = await generateText({
      model: openai('gpt-5-mini'),
      prompt: input.prompt,
    });
 
    return { response: text };
  },
});
 
export default agent;

Provider Imports

All supported providers route through the AI Gateway:

// Provider SDKs
import Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import Groq from 'groq-sdk';
 
// AI SDK providers
import { openai } from '@ai-sdk/openai';
import { anthropic } from '@ai-sdk/anthropic';
import { google } from '@ai-sdk/google';
import { xai } from '@ai-sdk/xai';
import { deepseek } from '@ai-sdk/deepseek';
import { groq } from '@ai-sdk/groq';
import { mistral } from '@ai-sdk/mistral';
import { cohere } from '@ai-sdk/cohere';
import { perplexity } from '@ai-sdk/perplexity';

Supported Providers

Provider	Example Models
OpenAI	`gpt-5.2`, `gpt-5-mini`, `gpt-5-nano`
Anthropic	`claude-opus-4-5`, `claude-sonnet-4-5`, `claude-haiku-4-5`
Google	`gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`
xAI	`grok-4-1-fast-reasoning`, `grok-code-fast-1`, `grok-4-fast-reasoning`
DeepSeek	`deepseek-chat`, `deepseek-reasoner`
Groq	`llama-4-scout-17b-16e`, `llama-4-maverick-17b-128e`
Mistral	`mistral-large-latest`, `devstral-large-latest`, `mistral-medium-latest`
Cohere	`command-a`, `command-r-plus`
Perplexity	`sonar-pro`, `sonar`

Model IDs Change Frequently

Provider catalogs and model IDs are updated often. Verify current availability in each provider's official docs.

BYO API Keys

Bypass the AI Gateway by setting your own API keys in .env:

dotenv.env

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_GENERATIVE_AI_API_KEY=...
XAI_API_KEY=...
DEEPSEEK_API_KEY=...
GROQ_API_KEY=...
MISTRAL_API_KEY=...

When these variables are set, requests go directly to the provider instead of through the AI Gateway.

AI Gateway vs BYO Keys

Aspect	AI Gateway	BYO API Keys
Setup	Just SDK key	Manage per-provider keys
Cost tracking	Automatic in console	Manual
Observability	Built-in token/latency metrics	Must configure separately
Rate limits	Shared pool	Your own limits

We recommend the AI Gateway for most projects.

Next Steps

Using the AI SDK: Structured output, tool calling, and multi-turn conversations
Returning Streaming Responses: Real-time chat UIs and progress indicators
Logging: Debug requests and track LLM performance