Put a control plane in front of your LLM calls

After this lesson you'll be able to repoint an existing Anthropic (or OpenAI, or Workers AI) SDK call through an AI Gateway endpoint and get logging, caching, rate limiting, and cost tracking for free, with no change to how you call the model.

AI Gateway is a reverse proxy that sits between your code and whatever LLM provider you're calling. You don't rewrite your integration — you change one string, the baseURL, so requests that used to go straight to api.anthropic.com now go through gateway.ai.cloudflare.com first, then on to Anthropic. The gateway forwards the request and streams back the response, but along the way it logs the request/response, can serve a cached response instead of calling the provider at all, can enforce a rate limit, and can retry or fall back to a different provider if the first one errors or times out. It's the same pattern as a database connection pool or an API gateway in front of microservices, applied to the specific pain points of calling LLMs: unpredictable cost, provider outages, and zero built-in observability.

How it works

Every AI Gateway you create gets an account- and gateway-scoped URL prefix:

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}

{provider} is a slug like anthropic, openai, workers-ai, google-ai-studio, or azure-openai. The gateway preserves each provider's own request/response schema — it's not translating Anthropic's API into OpenAI's — so your existing SDK code, types, and error handling keep working. The only change is the base URL and, optionally, an extra header if the gateway has authenticated mode turned on.

Request flow: your code calls the gateway URL with your normal provider API key → Cloudflare's edge receives it → checks cache (if caching is on and the request matches a cached entry, it returns immediately without touching the provider) → otherwise forwards to the real provider with your API key attached → logs the request metadata, token counts, and cost → streams the response back to you, cached for next time if caching applies.

What the gateway adds

• Analytics & logging — every request's provider, model, token counts, latency, and estimated cost, viewable in the dashboard without instrumenting your own code.
• Caching — identical requests can be served from Cloudflare's cache instead of hitting the provider, cutting both latency and spend.
• Rate limiting — cap requests per gateway to protect your provider quota or budget from a runaway loop or a traffic spike.
• Retries and fallback — via the separate Universal endpoint, define a priority list of providers/models; if one errors or times out, the gateway automatically tries the next.

Pricing

AI Gateway's core features — dashboard analytics, caching, and rate limiting — are free on every plan; you only pay the underlying provider (Anthropic, OpenAI, etc.) for the tokens you actually use. Cloudflare's own charges are limited to a few adjacent pieces:

Cloudflare notes it may add premium features later. Treat the numbers above as a snapshot — confirm current figures on the live pricing page linked below before budgeting.

Use cases

• Cost observability across a team — instead of every developer's Anthropic/OpenAI dashboard showing a blended number, route all calls through one gateway (or one per environment) and see spend broken out by model, endpoint, or time window in one place.
• Multi-provider fallback — configure a Universal endpoint that tries OpenAI first and falls back to Anthropic (or a different model) on error or timeout, so a single provider outage doesn't take your feature down.
• Cutting LLM spend with caching — for prompts that repeat verbatim (a fixed system prompt plus a common set of user questions, a docs chatbot answering FAQs), caching returns the prior response instantly and for free instead of re-billing the provider every time.
• Protecting quota from bugs — a rate limit on the gateway catches a retry loop or bad deploy before it burns through your provider's rate limit or your monthly budget.

Worked example

Create a gateway once in the dashboard (AI > AI Gateway > Create Gateway), note your account ID and the gateway name you chose, then repoint the Anthropic SDK's baseURL:

// Before: calling Anthropic directly
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

// After: same call, routed through AI Gateway
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY, // still your real Anthropic key
  baseURL: `https://gateway.ai.cloudflare.com/v1/${process.env.CF_ACCOUNT_ID}/my-gateway/anthropic`,
  defaultHeaders: {
    // Only needed if the gateway requires Cloudflare auth
    "cf-aig-authorization": `Bearer ${process.env.CF_AIG_TOKEN}`,
  },
});

const message = await anthropic.messages.create({
  model: "claude-sonnet-4-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Summarize this changelog in two sentences." }],
});

console.log(message.content);

No change to messages.create(), its response shape, or your error handling. Open the AI Gateway dashboard and this call now shows up as a logged request with its model, token counts, latency, and cost. To bypass the cache for a request you know must be fresh, add a header per-request:

const message = await anthropic.messages.create(
  {
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    messages: [{ role: "user", content: "What time is it in Tokyo right now?" }],
  },
  { headers: { "cf-aig-skip-cache": "true" } }
);