Run AI inference at the edge with Workers AI

After this lesson you'll be able to call an open-source model from a Worker with one binding call, stream the output back to a client, and pick a model size that matches your latency budget instead of defaulting to the biggest one available.

Workers AI is serverless GPU inference: you call a model — text generation, image generation, speech-to-text, embeddings, classification, and more — and Cloudflare runs it on GPUs distributed across its network. There's no GPU instance to provision, no model server to deploy or scale, and no idle capacity to pay for. You bind a Worker to AI, call env.AI.run() with a model name and input, and get a result back. The infrastructure — which GPU, in which data center, how the model is loaded and kept warm — is Cloudflare's problem, not yours.

How it works

Three things to know about the mechanics:

• The binding, not a fetch call. You declare an ai binding in your Wrangler config, and Cloudflare injects an AI object into env. Calling env.AI.run(model, input) is a method call on that binding — the request never leaves Cloudflare's network to hit a public API endpoint, it's routed internally to available inference capacity.
• Model names are catalog identifiers. Each model is addressed by a string like @cf/meta/llama-3.1-8b-instruct or @cf/baai/bge-m3 — the @cf/ prefix plus vendor and model name. The full catalog, with input/output schemas per model, is at the Workers AI models page; task categories include text generation, text-to-image, automatic speech recognition, text embeddings, image classification, object detection, translation, and summarization.
• Streaming is a flag, not a different API. Text-generation models accept stream: true in the input. Instead of a JSON object, env.AI.run() then resolves to a ReadableStream of server-sent events, which you can hand straight to the Response constructor — the tokens flow to the client as the model produces them, instead of the client waiting for the full generation to finish.

Worked example

Binding config:

// wrangler.jsonc
{
  "ai": {
    "binding": "AI"
  }
}

A Worker that accepts a prompt and streams a chat completion back to the caller:

export interface Env {
  AI: Ai;
}

export default {
  async fetch(request, env): Promise<Response> {
    const { prompt } = await request.json<{ prompt: string }>();

    const stream = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [
        { role: "system", content: "You are a concise technical assistant." },
        { role: "user", content: prompt },
      ],
      stream: true,
    });

    return new Response(stream, {
      headers: { "content-type": "text/event-stream" },
    });
  },
} satisfies ExportedHandler<Env>;

The client reads it like any other SSE stream — each chunk arrives as soon as the model emits it, so a user sees the reply appear token-by-token instead of staring at a spinner for the whole generation.

The same binding, called without stream, works identically for embeddings — the pattern you'd use to populate a Vectorize index:

const { data } = await env.AI.run("@cf/baai/bge-m3", {
  text: ["Cloudflare Workers run JavaScript at the edge."],
});
// data[0] is a float embedding vector, ready to upsert into Vectorize

Pricing

Workers AI bills in Neurons — a unit that normalizes GPU compute cost across very different model types (a text-generation token and an image-generation step consume GPU differently, so Neurons let Cloudflare price both on one scale). As of this writing:

Neuron cost per call scales with model size, as you'd expect: roughly 2,457 Neurons per million input tokens on a small model like Llama 3.2 1B, versus roughly 25,600 Neurons per million input tokens on Llama 3.1 8B, and roughly 45,000+ Neurons per million input tokens on a large model like DeepSeek R1 32B. Image, speech, and embedding models are priced per their own input unit (tiles, minutes, or tokens) rather than per generated text token. Treat these figures as illustrative — confirm exact current numbers on the pricing page linked below before quoting them in a proposal.

Use cases

• Chatbots and support assistants — a text-generation model behind a Worker, often paired with streaming so replies feel responsive.
• Summarization — condensing support tickets, documents, or transcripts, either with a dedicated summarization model or a general instruct model with a summarization prompt.
• Content moderation — classification models scoring text or images for policy violations before they're stored or served.
• Embeddings generation for Vectorize — running text through a BGE-family embedding model to produce vectors for semantic search, then upserting them into a Vectorize index.