Skip to main content

Vercel AI SDK

The Vercel AI SDK already speaks OpenAI. Point its OpenAI provider at a Ferro Labs AI Gateway endpoint and every streamText, generateText, generateObject, and tool-call goes through Ferro — with automatic provider routing, fallback, cost tracking, and the x-trace-id header surfacing on every response.

No extra adapter required for chat. The @ferro-labs-ai/sdk package below adds Ferro-specific niceties on top.

pnpm add ai @ai-sdk/openai
import { openai, createOpenAI } from "@ai-sdk/openai";
import { streamText } from "ai";

// Point the AI SDK's OpenAI provider at your Ferro gateway.
const ferro = createOpenAI({
  baseURL: process.env.FERRO_BASE_URL,    // e.g. "http://localhost:8080/v1"
  apiKey: process.env.FERRO_API_KEY,
});

const result = await streamText({
  model: ferro("gpt-4o"),
  prompt: "Tell me a story about a gateway.",
});

for await (const delta of result.textStream) {
  process.stdout.write(delta);
}

Swap providers by changing the model string — the gateway routes by name:

const result = await streamText({
  model: ferro("claude-3-5-sonnet-20241022"),   // → Anthropic
  prompt: "Same code, different provider.",
});

Read the trace_id from response headers​

The gateway emits the x-trace-id HTTP header on every response (frozen contract since ai-gateway v1.1.0). The AI SDK exposes raw provider responses via result.providerMetadata and the underlying fetch response:

const result = await generateText({
  model: ferro("gpt-4o"),
  prompt: "hi",
});

const traceId = result.response?.headers?.["x-trace-id"];
console.log({ traceId, providerMetadata: result.providerMetadata });

That traceId is the join key for any observability bridge plugin wired into the gateway — Langfuse, Phoenix, LangSmith, Datadog. Your Next.js app stays portable.

Use @ferro-labs-ai/sdk for typed Ferro extras​

The drop-in works for chat. For admin APIs, model catalog, image generation, and typed responses with trace_id, provider, cost_usd, latency_ms already broken out, install the official SDK:

pnpm add @ferro-labs-ai/sdk
import { FerroClient } from "@ferro-labs-ai/sdk";

const client = new FerroClient({
  baseUrl: process.env.FERRO_BASE_URL,
  apiKey: process.env.FERRO_API_KEY,
});

const response = await client.chat.completions.create({
  model: "claude-3-5-sonnet-20241022",
  messages: [{ role: "user", content: "hi" }],
});

console.log(response.choices[0].message.content);
console.log(response.traceId);     // x-trace-id
console.log(response.provider);    // "anthropic"
console.log(response.costUsd);     // computed cost

Mix and match: use the AI SDK for streaming UI flows, @ferro-labs-ai/sdk for admin and analytics.

Next.js Edge route example​

// app/api/chat/route.ts
import { createOpenAI } from "@ai-sdk/openai";
import { streamText } from "ai";

export const runtime = "edge";

const ferro = createOpenAI({
  baseURL: process.env.FERRO_BASE_URL!,
  apiKey: process.env.FERRO_API_KEY!,
});

export async function POST(req: Request) {
  const { messages, model = "gpt-4o-mini" } = await req.json();
  const result = streamText({ model: ferro(model), messages });
  return result.toDataStreamResponse({
    headers: { "x-ferro-trace-id": result.response?.headers?.["x-trace-id"] ?? "" },
  });
}

Forward the trace ID to the browser via a response header so client-side observability can correlate UI events with gateway-side traces.

Verify​

curl -i http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $FERRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"stream":true}'

Look for x-trace-id in the response headers and SSE chunks in the body.

Status​

CapabilityStatus
Chat / streaming via @ai-sdk/openai + baseURL✅ Live
Typed Ferro extras via @ferro-labs-ai/sdk✅ Live (@ferro-labs-ai/sdk 0.1.0)
@ai-sdk/ferro-labs first-party provider🚧 Planned — under evaluation
Cookbook recipe typescript/01-vercel-ai-sdk-fallback🚧 Planned

Runnable example​

ai-gateway-cookbook/typescript/01-vercel-ai-sdk-fallback is planned — until it lands, the Vercel AI SDK docs on custom providers plus the snippets above cover the full path.

See also​