Quickstart

Get the gateway running locally, generate an admin credential, and send an authenticated OpenAI-compatible request in a few minutes.

Run the gateway

Docker

docker run --rm -p 8080:8080 \
  -e OPENAI_API_KEY=sk-your-key \
  -e MASTER_KEY=fgw_your-master-key \
  ghcr.io/ferro-labs/ai-gateway:latest

MASTER_KEY is the bootstrap admin credential every /v1/* and /admin/* request authenticates with — pick your own value, or generate one the same way ferrogw init does:

export MASTER_KEY=fgw_$(openssl rand -hex 16)

With no config.yaml mounted, the gateway builds a default fallback config with one target per provider it finds credentials for — enough to route the request below. Mount a real config for retries, aliases, and plugins; see Configuration.

Downloaded binary

VER=$(curl -fsSL https://api.github.com/repos/ferro-labs/ai-gateway/releases/latest | grep '"tag_name"' | cut -d'"' -f4)
curl -fsSL "https://github.com/ferro-labs/ai-gateway/releases/download/${VER}/ferrogw_${VER#v}_linux_amd64.tar.gz" | tar xz
chmod +x ferrogw

export OPENAI_API_KEY=sk-your-key     # providers register at startup — export before init/serve
./ferrogw init                        # scaffolds config.yaml from detected provider keys, prints a MASTER_KEY
export GATEWAY_CONFIG=./config.yaml   # a config file is only loaded when this points at it
export MASTER_KEY=fgw_the-key-init-printed
./ferrogw

ferrogw init writes a minimal config.yaml naming every provider it found credentials for in the environment (a placeholder target if it found none), and generates the MASTER_KEY — printed once, never written to disk.

From source

git clone https://github.com/ferro-labs/ai-gateway.git
cd ai-gateway
make build

export OPENAI_API_KEY=sk-your-key
./bin/ferrogw init
export GATEWAY_CONFIG=./config.yaml
export MASTER_KEY=fgw_the-key-init-printed
make run

Check it's up

curl http://localhost:8080/readyz

{"status": "ready", "targets": [...]}

A 503 with "reason": "no routable targets" means no provider is both credentialed and named in targets — recheck the environment variables above and see targets is an allowlist.

Then open http://localhost:8080/ — the operations dashboard is compiled into the binary and served from the same port, no separate container. Sign in with MASTER_KEY to browse providers, routing, plugins, and live request logs.

Send a request

Replace model with a model you have access to. Every /v1/* request needs a bearer token — MASTER_KEY, or an API key created from it — since the gateway requires auth by default.

curl

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MASTER_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Hello from Ferro Labs AI Gateway"}
    ]
  }'

Python

Using the OpenAI SDK (pip install openai):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="fgw_your-master-key")

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from Ferro Labs AI Gateway"}],
)
print(response.choices[0].message.content)

Or the Ferro Labs SDK (pip install ferrolabsai):

from ferrolabsai import FerroClient

client = FerroClient(base_url="http://localhost:8080/v1", api_key="fgw_your-master-key")

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from Ferro Labs AI Gateway"}],
)
print(response.content)

Node / TypeScript

Using the OpenAI SDK (npm install openai):

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1",
  apiKey: "fgw_your-master-key",
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Ferro Labs AI Gateway" }],
});
console.log(response.choices[0]?.message.content);

Or the Ferro Labs SDK (npm install @ferro-labs-ai/sdk):

import { FerroClient } from "@ferro-labs-ai/sdk";

const client = new FerroClient({
  baseUrl: "http://localhost:8080/v1",
  apiKey: "fgw_your-master-key",
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Ferro Labs AI Gateway" }],
});
console.log(response.choices[0]?.message.content);

If you are using any OpenAI-compatible SDK, set the base URL to http://localhost:8080/v1, the API key to MASTER_KEY (or an issued fgw_... key), and keep the rest of your code unchanged.

A successful call returns:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "gpt-4o-mini",
  "choices": [
    {"index": 0, "message": {"role": "assistant", "content": "..."}, "finish_reason": "stop"}
  ],
  "usage": {"prompt_tokens": 12, "completion_tokens": 9, "total_tokens": 21}
}

Stream a response

Set stream: true for Server-Sent Events, same as the OpenAI API — every OpenAI SDK's streaming helper works unchanged against the gateway:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MASTER_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "stream": true,
    "messages": [{"role": "user", "content": "Count to 5"}]
  }'

data: {"id":"chatcmpl-...","choices":[{"index":0,"delta":{"content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","choices":[{"index":0,"delta":{"content":"2"},"finish_reason":null}]}

...

data: [DONE]

What's next

• Configuration — write a real config.yaml: targets, retry, aliases, plugins
• Providers — register more of the 30 supported providers and see the endpoint matrix
• Routing — fallback, load balance, cost-optimized, and 5 more strategies
• Plugins — budgets, rate limiting, response caching, and request logging
• Dashboard guide — tour the embedded operations console
• CLI reference — ferrogw validate, doctor, status, admin keys