Quickstart

Run with Docker

docker run --rm -p 8080:8080 \
  -e OPENAI_API_KEY=sk-your-key \
  ghcr.io/ferro-labs/ai-gateway:latest

Build from source

git clone https://github.com/ferro-labs/ai-gateway.git
cd ai-gateway

export OPENAI_API_KEY=sk-your-key
make run

Send a request

Replace model with a model you have access to. The gateway serves an OpenAI-compatible API at /v1, so any OpenAI SDK works by pointing base_url at http://localhost:8080/v1.

curl

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Hello from Ferro AI Gateway"}
    ]
  }'

Python

Using the OpenAI SDK (pip install openai):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="sk-your-key")

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from Ferro AI Gateway"}],
)
print(response.choices[0].message.content)

Or the Ferro Labs SDK (pip install ferrolabsai):

from ferrolabsai import FerroClient

client = FerroClient(base_url="http://localhost:8080/v1", api_key="sk-your-key")

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from Ferro AI Gateway"}],
)
print(response.content)

Node / TypeScript

Using the OpenAI SDK (npm install openai):

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1",
  apiKey: "sk-your-key",
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Ferro AI Gateway" }],
});
console.log(response.choices[0]?.message.content);

Or the Ferro Labs SDK (npm install @ferro-labs-ai/sdk):

import { FerroClient } from "@ferro-labs-ai/sdk";

const client = new FerroClient({
  baseUrl: "http://localhost:8080/v1",
  apiKey: "sk-your-key",
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Ferro AI Gateway" }],
});
console.log(response.choices[0]?.message.content);

If you are using any OpenAI-compatible SDK, set the base URL to http://localhost:8080/v1 and keep the rest of your code unchanged.