Skip to main content

Instructor

Instructor patches an existing OpenAI client to return validated Pydantic models. Because Ferro Labs AI Gateway is an OpenAI-compatible endpoint, Instructor works against it unchanged.

Install​

pip install instructor openai

Configure​

import os
import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(OpenAI(
    base_url=os.environ["FERRO_BASE_URL"] + "/v1",
    api_key=os.environ["FERRO_API_KEY"],
))

class User(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    model="claude-3-5-sonnet-20241022",     # routed to Anthropic by Ferro
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Alice is 30 years old."}],
)
print(user)  # User(name='Alice', age=30)

The retry / validation loop happens client-side in Instructor; the gateway sees standard chat-completion requests.

Verify​

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $FERRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

See also​