Go Integration

The Ferro Labs AI Gateway is written in Go. This guide covers everything a Go developer needs — from pointing a standard OpenAI client at the gateway, to embedding the engine as a library, writing custom plugins, propagating trace context, and profiling with pprof.

1. Using the go-openai Client

The sashabaranov/go-openai client works out of the box. Point it at your gateway URL and set your API key.

package main

import (
	"context"
	"fmt"
	"log"

	openai "github.com/sashabaranov/go-openai"
)

func main() {
	cfg := openai.DefaultConfig("your-ferro-api-key")
	cfg.BaseURL = "http://localhost:8080/v1" // Ferro Labs AI Gateway

	client := openai.NewClientWithConfig(cfg)

	resp, err := client.CreateChatCompletion(
		context.Background(),
		openai.ChatCompletionRequest{
			Model: openai.GPT4,
			Messages: []openai.ChatCompletionMessage{
				{
					Role:    openai.ChatMessageRoleUser,
					Content: "Explain the CAP theorem in two sentences.",
				},
			},
		},
	)
	if err != nil {
		log.Fatalf("completion error: %v", err)
	}

	fmt.Println(resp.Choices[0].Message.Content)
}

2. Embedding Ferro as a Library

You can embed the gateway engine directly into your Go application instead of running it as a standalone process. This is useful for custom orchestration or testing.

The public aigateway package does not expose an HTTP server — it gives you a routing engine. You create a Gateway, register providers, optionally load plugins, and then call Route / RouteStream / Embed / GenerateImage directly. (If you want the full HTTP server with the admin dashboard, run the ferrogw binary instead — see Quickstart.)

package main

import (
	"context"
	"fmt"
	"log"

	gateway "github.com/ferro-labs/ai-gateway"
	"github.com/ferro-labs/ai-gateway/providers"

	// Built-in plugins are compiled in via blank imports of internal/plugins/*.
	_ "github.com/ferro-labs/ai-gateway/internal/plugins/wordfilter"
)

func main() {
	// LoadConfig returns *Config; New takes a Config value, so dereference it.
	cfg, err := gateway.LoadConfig("ferro.yaml")
	if err != nil {
		log.Fatalf("failed to load config: %v", err)
	}

	gw, err := gateway.New(*cfg)
	if err != nil {
		log.Fatalf("failed to create gateway: %v", err)
	}
	defer gw.Close()

	// Register every provider whose required environment variables are set
	// (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY).
	for _, entry := range providers.AllProviders() {
		pcfg := providers.ProviderConfigFromEnv(entry)
		if pcfg == nil {
			continue // required env var unset — skip
		}
		p, err := entry.Build(pcfg)
		if err != nil {
			log.Fatalf("provider %s init failed: %v", entry.ID, err)
		}
		gw.RegisterProvider(p)
	}

	// Initialize and register plugins declared in the config file.
	if err := gw.LoadPlugins(); err != nil {
		log.Fatalf("failed to load plugins: %v", err)
	}

	// Route a request through the engine.
	resp, err := gw.Route(context.Background(), providers.Request{
		Model: "gpt-4o-mini",
		Messages: []providers.Message{
			{Role: "user", Content: "Explain the CAP theorem in two sentences."},
		},
	})
	if err != nil {
		log.Fatalf("route error: %v", err)
	}

	fmt.Println(resp.Choices[0].Message.Content)
}

Streaming uses gw.RouteStream, which returns a <-chan providers.StreamChunk you range over. Call gw.Close() once at shutdown to stop the background hook workers and catalog refresher.

3. Writing a Custom Plugin

Plugins implement the plugin.Plugin interface from github.com/ferro-labs/ai-gateway/plugin:

type Plugin interface {
	Name() string
	Type() PluginType
	Init(config map[string]interface{}) error
	Execute(ctx context.Context, pctx *Context) error
}

A plugin is registered at a lifecycle stage (before_request, after_request, or on_error) and is invoked via Execute at that stage. The *plugin.Context carries the request, the response (on later stages), and a Metadata map you can use to pass state between stages of the same request — there is no SetMeta/GetMeta.

package myplugin

import (
	"context"
	"log"
	"time"

	"github.com/ferro-labs/ai-gateway/plugin"
)

// TimingPlugin logs the round-trip duration for every request.
type TimingPlugin struct{}

// Name returns the plugin identifier used in configuration.
func (p *TimingPlugin) Name() string { return "timing" }

// Type categorizes the plugin.
func (p *TimingPlugin) Type() plugin.PluginType { return plugin.TypeLogging }

// Init receives the plugin's config block from the gateway configuration.
func (p *TimingPlugin) Init(config map[string]interface{}) error { return nil }

// Execute runs at whatever stage the plugin is registered for. Stash the start
// time on pctx.Metadata in the before_request stage and read it back later.
func (p *TimingPlugin) Execute(ctx context.Context, pctx *plugin.Context) error {
	if pctx.Response == nil {
		// before_request stage — record the start time.
		pctx.Metadata["timing_start"] = time.Now()
		log.Printf("[timing] request to model=%s started", pctx.Request.Model)
		return nil
	}

	// after_request stage — measure elapsed time.
	start, ok := pctx.Metadata["timing_start"].(time.Time)
	if !ok {
		return nil
	}
	log.Printf("[timing] model=%s completed in %s (tokens: %d)",
		pctx.Response.Model, time.Since(start), pctx.Response.Usage.TotalTokens)
	return nil
}

Register the same instance at both stages programmatically:

p := &myplugin.TimingPlugin{}
_ = gw.RegisterPlugin(plugin.StageBeforeRequest, p)
_ = gw.RegisterPlugin(plugin.StageAfterRequest, p)

Alternatively, register a factory in an init() so the plugin can be enabled from config by name. Add a blank import of your package to cmd/ferrogw/main.go (this is how the built-in plugins are wired in), then list it under plugins: in your config and let gw.LoadPlugins() construct it:

func init() {
	plugin.RegisterFactory("timing", func() plugin.Plugin {
		return &TimingPlugin{}
	})
}

4. Context Propagation and Trace ID Extraction

The AI Gateway injects an X-Request-ID header into every response. Use this to correlate gateway logs with your application traces.

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"

	openai "github.com/sashabaranov/go-openai"
)

func main() {
	cfg := openai.DefaultConfig("your-ferro-api-key")
	cfg.BaseURL = "http://localhost:8080/v1"

	// Use a custom HTTP client to capture response headers.
	var traceID string
	cfg.HTTPClient = &http.Client{
		Transport: &traceTransport{
			base: http.DefaultTransport,
			onResponse: func(resp *http.Response) {
				traceID = resp.Header.Get("X-Request-ID")
			},
		},
	}

	client := openai.NewClientWithConfig(cfg)

	resp, err := client.CreateChatCompletion(
		context.Background(),
		openai.ChatCompletionRequest{
			Model: openai.GPT4,
			Messages: []openai.ChatCompletionMessage{
				{Role: openai.ChatMessageRoleUser, Content: "Hello"},
			},
		},
	)
	if err != nil {
		log.Fatalf("completion error: %v", err)
	}

	fmt.Printf("Response: %s\n", resp.Choices[0].Message.Content)
	fmt.Printf("Trace ID: %s\n", traceID)
}

// traceTransport wraps an http.RoundTripper to expose response headers.
type traceTransport struct {
	base       http.RoundTripper
	onResponse func(*http.Response)
}

func (t *traceTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	resp, err := t.base.RoundTrip(req)
	if err != nil {
		return resp, err
	}
	if t.onResponse != nil {
		t.onResponse(resp)
	}
	return resp, nil
}

5. Benchmarking with pprof

Enable the pprof endpoint to profile the gateway under load.

Enable the pprof endpoint

The ferrogw binary mounts the standard net/http/pprof handlers on its own server (port 8080 by default) under /debug/pprof/* when the ENABLE_PPROF environment variable is truthy:

ENABLE_PPROF=1 ferrogw serve

The profiling endpoints are then available at http://localhost:8080/debug/pprof/.

If you are embedding the engine as a library (section 2), there is no HTTP server, so expose pprof yourself on a separate port using the standard library:

import "net/http"
import _ "net/http/pprof"

go func() {
	log.Println("pprof listening on :6060")
	log.Println(http.ListenAndServe("localhost:6060", nil))
}()

Run a load test and capture a profile

Use hey (or any HTTP load testing tool) alongside go tool pprof:

# Generate load — 1000 requests, 50 concurrent
hey -n 1000 -c 50 \
  -H "Authorization: Bearer your-ferro-api-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[{"role":"user","content":"ping"}]}' \
  http://localhost:8080/v1/chat/completions

# Capture a 30-second CPU profile
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# Capture a heap snapshot
go tool pprof http://localhost:6060/debug/pprof/heap

Analyze the profile

Inside the pprof interactive shell:

(pprof) top 20
(pprof) web          # opens a call-graph SVG in your browser
(pprof) list HandleRequest   # source-level annotation

Write a Go benchmark test

package gateway_test

import (
	"context"
	"testing"

	openai "github.com/sashabaranov/go-openai"
)

func BenchmarkChatCompletion(b *testing.B) {
	cfg := openai.DefaultConfig("your-ferro-api-key")
	cfg.BaseURL = "http://localhost:8080/v1"
	client := openai.NewClientWithConfig(cfg)

	req := openai.ChatCompletionRequest{
		Model: openai.GPT4,
		Messages: []openai.ChatCompletionMessage{
			{Role: openai.ChatMessageRoleUser, Content: "ping"},
		},
	}

	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, err := client.CreateChatCompletion(context.Background(), req)
		if err != nil {
			b.Fatalf("request failed: %v", err)
		}
	}
}

Run with:

go test -bench=BenchmarkChatCompletion -benchtime=30s -count=3 ./...

Related Pages

• Benchmarks
• Quickstart
• Configuration