Skip to main content

Providers

The Ferro Labs AI Gateway speaks to 30 LLM providers behind a single OpenAI-compatible API. Point any client written against the OpenAI SDK at the gateway and it reaches every provider unchanged — the gateway translates each request and response to and from the OpenAI shape, whether the upstream is an OpenAI-compatible surface or a native wire (Anthropic Messages, Gemini generateContent, Cohere v2, Bedrock InvokeModel, Vertex, Replicate).

This page is the provider directory: the logo grid links each provider to its per-provider setup, and the endpoint-support matrix shows which surfaces each one implements. For the catalogue of individual models (2,500+ across all providers), browse ferrolabs.ai/models — this page does not list models.

targets is an allowlist (v1.4.0+)

Setting a provider's environment variable registers it, but registration is not routing. A provider only serves traffic when it is listed under targets[] in your config. A request for a model owned only by an unlisted provider returns 404 model_not_found, even though the provider is present in the process and healthy. Registration is driven by which credentials the environment holds; routing is driven by which names the config lists — the two sets can be disjoint. See Routing and Provider configuration.

Supported providers​

AI21 Labs logoAI21 LabsAnthropic logoAnthropicAzure AI Foundry logoAzure AI FoundryAzure OpenAI logoAzure OpenAIAWS Bedrock logoAWS BedrockCerebras logoCerebrasCloudflare Workers AI logoCloudflare Workers AICohere logoCohereDatabricks logoDatabricksDeepInfra logoDeepInfraDeepSeek logoDeepSeekFireworks AI logoFireworks AIGoogle Gemini logoGoogle GeminiGroq logoGroqHugging Face logoHugging FaceMistral AI logoMistral AIMoonshot AI logoMoonshot AINovita logoNovitaNVIDIA NIM logoNVIDIA NIMOllama logoOllamaOllama Cloud logoOllama CloudOpenAI logoOpenAIOpenRouter logoOpenRouterPerplexity logoPerplexityQwen logoQwenReplicate logoReplicateSambaNova logoSambaNovaTogether AI logoTogether AIGoogle Vertex AI logoGoogle Vertex AIxAI logoxAI

Endpoint-support matrix​

Which OpenAI-compatible surface each provider implements, as of v1.5.2. This mirrors the machine-checked matrix in the gateway source (providers/README.md), which fails the build if a provider's real interface set ever drifts from it.

  • ✓ — a typed surface (Complete, Embed, Rerank, …), or a working /v1/* pass-through for the Proxy column.
  • — — not implemented, or (for Proxy) a native-wire provider whose raw /v1/* pass-through returns 501 by design. Reach those through the translated surfaces to their left.

Audio groups transcription (STT) and speech (TTS) — ✓ if a provider serves either. Rerank uses the Cohere-v2 contract; every other column uses the OpenAI contract. Every provider implements Chat and Stream by definition.

ProviderChatStreamEmbedImageAudioRerankModer.BatchResponsesProxy
AI21 Labs✓✓———————✓
Anthropic✓✓————————
Azure AI Foundry✓✓✓———————
Azure OpenAI✓✓✓✓✓——✓——
AWS Bedrock✓✓✓✓—✓————
Cerebras✓✓———————✓
Cloudflare Workers AI✓✓✓——————✓
Cohere✓✓✓——✓————
Databricks✓✓✓——————✓
DeepInfra✓✓✓✓✓✓———✓
DeepSeek✓✓———————✓
Fireworks AI✓✓✓—✓————✓
Google Gemini✓✓✓✓——————
Groq✓✓——✓——✓—✓
Hugging Face✓✓✓✓—————✓
Mistral AI✓✓✓—✓—✓——✓
Moonshot AI✓✓———————✓
Novita✓✓✓————✓—✓
NVIDIA NIM✓✓✓——✓———✓
Ollama✓✓✓——————✓
Ollama Cloud✓✓✓———————
OpenAI✓✓✓✓✓—✓✓✓✓
OpenRouter✓✓✓——————✓
Perplexity✓✓———————✓
Qwen✓✓✓————✓—✓
Replicate✓✓—✓——————
SambaNova✓✓✓—✓————✓
Together AI✓✓✓✓✓✓———✓
Google Vertex AI✓✓✓✓——————
xAI✓✓—✓————✓✓

Totals — Chat / Stream: 30 · Embeddings: 21 · Images: 10 · Audio: 9 · Rerank: 5 (bedrock, cohere, deepinfra, nvidia-nim, together) · Moderations: 2 (openai, mistral) · Batch: 5 (openai, azure-openai, groq, novita, qwen) · Responses: 2 (openai, xai) · Proxy pass-through: 21.

Nine providers show — for Proxy. Eight are native-wire — Anthropic, AWS Bedrock, Cohere, Google Gemini, Google Vertex AI, Replicate, and the two Azure surfaces — whose raw /v1/* pass-through returns 501 by design (reach them through the translated columns). Ollama Cloud is the one provider that is not proxiable at all. A — in a capability column is not a routing limitation: those providers still route every model the catalog, live discovery, or targets[].models names.

How a provider is declared​

A provider has two independent parts: credentials (from the environment) and a target (in your config). Both must be present for the provider to serve.

Credentials never live in config.yaml. Each provider reads its API key and any special settings from environment variables — OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_OPENAI_ENDPOINT, and so on. A provider is auto-registered the moment its required environment variables are present; no main.go edit and no config entry is needed to register it.

A target names a provider by virtual_key. In config, a routing target refers to a registered provider by its id — that id is the whole declaration:

strategy:
  mode: fallback
targets:
  - virtual_key: openai      # the provider id — this is the declaration
    retry:
      attempts: 3
  - virtual_key: anthropic

Only providers listed under targets[] route traffic (see the allowlist warning above). For every provider's exact environment variables, optional <PROVIDER>_BASE_URL override, and any special configuration keys, see Provider configuration.