Skip to main content

Conditional routing strategy

Conditional routes each request by matching a field of the request against rules you write, not by weight, latency, or cost. It optimizes for deterministic pinning — "this model always goes to this backend", "this tenant's traffic only ever goes here" — the shape a compliance or contractual requirement needs, where a pool mode's willingness to pick a different provider is the wrong behaviour, not a convenience. Set strategy.mode: conditional to use it.

In plain words

Write rules like "model X goes to provider Y" or "user vip goes to provider Z". The first matching rule wins; anything unmatched goes to the first target. A rule names either one target or an ordered chain of targets (target_keys). The gateway tries the chain in order, moving on only when a provider is at fault, and never reaches for a target the rule did not name. A rule with one target is exact: if that target is down, the client gets the corresponding error.

What happens to a request​

With the rules in the YAML below (gpt-4o → OpenAI, claude- prefix → Anthropic then Bedrock, user vip → OpenAI, OpenAI first in targets):

Client asks forRuleChainIf the first target fails
gpt-4oexact model matchOpenAIthe client gets OpenAI's error
claude-sonnet-4-6prefix claude-Anthropic, then BedrockBedrock answers after a provider-side failure (5xx, timeout, open circuit); a 400 from Anthropic goes back to the client
any model, "user": "vip" after the model rulesuser matchOpenAIthe client gets OpenAI's error
mistral-largeno matchOpenAI (targets[0])404 model_not_found if OpenAI does not serve it — even if another target does

Behaviour​

conditional is a named mode: the candidates for a request are exactly what the matched rule names — its target_keys chain, or target_key as a one-entry chain — and nothing else. Conditional.SelectTargets (internal/strategies/conditional.go) evaluates strategy.conditions[] in declared order; the first rule whose key/value matches the request wins, and when none matches, the no-match fallback (targets[0]) is the whole answer.

The pipeline walks a chain the way it walks a pool: it advances to the next member only after a failover-safe failure (a transport failure, an attempt that timed out waiting on the target, 408, 429, 5xx, a provider's own context-length overflow, an open circuit, or a full concurrency queue), skips a member whose circuit is open or that is parked after a 429, and returns any other 4xx to the client. What it never does is substitute a target outside the chain. A rule with one target is therefore exact: a model it does not serve, a surface it cannot handle, or an open circuit is the corresponding error (404, 404, 503) rather than a sibling's answer.

The matcher key is a closed set, validated at load: a key outside it is a ferrogw validate / startup error, not a live-request failure. Since v1.5.2 the set covers the request's shape as well as its model, and one allow-listed header — no other request header ever reaches a rule.

keyMatches whenvalue
modelthe request's model equals value exactlya model name
model_prefixthe request's model starts with valuea prefix such as claude-
userthe request's user field equals valuea user id
streamthe request is ("true") or is not ("false") a streaming request"true" | "false"
has_toolsthe request carries ("true") or does not carry ("false") a tools array"true" | "false"
metadatathe entry named by field in the X-Gateway-Metadata request header equals valuea string

X-Gateway-Metadata is a JSON object of at most 32 string, number or boolean values within 4 KiB, accepted on /v1/chat/completions and /v1/completions, and never forwarded to a provider; a malformed header is the caller's 400. user also applies to embeddings and image requests, which carry the field; stream, has_tools and metadata are chat-only and match nothing on the other surfaces.

Config keys​

KeyTypeDefaultDescription
strategy.modestring—Set to conditional.
strategy.conditions[].keystring—model | model_prefix | user | stream | has_tools | metadata. Closed set — an unrecognized value is a load error.
strategy.conditions[].valuestring—What key is matched against. stream and has_tools accept only "true" or "false".
strategy.conditions[].fieldstring—The metadata entry a key: metadata rule reads. Required there, refused elsewhere.
strategy.conditions[].target_keystring—The one target this rule routes to; must name a configured targets[].virtual_key. Sugar for a one-entry target_keys.
strategy.conditions[].target_keys[]string—The rule's ordered target chain. Every entry must be a declared target, none may repeat, and exactly one of target_key and target_keys is set.

Minimal working YAML​

strategy:
  mode: conditional
  conditions:
    - key: model
      value: gpt-4o
      target_key: openai
    - key: model_prefix
      value: claude-
      target_keys: [anthropic, bedrock]   # Anthropic first; Bedrock stands in only for a provider-side failure
    - key: user
      value: vip
      target_key: openai
    - key: metadata
      field: tier
      value: gold
      target_key: openai

targets:
  - virtual_key: openai   # targets[0] doubles as the no-match fallback
  - virtual_key: anthropic
  - virtual_key: bedrock

When to use​

  • Deterministic pinning: a specific model, tenant or request shape must always be served by a specific backend, for compliance, contractual, or data-residency reasons — not "prefer this provider," but "only this provider."
  • Routing whole model families by prefix (claude-, gpt-, gemini-) to their native provider without listing every model name individually.
  • A pinned rule that still needs a stand-in: name the stand-in in target_keys, and it is used only when the preferred member is at fault.
  • Any setup where fallback's or loadbalance's willingness to pick a different provider is the behaviour you need to prevent, not add.

Gotchas​

  • A model the matched chain doesn't serve is 404 model_not_found, even when another configured target serves it. The rule is a decision about which targets handle this request, not a preference among several. Reaching another target means naming it in the rule.
  • A one-target rule whose target is down answers 503, not a sibling. Before v1.5.2 an open circuit on the matched target borrowed a healthy sibling from targets[]; it no longer does. Put the sibling in target_keys if you want it used.
  • GET /v1/models mirrors the same restriction. A model missing from that listing under this mode is the signal that a rule is needed for it, before a request ever hits the 404.
  • The no-match fallback is targets[0], alone. Put the target you want unmatched requests to land on first.
  • The condition-key set is closed on purpose. Anything outside the six keys above — a typo, a field name from another mode, an arbitrary header — is rejected at load, not silently ignored or routed to the fallback.
  • The chain is walked like a pool, and stops at its end. Retry (targets[].retry) re-asks a member before the walk moves on; a member whose circuit is open or that is parked after a 429 is passed over; a deterministic 4xx from a member stops the walk; a stream that has begun is never failed over mid-stream.