Skip to main content

Prompt templates

Prompt templates let you define named, versioned prompts server-side and reference them by ID at request time. The gateway injects the rendered template into the request before forwarding it to the provider — clients never need to re-send long system prompts.

Defining a template​

Add a prompt_templates block to config.yaml:

prompt_templates:
  - id: customer-support
    description: "Tier-1 customer support agent"
    template: |
      You are a helpful customer support agent for {{company_name}}.
      Always respond in {{language}}.
      Escalation email: {{escalation_email}}

  - id: code-reviewer
    description: "Code review assistant"
    template: |
      You are an expert {{language}} code reviewer.
      Focus on: correctness, security, and readability.
      Be concise. Use inline code fences for suggestions.

Variable placeholders use double-brace syntax: {{variable_name}}.

Using a template in a request​

Pass template_id and template_vars in the request body alongside or instead of a system message:

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="http://localhost:8080",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "My order #12345 hasn't arrived."}],
    extra_body={
        "template_id": "customer-support",
        "template_vars": {
            "company_name": "Acme Corp",
            "language": "English",
            "escalation_email": "support-l2@acme.example",
        },
    },
)
print(response.choices[0].message.content)

The gateway renders the template, prepends it as the system message, and forwards the enriched request to the provider.

Mixing templates with system messages​

If the request already has a system message, the rendered template is prepended before the existing system message. The existing message is preserved.

# Rendered template becomes system[0]; your message becomes system[1]
messages = [
    {"role": "system", "content": "Additional context for this request."},
    {"role": "user",   "content": "Review this Go function for security issues."},
]

Listing templates via the admin API​

curl -H "Authorization: Bearer $ADMIN_API_KEY" \
  http://localhost:8080/admin/templates

Returns an array of all configured template IDs, descriptions, and variable names (extracted automatically from the template).

Best practices​

  • Keep templates in version control. Store them in config.yaml and track changes alongside your gateway configuration.
  • Use descriptive IDs. customer-support-en is clearer than tmpl-001.
  • Validate variables. The gateway returns 400 Bad Request if a required variable is missing from template_vars.
  • Avoid secrets in templates. Don't hardcode API keys or passwords — inject them from environment variables instead.