Manage the LLM Gateway

This guide builds a governed LLM Gateway setup as code: an upstream provider, model routes with a failover alias, an access policy, and usage controls.

Estimated time: 20–30 minutes. Complete Getting Started first. For the concepts behind budgets, rate limits, and model access, see LLM Controls.

Before You Begin

A configured provider "barndoor" block (see Getting Started)
An API key for at least one upstream LLM provider (this guide uses OpenAI)

Step 1: Add an upstream provider

resource "barndoor_llm_provider" "openai" {
  name           = "OpenAI"
  model_provider = "openai"
  base_url       = "https://api.openai.com/v1"
  api_key        = var.openai_api_key
}

variable "openai_api_key" {
  type      = string
  sensitive = true
}

Reference: barndoor_llm_provider The api_key is write-only: the platform stores it in its secret store and never returns it. Keep it in a variable or TF_VAR_openai_api_key, never in committed configuration.

To stage a provider without routing traffic to it yet, set enabled = false. The gateway also health-checks providers before routing to them; enforce_health_check = false bypasses that gate for providers that don’t answer probes.

Step 2: Enable models with 1:1 mappings

A model mapping makes an upstream model servable through the gateway. A 1:1 mapping (alias equals upstream model) is the enablement:

resource "barndoor_llm_model_mapping" "gpt_4o" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "gpt-4o"
  upstream_model = "gpt-4o"
}

Reference: barndoor_llm_model_mapping

Step 3: Add a custom alias with failover

Custom aliases decouple what your agents call from what serves it — you can re-point fast at a different model later without touching any agent:

resource "barndoor_llm_model_mapping" "fast" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "fast"
  upstream_model = "gpt-4o"

  priority                   = 10
  retry_on_429_count         = 3
  retry_on_429_max_wait_secs = 60

  depends_on = [barndoor_llm_model_mapping.gpt_4o]
}

A custom alias requires the 1:1 enablement for its (provider, upstream_model) pair to exist first — the API rejects orphan aliases. Terraform can’t infer this ordering from the attribute values, so declare it with depends_on, as above. Without it, a fresh apply can fail intermittently depending on creation order.

Multiple mappings with the same alias across providers form a failover chain ordered by priority (lower first). The retry_on_429_* settings control how the gateway rides out upstream rate limiting before failing over.

Step 4: Restrict which models can be used

An access policy allowlists (or denylists) models for a scope — the whole organization, an IdP group, or a single user:

resource "barndoor_llm_model_access" "frontier_only" {
  name        = "Frontier models only"
  scope_type  = "org"
  policy_type = "allowlist"

  targets = [
    { kind = "model_alias", alias = "gpt-*" },
  ]
}

Reference: barndoor_llm_model_access Targets can name a model alias pattern (as above), a specific upstream model (kind = "model"), an entire provider (kind = "provider"), or a provider+model pair.

Step 5: Add rate limits

Rate limits throttle a scope over a rolling 60-second window:

resource "barndoor_llm_rate_limit" "org_ceiling" {
  name                = "Org ceiling"
  scope_type          = "org"
  requests_per_minute = 600
}

resource "barndoor_llm_rate_limit" "engineering_tokens" {
  name              = "Engineering token ceiling"
  scope_type        = "group"
  scope_value       = "engineering"
  tokens_per_minute = 250000
  traffic_type      = "llm"
}

Reference: barndoor_llm_rate_limit

At least one of requests_per_minute / tokens_per_minute is required — and removing one from your configuration clears that metric on the platform rather than leaving it unchanged. One rate limit exists per (scope, traffic type); creating a duplicate fails with a conflict.

Step 6: Set token budgets

Budgets cap total consumption over a day, week, or month — the spend-control counterpart to rate limits:

resource "barndoor_llm_token_budget" "org_monthly" {
  name        = "Org monthly cap"
  scope_type  = "org"
  period      = "monthly"
  token_limit = 500000000
}

resource "barndoor_llm_token_budget" "contractors_weekly" {
  name        = "Contractors weekly cap"
  scope_type  = "group"
  scope_value = "contractors"
  period      = "weekly"
  token_limit = 5000000

  alert_thresholds  = [50, 75, 95]
  action_on_exhaust = "warn"
  traffic_type      = "llm"
}

Reference: barndoor_llm_token_budget Alert thresholds default to [80, 90] (percent), and action_on_exhaust defaults to block; the softer warn lets traffic continue while notifying. One budget exists per (scope, traffic type, period).

Behavior worth knowing

Scope immutability: the scope of a token budget forces replacement when changed; on model access policies, clearing a group/user scope back to org-wide also forces a new policy.
Two-step creates: some LLM Gateway resources are created with a follow-up update under the hood (not every field is settable on create). If an apply is interrupted at exactly the wrong moment, a partially-configured object can exist — a re-apply converges it, or the next plan shows the difference.
Verify in the app: everything in this guide is visible in the Barndoor app’s LLM Management hub — providers and routes under their sections, and budgets, rate limits, and model access under Controls (see LLM Controls).

Complete example

Show Full configuration from this guide

resource "barndoor_llm_provider" "openai" {
  name           = "OpenAI"
  model_provider = "openai"
  base_url       = "https://api.openai.com/v1"
  api_key        = var.openai_api_key
}

resource "barndoor_llm_model_mapping" "gpt_4o" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "gpt-4o"
  upstream_model = "gpt-4o"
}

resource "barndoor_llm_model_mapping" "fast" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "fast"
  upstream_model = "gpt-4o"

  priority                   = 10
  retry_on_429_count         = 3
  retry_on_429_max_wait_secs = 60

  depends_on = [barndoor_llm_model_mapping.gpt_4o]
}

resource "barndoor_llm_model_access" "frontier_only" {
  name        = "Frontier models only"
  scope_type  = "org"
  policy_type = "allowlist"

  targets = [
    { kind = "model_alias", alias = "gpt-*" },
  ]
}

resource "barndoor_llm_rate_limit" "org_ceiling" {
  name                = "Org ceiling"
  scope_type          = "org"
  requests_per_minute = 600
}

resource "barndoor_llm_token_budget" "org_monthly" {
  name        = "Org monthly cap"
  scope_type  = "org"
  period      = "monthly"
  token_limit = 500000000
}

variable "openai_api_key" {
  type      = string
  sensitive = true
}

Get Started

Use-Case Guides

Operations

Before You Begin

Step 1: Add an upstream provider

Step 2: Enable models with 1:1 mappings

Step 3: Add a custom alias with failover

Step 4: Restrict which models can be used

Step 5: Add rate limits

Step 6: Set token budgets

Behavior worth knowing

Complete example

​Before You Begin

​Step 1: Add an upstream provider

​Step 2: Enable models with 1:1 mappings

​Step 3: Add a custom alias with failover

​Step 4: Restrict which models can be used

​Step 5: Add rate limits

​Step 6: Set token budgets

​Behavior worth knowing

​Complete example

Before You Begin

Step 1: Add an upstream provider

Step 2: Enable models with 1:1 mappings

Step 3: Add a custom alias with failover

Step 4: Restrict which models can be used

Step 5: Add rate limits

Step 6: Set token budgets

Behavior worth knowing

Complete example