Manage the LLM Gateway

This guide builds a governed LLM Gateway setup as code: a connection and provider for an upstream vendor, model routes with a failover alias, an access policy, usage controls, pricing, and the organization’s governance settings.

Estimated time: 20–30 minutes. Complete Getting Started first. For the concepts behind budgets, rate limits, and model access, see LLM Controls.

Before You Begin

A configured provider "barndoor" block (see Getting Started), using provider version 0.9.0 or later
An API key for your upstream model vendor (this guide uses OpenAI), passed in as a sensitive variable rather than committed to configuration

Step 1: Create a connection and a provider

Upstream keys live on a connection (shown as a credential under LLM Management → Connections in the app), and providers reference the connection rather than storing a key of their own. One connection can back several providers, so rotating its key rotates it for all of them.

variable "openai_api_key" {
  type      = string
  sensitive = true
}

resource "barndoor_llm_connection" "openai" {
  name           = "OpenAI production key"
  model_provider = "openai"
  base_url       = "https://api.openai.com"
  api_key        = var.openai_api_key
}

resource "barndoor_llm_provider" "openai" {
  name           = "OpenAI"
  model_provider = "openai"
  connection_id  = barndoor_llm_connection.openai.id
}

Reference: barndoor_llm_connection, barndoor_llm_provider The provider takes its auth_type, resource settings and, when its own base_url is unset, its endpoint from the connection. Changing connection_id rebinds the provider in place. Removing it forces a replacement, because a provider can’t be detached from its credential. base_url stops at the host: don’t add /v1 for OpenAI-compatible vendors, because the gateway appends the version itself. The provider rejects a trailing /v1 at plan time and suggests the corrected URL. The connection’s api_key is write-only. The platform never returns it, and Terraform reports only its last four characters as key_last4, but the value you supply is kept in your Terraform state. Treat state as secret (see Best Practices).

Upgrading from a provider version before 0.8.0? Since platform v2.40.0, a provider can’t store its own key, so creating barndoor_llm_provider with api_key fails with a 400 (“a provider cannot store its own key”). Move the key to a barndoor_llm_connection and set connection_id. Setting api_key or auth_type together with connection_id is a plan-time error, and api_key on the provider is deprecated.

To reference a provider created in the Barndoor app instead, look it up by name with the barndoor_llm_provider data source and use data.barndoor_llm_provider.openai.id wherever this guide uses barndoor_llm_provider.openai.id:

data "barndoor_llm_provider" "openai" {
  name = "OpenAI"
}

The name match is case-insensitive. The data source also exposes connection_id and health_status, so you can surface connectivity problems as a Terraform output.

Bedrock with an IAM role

A connection can assume an IAM role instead of storing a key. The platform generates the external ID the role’s trust policy must require, and the connection exposes it in effective_settings:

resource "barndoor_llm_connection" "bedrock" {
  name           = "Bedrock (us-east-1)"
  model_provider = "bedrock"
  auth_type      = "aws_role"
  base_url       = "https://bedrock-runtime.us-east-1.amazonaws.com"
  settings = jsonencode({
    region       = "us-east-1"
    iam_role_arn = "arn:aws:iam::123456789012:role/barndoor-bedrock"
  })
}

output "bedrock_trust_policy_external_id" {
  value = jsondecode(barndoor_llm_connection.bedrock.effective_settings).external_id
}

Subscriptions and unmetered providers

Providers with a request-scoped OAuth passthrough (claude_oauth, codex_oauth) need no connection: callers bring their own subscription. Such a provider is billed flat rather than per token, so mark it not_metered. Token usage is still reported, but its cost is recorded as $0:

resource "barndoor_llm_provider" "claude_subscription" {
  name           = "Claude (subscription)"
  model_provider = "anthropic"
  auth_type      = "claude_oauth"
  base_url       = "https://api.anthropic.com"

  billing_mode   = "not_metered"
  billing_reason = "subscription"
}

not_metered requires a billing_reason (subscription, local, external or other), and the optional billing_note holds up to 200 characters of context. If your configuration never sets the billing attributes, Terraform leaves billing set in the app alone. Changing billing_mode doesn’t change the cost of usage already recorded.

Step 2: Enable models with 1:1 mappings

A model mapping makes an upstream model servable through the gateway. A 1:1 mapping (alias equals upstream model) is the enablement:

resource "barndoor_llm_model_mapping" "gpt_4o" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "gpt-5.5"
  upstream_model = "gpt-5.5"
}

Reference: barndoor_llm_model_mapping A 1:1 enablement is addressable as <provider>/gpt-5.5. Set bare_alias = true to also serve the bare name gpt-5.5. If the enablement already exists on the platform, for example because it was created in the app, applying this resource adopts and updates the existing row instead of failing.

Step 3: Add a custom alias with failover

Custom aliases decouple what your agents call from what serves it. You can re-point fast at a different model later without touching any agent:

resource "barndoor_llm_model_mapping" "fast" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "fast"
  upstream_model = "gpt-5.5"

  priority                   = 10
  retry_on_429_count         = 3
  retry_on_429_max_wait_secs = 60

  depends_on = [barndoor_llm_model_mapping.gpt_4o]
}

A custom alias requires the 1:1 enablement for its (provider, upstream_model) pair to exist first. The API rejects orphan aliases. Terraform can’t infer this ordering from the attribute values, so declare it with depends_on, as above. Without it, a fresh apply can fail intermittently depending on creation order.

Multiple mappings with the same alias across providers form a failover chain ordered by priority (lower first). The retry_on_429_* settings control how the gateway rides out upstream rate limiting before failing over: retry_on_429_count is 0–10 retries, and retry_on_429_max_wait_secs caps how long it honors Retry-After (0–180 seconds). The optional request_timeout_secs (1–600, default 120) and stream_idle_timeout_secs (1–300, default 180) set per-route timeouts. The six optional cooldown_* attributes set when a failing route is taken out of rotation, and for how long. cooldown_failure_threshold failures within cooldown_window_secs start a cooldown of cooldown_base_secs, which doubles on each failed recovery probe up to cooldown_max_secs. cooldown_429_default_secs applies to a 429 without Retry-After, and cooldown_overloaded_secs to a 529:

resource "barndoor_llm_model_mapping" "fast" {
  # ...the arguments above, plus:
  cooldown_failure_threshold = 5
  cooldown_window_secs       = 60
  cooldown_base_secs         = 30
  cooldown_max_secs          = 600
}

cooldown_failure_threshold = 0 turns cooldowns off for the route. cooldown_base_secs, cooldown_429_default_secs and a non-zero cooldown_overloaded_secs can’t exceed cooldown_max_secs, and the provider checks this at plan time. Unset attributes take the platform defaults on create and keep their stored value afterwards, so a policy tuned in the app isn’t reset. Cooldown attributes need platform v2.43.0 or later. See Failover, Cooldowns, and Route Health for how cooldowns work.

Step 4: Restrict which models can be used

An access policy allowlists (or denylists) models for a scope, such as the whole organization, an IdP group, or a single user:

resource "barndoor_llm_model_access" "frontier_only" {
  name        = "Frontier models only"
  scope_type  = "org"
  policy_type = "allowlist"

  targets = [
    { kind = "model_alias", alias = "gpt-*" },
  ]
}

Reference: barndoor_llm_model_access A target can name a model alias pattern (as above), a specific upstream model (kind = "model"), an entire provider (kind = "provider"), a provider and alias pair (kind = "provider_model"), or a route group (kind = "route_group"). Policies apply to LLM traffic unless you set traffic_type. A route group is a named set of route aliases that policies can grant or deny as one:

resource "barndoor_llm_model_route_group" "approved" {
  name          = "Approved for production"
  model_aliases = ["fast", "gpt-5.5"]
}

resource "barndoor_llm_model_access" "production_agents" {
  name        = "Production agents: approved models"
  scope_type  = "group"
  scope_value = "production-agents"
  policy_type = "allowlist"

  targets = [
    { kind = "route_group", group_id = barndoor_llm_model_route_group.approved.id },
  ]
}

Reference: barndoor_llm_model_route_group The platform also changes a group’s membership on its own. Renaming a route’s alias moves the alias’s memberships, and deleting a route’s last mapping removes the alias from every group. Terraform reads model_aliases back, so either change shows up as a diff on the next plan. The platform doesn’t check that a targeted group exists: an allowlist whose only target is an empty or deleted group denies every model.

Step 5: Add rate limits

Rate limits throttle a scope over a rolling 60-second window:

resource "barndoor_llm_rate_limit" "org_ceiling" {
  name                = "Org ceiling"
  scope_type          = "org"
  requests_per_minute = 600
}

resource "barndoor_llm_rate_limit" "engineering_tokens" {
  name              = "Engineering token ceiling"
  scope_type        = "group"
  scope_value       = "engineering"
  tokens_per_minute = 250000
  traffic_type      = "llm"
}

Reference: barndoor_llm_rate_limit

At least one of requests_per_minute / tokens_per_minute is required, and removing one from your configuration clears that metric on the platform rather than leaving it unchanged. One rate limit exists per scope, traffic type and set of targets; creating a duplicate fails with a conflict.

Don’t use the llm_provider or model scope types to limit one provider or model: the platform accepts them only for compatibility, and they don’t match any traffic. Narrow the rule with a target instead:

resource "barndoor_llm_rate_limit" "fast_alias" {
  name              = "Fast alias ceiling"
  scope_type        = "org"
  tokens_per_minute = 100000
  traffic_type      = "llm"

  target_model_alias = "fast"
}

There are four targets:

target_provider_id narrows the rule to one provider. Add target_upstream_model to narrow it to one of that provider’s models.
target_model_alias narrows it to one alias, matched against what the caller sent before routing.
target_mcp_server_id narrows it to one MCP server’s tool traffic. It needs traffic_type mcp or all.

An alias target can’t be combined with a provider target, and an MCP target can’t be combined with a model target. The provider checks these combinations at plan time. Rate-limit targets change in place, and removing one clears it. A target is part of a rule’s identity, so an org-wide ceiling and a ceiling on one alias can coexist.

Step 6: Set budgets

Budgets cap total token consumption over a day, week, or month, the spend-control counterpart to rate limits:

resource "barndoor_llm_token_budget" "org_monthly" {
  name        = "Org monthly cap"
  scope_type  = "org"
  period      = "monthly"
  token_limit = 500000000
}

resource "barndoor_llm_token_budget" "contractors_weekly" {
  name        = "Contractors weekly cap"
  scope_type  = "group"
  scope_value = "contractors"
  period      = "weekly"
  token_limit = 5000000

  alert_thresholds  = [50, 75, 95]
  action_on_exhaust = "warn"
  traffic_type      = "llm"
}

Reference: barndoor_llm_token_budget Alert thresholds default to [80, 90] (percent), and action_on_exhaust defaults to block. warn lets traffic continue while notifying. One budget exists per scope, traffic type, period and set of targets. Periods reset at UTC boundaries: daily at 00:00, weekly on Monday, monthly on the 1st. To give every member of an IdP group their own allowance from a single rule, use a per-user budget with member_of_group. scope_type = "group" is the opposite: one allowance shared by the whole group.

resource "barndoor_llm_token_budget" "contractor_each_weekly" {
  name            = "Per-contractor weekly cap"
  scope_type      = "user"
  member_of_group = "contractors"
  period          = "weekly"
  token_limit     = 1000000
}

member_of_group requires scope_type = "user" with neither scope_id nor scope_value set. Someone who joins the group picks up the allowance on their next request. Changing or removing it replaces the budget. barndoor_llm_rate_limit takes the same attribute, for per-member rate limits. A budget can cap spend instead of, or as well as, tokens. Spend is computed from your pricing rules (Step 7). Set cost_limit without token_limit for a cost-only budget. Budgets take the same target_* attributes as rate limits:

resource "barndoor_llm_token_budget" "fast_daily_spend" {
  name               = "Per-user daily spend on fast"
  scope_type         = "user"
  period             = "daily"
  cost_limit         = 25
  traffic_type       = "llm"
  target_model_alias = "fast"
}

cost_limit is in currency (default USD), with at most four decimal places. Removing a limit clears it in place. Unlike on rate limits, a budget’s targets and currency can’t be updated, so changing one replaces the budget.

Step 7: Price your models

Pricing rules set the per-million-token costs that usage reports and cost figures are calculated from:

resource "barndoor_llm_model_pricing" "gpt55" {
  model_pattern  = "gpt-5.5"
  model_provider = "openai"

  input_cost_per_million_tokens  = 5
  output_cost_per_million_tokens = 30

  change_reason = "Managed by Terraform"
}

Reference: barndoor_llm_model_pricing For models whose vendor charges more for long prompts, add a long_context tier. A request whose prompt exceeds threshold_prompt_tokens bills all its tokens at the tier’s rates:

resource "barndoor_llm_model_pricing" "gemini_pro" {
  model_pattern  = "gemini-2.5-pro"
  model_provider = "google_ai"

  input_cost_per_million_tokens  = 1.25
  output_cost_per_million_tokens = 10

  long_context = {
    threshold_prompt_tokens        = 200000
    input_cost_per_million_tokens  = 2.5
    output_cost_per_million_tokens = 15
  }
}

Provider versions before 0.9.0 drop long-context tiers. Each price change appends a version that copies nothing from the one before it, and older versions never sent the tier. So a price change from Terraform deleted any tier set in the app. From 0.9.0 the tier is sent with every version, and a tier set in the app shows up as a diff.

Pricing is versioned on the platform. Each price change appends a new version, so the resource’s id changes on update by design. Set a future effective_from to schedule a change. terraform destroy archives the rule, keeping its history, rather than deleting it. Leave sync_mode at its default, pinned, so Barndoor’s default-pricing updates never change a price Terraform owns. See Manage Model Pricing for how rules are matched.

Step 8: Route requests with a routing policy

A routing policy is a caller-facing alias that picks one of several model slots for each request, instead of always calling one model. A small determiner model reads the request and chooses a slot. Routing rules, written in plain English, then set a minimum slot or ban slots for particular kinds of work:

resource "barndoor_llm_routing_policy" "auto" {
  model_alias            = "auto"
  determiner_model_alias = "fast"
  posture                = "balanced"

  slots = [
    { label = "Everyday", model_alias = "fast" },
    { label = "Strong", model_alias = "OpenAI/gpt-5.5" },
  ]
  context_breakpoints = [128000]

  depends_on = [barndoor_llm_model_mapping.fast]
}

resource "barndoor_llm_routing_rule" "contracts" {
  policy_id   = barndoor_llm_routing_policy.auto.id
  name        = "Contract review"
  description = "Contract review, legal analysis, or anything a customer will sign must use the strong slot."
  floor_slot  = 1
}

Reference: barndoor_llm_routing_policy, barndoor_llm_routing_rule Callers send auto as the request model. Each slot’s model_alias must already resolve to enabled routes. That means either a bare-callable alias, such as the custom fast alias from Step 3, or the provider-scoped form <provider name>/<model> of a 1:1 enablement. context_breakpoints needs one entry fewer than the number of slots. Its default, [128000, 512000], only fits three slots, so set it explicitly for any other count. The provider checks the slot, breakpoint and default-slot rules at plan time. Rules refer to slots by index. List slots from cheapest to strongest, and floor_slot = 1 then means “at least the second slot”. The platform doesn’t check the order against prices. Use deny_slots to ban slots. Rule conflicts the platform reports appear as warnings and don’t fail the apply. Import a rule with <policy_id>/<rule_id>.

Step 9: Governance settings

barndoor_llm_governance_config manages the organization’s single LLM Gateway governance configuration. With require_pricing_for_mappings = true, the gateway refuses to enable a model route that no pricing rule matches:

resource "barndoor_llm_governance_config" "org" {
  require_pricing_for_mappings = true

  depends_on = [barndoor_llm_model_pricing.gpt55]
}

Reference: barndoor_llm_governance_config Two optional attributes manage the other settings:

default_model_access: allow (the default) or deny. deny is Require an explicit grant to use a model on the Model Access page. It requires an enabled allowlist, such as the one in Step 4.
require_routing_policy: Require requests to use a routing policy on the Routing Policies page, so callers must send a policy alias such as auto from Step 8. The /v1/systemone classifier endpoint is exempt.

An attribute you don’t set keeps whatever is stored, so you can manage pricing enforcement in Terraform and leave the other settings to the app.

Provider versions before 0.8.0 reset these settings on every apply. They sent only require_pricing_for_mappings, and the platform replaces the whole configuration on each save, so an organization set to deny in the app went back to allow. Upgrade before managing this resource in an organization that uses either setting.

terraform destroy can’t delete the configuration. It resets only require_pricing_for_mappings, so removing the resource never loosens model access.

Step 10: Issue API keys and limit their lifetime

Apps and agents call the gateway with a Barndoor API key. barndoor_llm_api_key issues one: here a service key bound to an IdP group, so the group’s budgets, rate limits and model access apply to it:

resource "barndoor_llm_api_key" "billing" {
  name       = "billing-service"
  group_name = "engineering"
  expires_at = "2027-06-30T23:59:59Z"

  # Change this value to rotate the secret in place.
  rotation_trigger = {
    rotated = "2026-11"
  }
  grace_period_secs = 3600
}

output "billing_api_key" {
  value     = barndoor_llm_api_key.billing.key
  sensitive = true
}

Reference: barndoor_llm_api_key

The key’s secret is stored in your Terraform state. The platform returns it only when the key is created or rotated, so key is kept in state, marked sensitive, for you to pass on, for example to a secrets manager. Use an encrypted remote backend with restricted access (see Secrets and state hygiene).

Rotation: changing rotation_trigger issues a new secret in place. The key keeps its ID, agent binding and usage history. With grace_period_secs (up to 7 days), the previous secret keeps working for that long so clients can move over. A time_rotating resource’s ID makes a good trigger for scheduled rotation.
In-place changes: expires_at and disabled change without a new secret, and extending an expired key reactivates it. Changing name, user_id, group_name or scopes creates a new key with a new secret.
Destroy revokes the key.

To limit how long keys that people create for themselves may live, set an organization default and, where needed, exceptions for a group or a user:

# Every self-serve key gets exactly 60 days; users don't pick an expiry.
resource "barndoor_llm_api_key_lifetime_policy" "org" {
  max_lifetime_days = 60
  fixed_lifetime    = true
}

# Contractors' keys expire within a week.
resource "barndoor_llm_api_key_lifetime_group_override" "contractors" {
  group_name        = "contractors"
  max_lifetime_days = 7
}

References: barndoor_llm_api_key_lifetime_policy, barndoor_llm_api_key_lifetime_group_override, barndoor_llm_api_key_lifetime_user_override A user’s own override wins. Otherwise the strictest of their groups’ overrides applies, and otherwise the default. An override with no limits exempts its group or user. Keys created by admins, including those Terraform creates, are exempt, and existing keys keep their expiry: the policy is checked when a key is created or rotated. These resources need provider 0.11.0 and platform v2.52.0 or later.

Behavior worth knowing

Scope immutability: changing the scope of a budget forces a replacement. On model access policies and rate limits, removing scope_id or scope_value also forces a new resource.
Two-step creates: some LLM Gateway resources are created with a follow-up update under the hood, because not every field is settable on create. If an apply is interrupted at exactly the wrong moment, a partially configured object can exist. A re-apply converges it, or the next plan shows the difference.
Verify in the app: everything in this guide is visible under LLM Management in the Barndoor app. Model routes and pricing are in the Models section, and budgets, rate limits, and model access are under Controls (see LLM Controls). API keys and key policies are under Access.

Complete example

Show Full configuration from this guide

variable "openai_api_key" {
  type      = string
  sensitive = true
}

resource "barndoor_llm_connection" "openai" {
  name           = "OpenAI production key"
  model_provider = "openai"
  base_url       = "https://api.openai.com"
  api_key        = var.openai_api_key
}

resource "barndoor_llm_provider" "openai" {
  name           = "OpenAI"
  model_provider = "openai"
  connection_id  = barndoor_llm_connection.openai.id
}

resource "barndoor_llm_model_mapping" "gpt_4o" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "gpt-5.5"
  upstream_model = "gpt-5.5"
}

resource "barndoor_llm_model_mapping" "fast" {
  provider_id    = barndoor_llm_provider.openai.id
  model_alias    = "fast"
  upstream_model = "gpt-5.5"

  priority                   = 10
  retry_on_429_count         = 3
  retry_on_429_max_wait_secs = 60

  depends_on = [barndoor_llm_model_mapping.gpt_4o]
}

resource "barndoor_llm_model_access" "frontier_only" {
  name        = "Frontier models only"
  scope_type  = "org"
  policy_type = "allowlist"

  targets = [
    { kind = "model_alias", alias = "gpt-*" },
  ]
}

resource "barndoor_llm_rate_limit" "org_ceiling" {
  name                = "Org ceiling"
  scope_type          = "org"
  requests_per_minute = 600
}

resource "barndoor_llm_token_budget" "org_monthly" {
  name        = "Org monthly cap"
  scope_type  = "org"
  period      = "monthly"
  token_limit = 500000000
}

resource "barndoor_llm_model_pricing" "gpt55" {
  model_pattern  = "gpt-5.5"
  model_provider = "openai"

  input_cost_per_million_tokens  = 5
  output_cost_per_million_tokens = 30

  change_reason = "Managed by Terraform"
}

resource "barndoor_llm_governance_config" "org" {
  require_pricing_for_mappings = true

  depends_on = [barndoor_llm_model_pricing.gpt55]
}

resource "barndoor_llm_api_key" "billing" {
  name       = "billing-service"
  group_name = "engineering"
  expires_at = "2027-06-30T23:59:59Z"

  rotation_trigger = {
    rotated = "2026-11"
  }
  grace_period_secs = 3600
}

output "billing_api_key" {
  value     = barndoor_llm_api_key.billing.key
  sensitive = true
}

resource "barndoor_llm_api_key_lifetime_policy" "org" {
  max_lifetime_days = 60
  fixed_lifetime    = true
}

resource "barndoor_llm_api_key_lifetime_group_override" "contractors" {
  group_name        = "contractors"
  max_lifetime_days = 7
}

Get Started

Use-Case Guides

Operations

Before You Begin

Step 1: Create a connection and a provider

Bedrock with an IAM role

Subscriptions and unmetered providers

Step 2: Enable models with 1:1 mappings

Step 3: Add a custom alias with failover

Step 4: Restrict which models can be used

Step 5: Add rate limits

Step 6: Set budgets

Step 7: Price your models

Step 8: Route requests with a routing policy

Step 9: Governance settings

Step 10: Issue API keys and limit their lifetime

Behavior worth knowing

Complete example

​Before You Begin

​Step 1: Create a connection and a provider

​Bedrock with an IAM role

​Subscriptions and unmetered providers

​Step 2: Enable models with 1:1 mappings

​Step 3: Add a custom alias with failover

​Step 4: Restrict which models can be used

​Step 5: Add rate limits

​Step 6: Set budgets

​Step 7: Price your models

​Step 8: Route requests with a routing policy

​Step 9: Governance settings

​Step 10: Issue API keys and limit their lifetime

​Behavior worth knowing

​Complete example

Before You Begin

Step 1: Create a connection and a provider

Bedrock with an IAM role

Subscriptions and unmetered providers

Step 2: Enable models with 1:1 mappings

Step 3: Add a custom alias with failover

Step 4: Restrict which models can be used

Step 5: Add rate limits

Step 6: Set budgets

Step 7: Price your models

Step 8: Route requests with a routing policy

Step 9: Governance settings

Step 10: Issue API keys and limit their lifetime

Behavior worth knowing

Complete example