Connecting a Self-Hosted Model - Barndoor Documentation

If your model server speaks an OpenAI-compatible API, the Barndoor LLM Gateway can route to it. That covers vLLM, Ollama, SGLang, LM Studio, llama.cpp’s llama-server, and in-house wrappers alike. Connecting one is configuration on both sides — no adapter, no SDK, no code. What you get for doing it: every request to your own GPUs carries the same user identity, budget enforcement, rate limits, and audit record as a request to OpenAI or Anthropic. Your developers point at one endpoint and stop caring which side of the line a model lives on. And if your self-hosted capacity runs out, a model route can fall back to a hosted provider automatically — see Provider Failover.

This guide assumes you already have an admin account and know your way around LLM Management. If not, read Using the LLM Gateway first.

Only Step 1 differs between servers — everything after it is identical, because Barndoor sees the same OpenAI-compatible endpoint either way.

Before you start

You’ll need three things:

A running model server, reachable over the network from Barndoor (see Making the server reachable — this is the step that trips people up).
The API key you started that server with.
Admin access to LLM Management in the Barndoor portal.

Step 1: Start your server so Barndoor can reach it

Two defaults bite on nearly every model server, and both must change before Barndoor can use it:

It listens on loopback only, so nothing off that machine can connect.
It requires no credential, so anything that can connect may use your GPUs freely.

vLLM
Ollama
Anything else

vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --port 8000 --api-key <your-secret-token>

--host defaults to 127.0.0.1. Without --host 0.0.0.0, vLLM binds to loopback only and every connection from outside that machine — including Barndoor’s — is refused. This is the single most common cause of a provider that saves as unhealthy.

--api-key turns on bearer-token authentication for the /v1 paths. Treat the value as a secret: it is the only thing standing between your GPUs and anyone who can reach the port. Barndoor stores it encrypted and never exposes it to end users.Flags worth knowing about:

--served-model-name — control the name clients use

By default vLLM advertises the model under its Hugging Face repo id, so vllm serve Qwen/Qwen3-8B reports Qwen/Qwen3-8B in /v1/models and expects that exact string in the model field of a request.--served-model-name qwen3-8b overrides that with something friendlier. Whatever you choose, it must match what you enable in Barndoor in Step 3 — the gateway passes the name through untouched.

--enable-auto-tool-choice / --tool-call-parser — tool calling

vLLM only emits OpenAI-style tool_calls when the server was started for it, and the correct parser depends on the model family:

vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --api-key <token> \
  --enable-auto-tool-choice --tool-call-parser hermes

Without these flags the model will describe a tool call in prose instead of returning a structured one, and clients that expect tool use will appear to hang or loop. Barndoor passes tools through unchanged either way — it can’t compensate for a server that wasn’t started for tool calling.

--reasoning-parser — separate reasoning from the answer

For reasoning models, --reasoning-parser qwen3 (or deepseek_r1, depending on the family) splits thinking tokens into their own field rather than leaving them inline in content. Recommended if your clients render responses directly to users.

--max-model-len — cap context length

Sets the maximum combined prompt and output length. If unspecified, vLLM derives it from the model config, which can be more than your GPUs can actually hold. Setting it explicitly turns an out-of-memory crash into a clean HTTP error that Barndoor reports and can fail over from.

Ollama binds 127.0.0.1:11434 by default. Override it with an environment variable rather than a flag:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

Then pull whatever you intend to serve — ollama pull llama3.2:1b. The name in /v1/models is the Ollama tag (llama3.2:1b), not a Hugging Face repo id.

Ollama has no authentication and ignores the Authorization header entirely. Any key is accepted, so anything that can reach the port can use your hardware. Barndoor still requires an API key on the provider, but Ollama will not check it.Do not expose Ollama beyond a trusted network on its own. Put a reverse proxy in front that enforces a bearer token on /v1 and forwards to Ollama, and point Barndoor at the proxy. brew services start ollama is also not a substitute for the command above — the service definition doesn’t set OLLAMA_HOST, so it binds loopback.

Ollama is a strong fit for a shared on-prem box or an edge deployment. For multi-GPU production serving, vLLM or SGLang are the better targets.

The gateway needs three things from your server, whatever it is:

It listens on an address Barndoor can reach — usually meaning bind 0.0.0.0 rather than loopback.
It serves /v1/models and /v1/chat/completions off a host root, in OpenAI format.
It accepts a bearer token on those paths. If it can’t, put a reverse proxy in front that does and point Barndoor at the proxy.

SGLang, LM Studio, llama.cpp’s llama-server, TGI and in-house FastAPI wrappers all satisfy the first two out of the box; check your server’s own documentation for the bind address and auth flags, since those are the two that differ. Everything from Step 2 onward is identical regardless.

Confirm the server answers before you go anywhere near the portal:

curl -s http://<your-host>:8000/v1/models -H "Authorization: Bearer <your-secret-token>"

You should get a JSON list containing the model id you intend to use. Run this from a different machine than the one hosting the server — running it locally passes even when the loopback-binding problem above is present, which is exactly how that problem stays hidden.

Making the server reachable

Barndoor connects to your server over the network like any other client, which means the gateway has to be able to route to it. This is independent of which server you run.

Barndoor SaaS
Self-hosted Barndoor

A server on a private subnet is not reachable from app.barndoor.ai. You need to give it a routable address:

Put it behind your own load balancer or ingress with a public DNS name and TLS.
Restrict who can reach that address — an IP allowlist for Barndoor’s egress addresses, or mutual TLS at your edge. --api-key is authentication, not network isolation.

The certificate must be issued by a publicly trusted CA. The gateway validates TLS and will reject a self-signed or private-CA certificate — it surfaces as could not reach upstream provider, which reads like a connectivity problem rather than a certificate one.

When Barndoor runs inside your own infrastructure, no public exposure is needed. Use whatever address the gateway can resolve — a Kubernetes service DNS name such as http://vllm.ml.svc.cluster.local:8000, or a private VPC address.Plain http:// is accepted, so an internal deployment doesn’t need certificates.

Step 2: Add the provider

In the Barndoor portal, go to LLM Management → Providers → Add and choose Custom Provider — not one of the named vendor cards. That’s the right choice rather than a compromise. The named cards exist to carry facts about a vendor’s hosted endpoint: its URL, the models it serves, its per-token prices. None of those are knowable for a server you run yourself — a card for one could only ever hold a placeholder URL, an empty model list, and no pricing. The Custom Provider flow asks you for the three things that actually vary and assumes nothing else, which is why it works the same for every server in Step 1.

Field	Value
Name	Anything recognizable — `vLLM production`, `Qwen cluster`. This is what appears in usage reports.
Model Provider	Fixed at Custom (OpenAI-compatible). Not editable, and correct for any OpenAI-compatible server.
Base URL	Your server’s host root, for example `https://llm.internal.example.com` or `http://10.0.4.12:8000`.
API Key	The value you passed to `--api-key`.

The base URL must not end in /v1.vLLM’s own documentation shows http://localhost:8000/v1 because that’s what OpenAI SDK clients expect. Barndoor is not an SDK client — it appends the version segment itself, requesting {base_url}/v1/models and {base_url}/v1/chat/completions. A /v1 on the end produces /v1/v1/models, which 404s. The provider then saves as unverified rather than failing outright, so this mistake is quiet until the first real request.

On save, Barndoor runs a connectivity check: a GET {base_url}/v1/models with your key, timing out after 10 seconds. The result is recorded as the provider’s health:

Health	What it means
Healthy	The server answered and accepted the key. You’re done here.
Unhealthy	Barndoor couldn’t reach the server, or it returned an error. See Troubleshooting.
Unverified	Barndoor reached something, but no model list at that path — almost always the `/v1` suffix above.

An unhealthy provider is automatically suspended from routing while it stays that way, and resumes on its own once a later check succeeds. You can override this per provider with Block traffic on failed health check, which is worth turning off only if your server is behind something that blocks the model-list endpoint but serves completions fine.

Step 3: Enable the models

Open the provider and choose Add Models, then type your model name in. A custom provider has no catalog to pick from, which is the honest state of affairs — only your server knows what it loaded. The name must match what /v1/models reports, exactly. Take it from the server rather than from memory:

curl -s https://llm.internal.example.com/v1/models \
  -H "Authorization: Bearer <your-secret-token>"

That’s the Hugging Face repo id (Qwen/Qwen3-8B) unless you set --served-model-name, in which case it’s whatever you chose. Copy it character for character — Qwen/Qwen3-8B and qwen3-8b are different models as far as the gateway is concerned, and a mismatch surfaces only when the first request 404s. Models added this way carry a Custom badge. That’s provenance, not a warning: you supplied the name rather than picking it from a vendor catalog. It’s also the first place to look when requests to a model fail.

Step 4: Create a model route

Models attached to a provider aren’t yet callable by a client name. Under Model Routes → Create Route, define the alias your developers will actually use — say qwen3 — and point it at the model you just enabled. This indirection is what lets you move traffic later without touching a single client: repoint the route at a bigger GPU node, or add a hosted provider as a second target so requests spill over when your own capacity is saturated.

Step 5: Verify the connection

Test in layers. Each one isolates a different failure, and a green result at one layer tells you nothing about the next — so resist skipping ahead when something breaks. Almost every failed setup is diagnosed by finding the lowest layer that fails.

Your server is serving — on the server host

curl -s http://localhost:8000/v1/models -H "Authorization: Bearer <your-secret-token>"

Proves the process is up and the weights finished loading. A large model can take minutes; until it’s ready this returns nothing useful no matter how correct your configuration is.

It's reachable from somewhere else

Run the same curl from a different machine — your laptop, a bastion, anything that isn’t the server host — against the exact address you plan to give Barndoor:

curl -s https://llm.internal.example.com/v1/models -H "Authorization: Bearer <your-secret-token>"

Do not skip this by testing on the server host. Loopback succeeds even when the server is bound to 127.0.0.1 and unreachable by everything else, which is the most common reason a provider saves as unhealthy. This layer is the only one that catches it, along with firewall and security-group problems.

Barndoor can reach it

There is no separate “test connection” button — saving the provider is the connectivity test. Barndoor issues GET {base_url}/v1/models with your stored key and writes the result to the provider’s health badge.To re-test after changing something on your side, open the provider and Save again without editing any field. Health is only re-evaluated on create and update, so a server you just fixed keeps its stale badge until you do.

Badge	Reading
Healthy	Reachable, and the key was accepted. Move on.
Unhealthy — `could not reach upstream provider`	Network, host binding, or a certificate Barndoor doesn’t trust.
Unhealthy — `connection to upstream timed out after 10s`	Something accepted the connection but didn’t answer. Often a model still loading — the probe waits 10 seconds and no longer.
Unhealthy — `upstream rejected the credentials (HTTP 401)`	The key here doesn’t match the server’s `--api-key`.
Unverified	Reached the host, found no model list. Nearly always a `/v1` on the end of the base URL.

The model is exposed to a caller

Mint a key under Settings → My Models, then ask the gateway what that key can actually use:

curl -s https://app.barndoor.ai/api/llm-gateway/v1/models \
  -H "Authorization: Bearer bd-…"

Your route alias should be listed, with your provider named as what it resolves to. This is a genuinely different question from the previous layer: it proves the model route resolves and that model-access policy permits this caller.If the provider is healthy but the model isn’t listed here, the problem is the route or the access policy — not your server. Go back to Step 4, and check Controls → Model Access.

A request completes — both ways

curl https://app.barndoor.ai/api/llm-gateway/v1/chat/completions \
  -H "Authorization: Bearer bd-…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

Then run it again with "stream": true added. Streaming goes through a different path — an SSE relay with its own idle timeout — and it’s what coding assistants and chat UIs actually use. A passing non-streaming request is not evidence that streaming works.

curl -N https://app.barndoor.ai/api/llm-gateway/v1/chat/completions \
  -H "Authorization: Bearer bd-…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3",
    "messages": [{"role": "user", "content": "Count to five"}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Expect incremental data: chunks, a final chunk carrying usage, then data: [DONE]. The usage chunk is what makes a streamed request billable and countable — if it’s missing, tokens won’t reach your reports.

Governance actually applied

This is the layer that justifies the gateway existing, and the one most people forget to check.Open Reporting → LLM Usage Dashboard and confirm your requests appear, attributed to the user whose key sent them, with a token count. Cost will read zero — expected for self-hosted until you set rates, see below.For real assurance that controls apply to self-hosted traffic the same as hosted, set a deliberately tiny token budget scoped to your test key under Controls → Budgets, with action Block, and confirm the next request is refused. Budget definitions are cached for up to five minutes, so allow for that before concluding it doesn’t work.

Cost reporting for self-hosted models

Self-hosted inference has no market rate — the cost is your own GPU time — so Barndoor ships no default pricing for self-hosted models. Until you say otherwise, usage reporting counts tokens accurately and reports the cost as zero, and models show an Unpriced badge. If you want self-hosted traffic to show up in cost reports and count against budgets, set your own rates under Model Pricing. A reasonable approach is to divide the fully-loaded hourly cost of the instance by the tokens it produces in that hour, and enter the result as an input and output rate. It doesn’t need to be exact to be useful: even a rough number makes “what did this team actually consume” answerable, and lets a token budget act as a real ceiling. See Managing Model Pricing for the mechanics.

Troubleshooting

could not reach upstream provider

Barndoor couldn’t open a connection. In rough order of likelihood:

vLLM is bound to loopback — restart it with --host 0.0.0.0.
A firewall or security group blocks the port from Barndoor’s side.
You’re on Barndoor SaaS and the address is private. See Making the server reachable.
You’re using https:// with a self-signed or private-CA certificate.

Test from a third machine, not the server host — that’s what distinguishes “not listening publicly” from “not running”.

connection to upstream timed out after 10s

Something is accepting the connection but not answering in time. Usually a load balancer or proxy in front of your server that’s routing to a dead backend, or a server still loading model weights — a large model can take minutes to become ready. Wait for the server’s own logs to report it’s serving, then re-save the provider to re-run the check.

upstream rejected the credentials (HTTP 401)

The key in Barndoor doesn’t match the token your server was started with. Check for a trailing newline or a shell-quoting artifact — these are usually copy-paste damage rather than the wrong secret. Barndoor also raises a critical admin alert when an upstream rejects a provider’s credential, so a key rotated on the server side surfaces on its own.

Provider saved as Unverified

Barndoor reached the host but got a 404 or 405 from the model-list path. Nearly always a base URL ending in /v1 — remove it and save again. If your base URL is already the host root, check whether a reverse proxy in front of your server is rewriting or blocking /v1/models; the gateway treats an inaccessible model list as inconclusive rather than broken, which is why this is a distinct state from unhealthy.

Requests fail with an unknown-model error

The name enabled in Barndoor doesn’t match what the server serves. Compare against curl http://<host>:8000/v1/models — the id there is what the model field must contain, letter for letter. If you added --served-model-name after configuring Barndoor, the enabled name is now stale.

Tool calls come back as prose

The server wasn’t started with --enable-auto-tool-choice and a --tool-call-parser matching the model family. Barndoor forwards the tools array unchanged; it cannot synthesize structured tool calls from a server that isn’t producing them.

Current limits

Chat completions and streaming are the supported surface. Many servers also expose embeddings, completions, and the Responses API, but those are not verified through the gateway on this path.
Tool calling and reasoning output depend entirely on how you started the server, so Barndoor makes no promise about them on your behalf. They pass through when your server produces them.
Barndoor SaaS cannot reach a server on a private network. There is no tunnel or agent for this today; the server needs a routable address, or Barndoor needs to run inside your infrastructure.

​Before you start

​Step 1: Start your server so Barndoor can reach it

​Making the server reachable

​Step 2: Add the provider

​Step 3: Enable the models

​Step 4: Create a model route

​Step 5: Verify the connection

​Cost reporting for self-hosted models

​Troubleshooting

​Current limits

Before you start

Step 1: Start your server so Barndoor can reach it

Making the server reachable

Step 2: Add the provider

Step 3: Enable the models

Step 4: Create a model route

Step 5: Verify the connection

Cost reporting for self-hosted models

Troubleshooting

Current limits