> ## Documentation Index
> Fetch the complete documentation index at: https://docs.barndoor.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Connecting a Self-Hosted Model
> Route traffic from your own model server — vLLM, Ollama, SGLang, or anything with an OpenAI-compatible API — through the Barndoor LLM Gateway, so in-house models get the same budgets, audit trail, and usage reporting as your hosted providers.
If your model server speaks an OpenAI-compatible API, the Barndoor LLM Gateway can route to it. That covers [vLLM](https://docs.vllm.ai), [Ollama](https://ollama.com), [SGLang](https://docs.sglang.ai), [LM Studio](https://lmstudio.ai), llama.cpp's `llama-server`, and in-house wrappers alike. Connecting one is configuration on both sides — no adapter, no SDK, no code.
What you get for doing it: every request to your own GPUs carries the same user identity, budget enforcement, rate limits, and audit record as a request to OpenAI or Anthropic. Your developers point at one endpoint and stop caring which side of the line a model lives on. And if your self-hosted capacity runs out, a model route can fall back to a hosted provider automatically — see [Provider Failover](/how-tos/llm-gateway-failover).
This guide assumes you already have an admin account and know your way around **LLM Management**. If not, read [Using the LLM Gateway](/how-tos/use-llm-gateway) first.
Only [Step 1](#step-1-start-your-server-so-barndoor-can-reach-it) differs between servers — everything after it is identical, because Barndoor sees the same OpenAI-compatible endpoint either way.
## Before you start
You'll need three things:
* A running model server, reachable over the network from Barndoor (see [Making the server reachable](#making-the-server-reachable) — this is the step that trips people up).
* The API key you started that server with.
* Admin access to **LLM Management** in the Barndoor portal.
***
## Step 1: Start your server so Barndoor can reach it
Two defaults bite on nearly every model server, and both must change before Barndoor can use it:
1. **It listens on loopback only**, so nothing off that machine can connect.
2. **It requires no credential**, so anything that *can* connect may use your GPUs freely.
```bash theme={null}
vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --port 8000 --api-key
```
**`--host` defaults to `127.0.0.1`.** Without `--host 0.0.0.0`, vLLM binds to loopback only and every connection from outside that machine — including Barndoor's — is refused. This is the single most common cause of a provider that saves as unhealthy.
`--api-key` turns on bearer-token authentication for the `/v1` paths. Treat the value as a secret: it is the only thing standing between your GPUs and anyone who can reach the port. Barndoor stores it encrypted and never exposes it to end users.
**Flags worth knowing about:**
By default vLLM advertises the model under its Hugging Face repo id, so `vllm serve Qwen/Qwen3-8B` reports `Qwen/Qwen3-8B` in `/v1/models` and expects that exact string in the `model` field of a request.
`--served-model-name qwen3-8b` overrides that with something friendlier. Whatever you choose, it must match what you enable in Barndoor in [Step 3](#step-3-enable-the-models) — the gateway passes the name through untouched.
vLLM only emits OpenAI-style `tool_calls` when the server was started for it, and the correct parser depends on the model family:
```bash theme={null}
vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --api-key \
--enable-auto-tool-choice --tool-call-parser hermes
```
Without these flags the model will describe a tool call in prose instead of returning a structured one, and clients that expect tool use will appear to hang or loop. Barndoor passes `tools` through unchanged either way — it can't compensate for a server that wasn't started for tool calling.
For reasoning models, `--reasoning-parser qwen3` (or `deepseek_r1`, depending on the family) splits thinking tokens into their own field rather than leaving them inline in `content`. Recommended if your clients render responses directly to users.
Sets the maximum combined prompt and output length. If unspecified, vLLM derives it from the model config, which can be more than your GPUs can actually hold. Setting it explicitly turns an out-of-memory crash into a clean HTTP error that Barndoor reports and can fail over from.
Ollama binds `127.0.0.1:11434` by default. Override it with an environment variable rather than a flag:
```bash theme={null}
OLLAMA_HOST=0.0.0.0:11434 ollama serve
```
Then pull whatever you intend to serve — `ollama pull llama3.2:1b`. The name in `/v1/models` is the Ollama tag (`llama3.2:1b`), not a Hugging Face repo id.
**Ollama has no authentication and ignores the `Authorization` header entirely.** Any key is accepted, so anything that can reach the port can use your hardware. Barndoor still requires an API key on the provider, but Ollama will not check it.
Do not expose Ollama beyond a trusted network on its own. Put a reverse proxy in front that enforces a bearer token on `/v1` and forwards to Ollama, and point Barndoor at the proxy. `brew services start ollama` is also not a substitute for the command above — the service definition doesn't set `OLLAMA_HOST`, so it binds loopback.
Ollama is a strong fit for a shared on-prem box or an edge deployment. For multi-GPU production serving, vLLM or SGLang are the better targets.
The gateway needs three things from your server, whatever it is:
* It **listens on an address Barndoor can reach** — usually meaning bind `0.0.0.0` rather than loopback.
* It **serves `/v1/models` and `/v1/chat/completions`** off a host root, in OpenAI format.
* It **accepts a bearer token** on those paths. If it can't, put a reverse proxy in front that does and point Barndoor at the proxy.
SGLang, LM Studio, llama.cpp's `llama-server`, TGI and in-house FastAPI wrappers all satisfy the first two out of the box; check your server's own documentation for the bind address and auth flags, since those are the two that differ. Everything from [Step 2](#step-2-add-the-provider) onward is identical regardless.
Confirm the server answers before you go anywhere near the portal:
```bash theme={null}
curl -s http://:8000/v1/models -H "Authorization: Bearer "
```
You should get a JSON list containing the model id you intend to use. Run this from a *different* machine than the one hosting the server — running it locally passes even when the loopback-binding problem above is present, which is exactly how that problem stays hidden.
***
## Making the server reachable
Barndoor connects to your server over the network like any other client, which means the gateway has to be able to route to it. This is independent of which server you run.
A server on a private subnet is not reachable from `app.barndoor.ai`. You need to give it a routable address:
* Put it behind your own load balancer or ingress with a public DNS name and TLS.
* Restrict who can reach that address — an IP allowlist for Barndoor's egress addresses, or mutual TLS at your edge. `--api-key` is authentication, not network isolation.
The certificate must be issued by a publicly trusted CA. The gateway validates TLS and will reject a self-signed or private-CA certificate — it surfaces as `could not reach upstream provider`, which reads like a connectivity problem rather than a certificate one.
When Barndoor runs inside your own infrastructure, no public exposure is needed. Use whatever address the gateway can resolve — a Kubernetes service DNS name such as `http://vllm.ml.svc.cluster.local:8000`, or a private VPC address.
Plain `http://` is accepted, so an internal deployment doesn't need certificates.
***
## Step 2: Add the provider
In the Barndoor portal, go to **LLM Management → Providers → Add** and choose **Custom Provider** — not one of the named vendor cards.
That's the right choice rather than a compromise. The named cards exist to carry facts about a vendor's hosted endpoint: its URL, the models it serves, its per-token prices. None of those are knowable for a server you run yourself — a card for one could only ever hold a placeholder URL, an empty model list, and no pricing. The Custom Provider flow asks you for the three things that actually vary and assumes nothing else, which is why it works the same for every server in [Step 1](#step-1-start-your-server-so-barndoor-can-reach-it).
| Field | Value |
| ------------------ | ------------------------------------------------------------------------------------------------------- |
| **Name** | Anything recognizable — `vLLM production`, `Qwen cluster`. This is what appears in usage reports. |
| **Model Provider** | Fixed at **Custom (OpenAI-compatible)**. Not editable, and correct for any OpenAI-compatible server. |
| **Base URL** | Your server's **host root**, for example `https://llm.internal.example.com` or `http://10.0.4.12:8000`. |
| **API Key** | The value you passed to `--api-key`. |
**The base URL must not end in `/v1`.**
vLLM's own documentation shows `http://localhost:8000/v1` because that's what OpenAI SDK clients expect. Barndoor is not an SDK client — it appends the version segment itself, requesting `{base_url}/v1/models` and `{base_url}/v1/chat/completions`. A `/v1` on the end produces `/v1/v1/models`, which 404s. The provider then saves as **unverified** rather than failing outright, so this mistake is quiet until the first real request.
On save, Barndoor runs a connectivity check: a `GET {base_url}/v1/models` with your key, timing out after 10 seconds. The result is recorded as the provider's health:
| Health | What it means |
| -------------- | ----------------------------------------------------------------------------------------------------- |
| **Healthy** | The server answered and accepted the key. You're done here. |
| **Unhealthy** | Barndoor couldn't reach the server, or it returned an error. See [Troubleshooting](#troubleshooting). |
| **Unverified** | Barndoor reached something, but no model list at that path — almost always the `/v1` suffix above. |
An unhealthy provider is automatically suspended from routing while it stays that way, and resumes on its own once a later check succeeds. You can override this per provider with **Block traffic on failed health check**, which is worth turning off only if your server is behind something that blocks the model-list endpoint but serves completions fine.
***
## Step 3: Enable the models
Open the provider and choose **Add Models**, then type your model name in. A custom provider has no catalog to pick from, which is the honest state of affairs — only your server knows what it loaded.
**The name must match what `/v1/models` reports, exactly.** Take it from the server rather than from memory:
```bash theme={null}
curl -s https://llm.internal.example.com/v1/models \
-H "Authorization: Bearer "
```
That's the Hugging Face repo id (`Qwen/Qwen3-8B`) unless you set `--served-model-name`, in which case it's whatever you chose. Copy it character for character — `Qwen/Qwen3-8B` and `qwen3-8b` are different models as far as the gateway is concerned, and a mismatch surfaces only when the first request 404s.
Models added this way carry a **Custom** badge. That's provenance, not a warning: you supplied the name rather than picking it from a vendor catalog. It's also the first place to look when requests to a model fail.
***
## Step 4: Create a model route
Models attached to a provider aren't yet callable by a client name. Under **Model Routes → Create Route**, define the alias your developers will actually use — say `qwen3` — and point it at the model you just enabled.
This indirection is what lets you move traffic later without touching a single client: repoint the route at a bigger GPU node, or add a hosted provider as a second target so requests spill over when your own capacity is saturated.
***
## Step 5: Verify the connection
Test in layers. Each one isolates a different failure, and a green result at one layer tells you nothing about the next — so resist skipping ahead when something breaks. Almost every failed setup is diagnosed by finding the lowest layer that fails.
```bash theme={null}
curl -s http://localhost:8000/v1/models -H "Authorization: Bearer "
```
Proves the process is up and the weights finished loading. A large model can take minutes; until it's ready this returns nothing useful no matter how correct your configuration is.
Run the *same* curl from a different machine — your laptop, a bastion, anything that isn't the server host — against the exact address you plan to give Barndoor:
```bash theme={null}
curl -s https://llm.internal.example.com/v1/models -H "Authorization: Bearer "
```
Do not skip this by testing on the server host. Loopback succeeds even when the server is bound to `127.0.0.1` and unreachable by everything else, which is the most common reason a provider saves as unhealthy. This layer is the only one that catches it, along with firewall and security-group problems.
There is no separate "test connection" button — **saving the provider is the connectivity test.** Barndoor issues `GET {base_url}/v1/models` with your stored key and writes the result to the provider's health badge.
To re-test after changing something on your side, open the provider and **Save** again without editing any field. Health is only re-evaluated on create and update, so a server you just fixed keeps its stale badge until you do.
| Badge | Reading |
| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Healthy** | Reachable, and the key was accepted. Move on. |
| **Unhealthy** — `could not reach upstream provider` | Network, host binding, or a certificate Barndoor doesn't trust. |
| **Unhealthy** — `connection to upstream timed out after 10s` | Something accepted the connection but didn't answer. Often a model still loading — the probe waits 10 seconds and no longer. |
| **Unhealthy** — `upstream rejected the credentials (HTTP 401)` | The key here doesn't match the server's `--api-key`. |
| **Unverified** | Reached the host, found no model list. Nearly always a `/v1` on the end of the base URL. |
Mint a key under **Settings → My Models**, then ask the gateway what that key can actually use:
```bash theme={null}
curl -s https://app.barndoor.ai/api/llm-gateway/v1/models \
-H "Authorization: Bearer bd-…"
```
Your route alias should be listed, with your provider named as what it resolves to. This is a genuinely different question from the previous layer: it proves the model route resolves *and* that model-access policy permits this caller.
If the provider is healthy but the model isn't listed here, the problem is the route or the access policy — not your server. Go back to [Step 4](#step-4-create-a-model-route), and check **Controls → Model Access**.
```bash theme={null}
curl https://app.barndoor.ai/api/llm-gateway/v1/chat/completions \
-H "Authorization: Bearer bd-…" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3",
"messages": [{"role": "user", "content": "Say hello"}]
}'
```
Then run it again with `"stream": true` added. Streaming goes through a different path — an SSE relay with its own idle timeout — and it's what coding assistants and chat UIs actually use. A passing non-streaming request is not evidence that streaming works.
```bash theme={null}
curl -N https://app.barndoor.ai/api/llm-gateway/v1/chat/completions \
-H "Authorization: Bearer bd-…" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3",
"messages": [{"role": "user", "content": "Count to five"}],
"stream": true,
"stream_options": {"include_usage": true}
}'
```
Expect incremental `data:` chunks, a final chunk carrying `usage`, then `data: [DONE]`. The usage chunk is what makes a streamed request billable and countable — if it's missing, tokens won't reach your reports.
This is the layer that justifies the gateway existing, and the one most people forget to check.
Open **Reporting → LLM Usage Dashboard** and confirm your requests appear, attributed to the user whose key sent them, with a token count. Cost will read **zero** — expected for self-hosted until you set rates, see below.
For real assurance that controls apply to self-hosted traffic the same as hosted, set a deliberately tiny token budget scoped to your test key under **Controls → Budgets**, with action **Block**, and confirm the next request is refused. Budget definitions are cached for up to five minutes, so allow for that before concluding it doesn't work.
***
## Cost reporting for self-hosted models
Self-hosted inference has no market rate — the cost is your own GPU time — so Barndoor ships no default pricing for self-hosted models. Until you say otherwise, usage reporting counts tokens accurately and reports the cost as **zero**, and models show an **Unpriced** badge.
If you want self-hosted traffic to show up in cost reports and count against budgets, set your own rates under **Model Pricing**. A reasonable approach is to divide the fully-loaded hourly cost of the instance by the tokens it produces in that hour, and enter the result as an input and output rate. It doesn't need to be exact to be useful: even a rough number makes "what did this team actually consume" answerable, and lets a token budget act as a real ceiling. See [Managing Model Pricing](/how-tos/manage-model-pricing) for the mechanics.
***
## Troubleshooting
Barndoor couldn't open a connection. In rough order of likelihood:
* vLLM is bound to loopback — restart it with `--host 0.0.0.0`.
* A firewall or security group blocks the port from Barndoor's side.
* You're on Barndoor SaaS and the address is private. See [Making the server reachable](#making-the-server-reachable).
* You're using `https://` with a self-signed or private-CA certificate.
Test from a third machine, not the server host — that's what distinguishes "not listening publicly" from "not running".
Something is accepting the connection but not answering in time. Usually a load balancer or proxy in front of your server that's routing to a dead backend, or a server still loading model weights — a large model can take minutes to become ready. Wait for the server's own logs to report it's serving, then re-save the provider to re-run the check.
The key in Barndoor doesn't match the token your server was started with. Check for a trailing newline or a shell-quoting artifact — these are usually copy-paste damage rather than the wrong secret. Barndoor also raises a critical admin alert when an upstream rejects a provider's credential, so a key rotated on the server side surfaces on its own.
Barndoor reached the host but got a 404 or 405 from the model-list path. Nearly always a base URL ending in `/v1` — remove it and save again. If your base URL is already the host root, check whether a reverse proxy in front of your server is rewriting or blocking `/v1/models`; the gateway treats an inaccessible model list as inconclusive rather than broken, which is why this is a distinct state from unhealthy.
The name enabled in Barndoor doesn't match what the server serves. Compare against `curl http://:8000/v1/models` — the id there is what the `model` field must contain, letter for letter. If you added `--served-model-name` after configuring Barndoor, the enabled name is now stale.
The server wasn't started with `--enable-auto-tool-choice` and a `--tool-call-parser` matching the model family. Barndoor forwards the `tools` array unchanged; it cannot synthesize structured tool calls from a server that isn't producing them.
***
## Current limits
* **Chat completions and streaming** are the supported surface. Many servers also expose embeddings, completions, and the Responses API, but those are not verified through the gateway on this path.
* **Tool calling and reasoning output** depend entirely on how you started the server, so Barndoor makes no promise about them on your behalf. They pass through when your server produces them.
* **Barndoor SaaS cannot reach a server on a private network.** There is no tunnel or agent for this today; the server needs a routable address, or Barndoor needs to run inside your infrastructure.