Custom LLM provider URLs

Custom LLM provider URLs

Use this guide when your tenant must call an LLM through something other than the vendor’s public API: Azure OpenAI, Azure AI Foundry, a corporate HTTP proxy, or an OpenAI-compatible gateway (LiteLLM, OpenRouter, Ollama behind a reverse proxy, and similar).

Aiden OS still registers the provider as type openai. You point it at your endpoint with a custom base URL and, for Azure, an API version. Chat, model listing, and the Test model button all use the same URL configuration and precedence.

ELI5: You tell Aiden where the “calculator factory” lives instead of using the default OpenAI address. The factory name (model) and the key stay the same; only the street address changes.

Related: Lesson 04 — Add an LLM model, Agent mute — models & tools FAQ, aios-foundation.

---

What gets configured

Piece	Where it lives	Purpose
API key	Vault secret (OPENAI_API_KEY)	Authenticates to the endpoint
Base URL	Vault secret (OPENAI_API_URL) or provider host	API base for generic providers, or resource root for Azure
API version	Vault secret (OPENAI_API_VERSION)	Required for Azure; leave empty for standard OpenAI and most proxies
Provider type	sg_llm_provider.provider_type	Always openai for OpenAI-compatible endpoints (including Azure)
Model id	sg_llm_model.model_id	OpenAI model name, or Azure deployment name in Azure mode

Precedence: If both are set, sg_llm_provider.host wins over OPENAI_API_URL in the secret. Model discovery and inference use the same rule, so listing models and running chat stay aligned.

Leave the URL blank to use the public OpenAI default (https://api.openai.com/v1).

---

Option A — Aiden UI (Settings → LLM providers)

Add a provider

1. Open Settings → LLM providers → Add provider → OpenAI (or your workspace’s equivalent).
2. On Credentials:
   • Provider name — unique label in this org (for example acme-azure-openai).
   • OpenAI Key — API key from Azure, your proxy, or OpenAI.
   • OpenAI API URL — your resource or gateway base URL (see examples below).
   • API Version — set for Azure; leave empty for standard OpenAI and most OpenAI-compatible gateways.
3. Continue to Models, pick deployments/models, register names, and attach them to agents.

The wizard stores OpenAI API URL and API Version in the Vault secret together with the key. You can later change the endpoint under Edit provider → Custom Endpoint (host on the provider record).

Edit an existing provider

Edit LLM Provider → Custom Endpoint updates host without recreating the provider. Use this when only the URL changes (new region, new proxy hostname). To rotate keys, point Vault Secret at a different secret or update the secret in Vault.

---

Option B — Terraform / OpenTofu (agent-as-code)

Extend the Lesson 04 pattern: secret → provider → model → agent.

Standard OpenAI-compatible proxy (LiteLLM, OpenRouter, etc.)

resource "sg_secret" "openai_proxy" {
  name        = "acme-openai-proxy-vault"
  description = "OpenAI-compatible gateway"
  category    = "LLM"
  subcategory = "openai"
  metadata = {
    OPENAI_API_KEY = var.openai_proxy_api_key
    OPENAI_API_URL = "https://openrouter.ai/api/v1" # example; use your gateway base
  }
}

resource "sg_llm_provider" "openai_proxy" {
  name            = "openrouter"
  provider_type   = "openai"
  token_reference = sg_secret.openai_proxy.name
  # Optional: host = "https://openrouter.ai/api/v1"  # overrides OPENAI_API_URL when set
}

resource "sg_llm_model" "primary" {
  name          = "acme-primary-llm"
  provider_name = sg_llm_provider.openai_proxy.name
  model_id      = "openai/gpt-4o" # must match what your gateway expects
  thinking      = false
}

Do not set OPENAI_API_VERSION unless the gateway documents an Azure-style version query parameter.

Azure OpenAI / Azure AI Foundry

Use provider type openai. Put the resource endpoint in the URL field (from the Azure portal or Foundry project). Set OPENAI_API_VERSION to the dated deployment-based API version Azure expects, such as 2024-02-01.

resource "sg_secret" "azure_openai" {
  name        = "acme-azure-openai-vault"
  description = "Azure OpenAI"
  category    = "LLM"
  subcategory = "openai"
  metadata = {
    OPENAI_API_KEY     = var.azure_openai_api_key
    OPENAI_API_URL     = "https://myresource.openai.azure.com"
    OPENAI_API_VERSION = "2024-02-01"
  }
}

resource "sg_llm_provider" "azure_openai" {
  name            = "azure-openai"
  provider_type   = "openai"
  token_reference = sg_secret.azure_openai.name
}

resource "sg_llm_model" "gpt4o" {
  name          = "acme-gpt-4o-azure"
  provider_name = sg_llm_provider.azure_openai.name
  model_id      = "my-gpt-4o-deployment" # Azure deployment name, not always the public model id
  thinking      = false
}

For Azure AI Foundry, use the resource root, such as https://<name>.services.ai.azure.com. Do not include /openai/; Aiden adds the Azure operation path. Set OPENAI_API_VERSION so model listing and inference both use Azure mode.

---

Option C — curl (HTTP API)

Use this when you automate setup outside the UI or Terraform. All Guild calls use the /guild/api/v1 prefix and need orgId for workspace-scoped resources (same rule as the Agent mute FAQ). Vault secrets use /api/vault/v1.

BASE='https://<tenant-host>'
TOKEN='<aiden-bearer-token>'
ORG='<org-id>'   # workspace the provider and models live in

Field map (UI ↔ Vault ↔ API ↔ Terraform)

UI label (Add OpenAI)	Vault secret key	Guild provider JSON	Terraform
Provider name	—	name on POST …/model-providers	sg_llm_provider.name
OpenAI Key	OPENAI_API_KEY	via token_reference → secret	sg_secret.metadata.OPENAI_API_KEY
OpenAI API URL	OPENAI_API_URL	optional host (overrides secret URL)	sg_secret.metadata.OPENAI_API_URL or sg_llm_provider.host
API Version	OPENAI_API_VERSION	stored in secret only	sg_secret.metadata.OPENAI_API_VERSION
(provider type, fixed)	subcategory: openai	provider_type: "openai"	sg_llm_provider.provider_type
Registered model name	—	name on POST …/models	sg_llm_model.name
Model / deployment id	—	model_id on POST …/models	sg_llm_model.model_id

1. Store credentials in Vault

Maps to the Credentials tab in the Add OpenAI wizard.

curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "$BASE/api/vault/v1/secrets?orgId=$ORG" \
  -d '{
    "name": "acme-azure-openai-vault",
    "description": "Azure OpenAI credentials",
    "category": "LLM",
    "subcategory": "openai",
    "value": [
      { "key": "OPENAI_API_KEY",     "value": "<azure-api-key>" },
      { "key": "OPENAI_API_URL",     "value": "https://myresource.services.ai.azure.com" },
      { "key": "OPENAI_API_VERSION", "value": "2024-02-01" }
    ]
  }'

JSON fragment	Field
"name": "acme-azure-openai-vault"	Vault secret name; later used as token_reference
"subcategory": "openai"	LLM provider family (matches UI OpenAI)
"key": "OPENAI_API_KEY"	UI OpenAI Key
"key": "OPENAI_API_URL"	UI OpenAI API URL (resource or gateway base)
"key": "OPENAI_API_VERSION"	UI API Version (Azure); omit the whole entry for public OpenAI or most proxies

For OpenRouter or LiteLLM, drop the OPENAI_API_VERSION entry and set OPENAI_API_URL to your gateway base (for example https://openrouter.ai/api/v1).

2. Register the model provider

Maps to finishing the wizard’s provider step (name + link to the secret). Optional host is the same as Edit provider → Custom Endpoint; when set, it overrides OPENAI_API_URL in the secret.

curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "$BASE/guild/api/v1/model-providers?orgId=$ORG" \
  -d '{
    "name": "acme-azure-openai",
    "provider_type": "openai",
    "token_reference": "acme-azure-openai-vault"
  }'

JSON fragment	Field
"name": "acme-azure-openai"	UI Provider name
"provider_type": "openai"	Always openai for OpenAI-compatible endpoints (including Azure)
"token_reference"	Vault secret name from step 1
"host" (optional)	UI Custom Endpoint; wins over OPENAI_API_URL in the secret

Example with an explicit endpoint override:

# Same as setting host on sg_llm_provider in Terraform
-d '{
  "name": "acme-azure-openai",
  "provider_type": "openai",
  "token_reference": "acme-azure-openai-vault",
  "host": "https://myresource.services.ai.azure.com"
}'

3. List models from your endpoint

Confirms the URL, key, and version reach your host (not api.openai.com).

curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  "$BASE/guild/api/v1/model-providers/acme-azure-openai/available-models?orgId=$ORG" | jq .

Each "id" in the response is a candidate for model_id (Azure deployment name in Azure mode).

4. Register a named model

Maps to picking models on the wizard Models tab.

curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "$BASE/guild/api/v1/models?orgId=$ORG" \
  -d '{
    "name": "acme-gpt-4o-azure",
    "provider_name": "acme-azure-openai",
    "model_id": "my-gpt-4o-deployment",
    "thinking": false
  }'

JSON fragment	Field
"name"	Registered model name agents use in model_names
"provider_name"	Provider name from step 2
"model_id"	Provider model or Azure deployment id from step 3

5. Live-probe the registered model

Same check as Test model in the UI.

curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "$BASE/guild/api/v1/models/acme-gpt-4o-azure/test?orgId=$ORG" | jq .

Expect HTTP 200 and "success": true. A 400 here usually means wrong model_id, key, URL, or API version, not agent wiring.

To probe before registering, use POST /guild/api/v1/models/test with provider_name and model_id only (no named model row yet):

curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "$BASE/guild/api/v1/models/test?orgId=$ORG" \
  -d '{
    "provider_name": "acme-azure-openai",
    "model_id": "my-gpt-4o-deployment"
  }' | jq .

---

How the base URL is used

For generic OpenAI-compatible endpoints, configure the complete API base documented by the vendor, normally ending in /v1. Inference uses that base directly. For model discovery, Guild adds /v1/models when the configured value does not end in /v1.

You configure	Model list request
(empty)	https://api.openai.com/v1/models
https://api.x.ai/v1	https://api.x.ai/v1/models
https://proxy.example	https://proxy.example/v1/models
https://openrouter.ai/api/v1	https://openrouter.ai/api/v1/models

Tips:

• Include /v1 in a generic endpoint’s configured base. Guild will not add a second /v1.
• Do not rely on model discovery adding /v1 for inference; inference uses the configured base directly.
• Do not paste a full chat URL (paths containing chat/completions or deployments/.../chat). Use the resource or gateway base only.

Azure vs generic OpenAI-compatible

Mode	When it activates	model_id meaning	Auth
Azure	OPENAI_API_VERSION is set	Deployment name in your Azure resource	api-key header
Generic OpenAI-compatible	Custom URL, no Azure version / non-Azure host	Model id your gateway exposes	Authorization: Bearer

If Azure listing fails with “could not load models”, check the URL and version first. Always set OPENAI_API_VERSION; hostname detection during model discovery does not enable Azure mode for inference.

---

Verify

UI

1. Settings → LLM providers — open the provider; OpenAI API URL (or Custom Endpoint) shows the resolved base.
2. Models tab — Refresh / list models returns deployments or model ids from your endpoint, not a 401 from api.openai.com.
3. Test model on a registered model — short generation succeeds.

HTTP (same org as the agent)

Use the Agent mute FAQ probe pattern:

export BASE="https://<tenant-host>"
export ORG="<org-id>"
export TOKEN="<bearer-token>"

curl -sS "$BASE/guild/api/v1/models?orgId=$ORG" -H "Authorization: Bearer $TOKEN"
curl -sS -X POST "$BASE/guild/api/v1/models/<registered-model-name>/test?orgId=$ORG" \
  -H "Authorization: Bearer $TOKEN"

Expect HTTP 200 and a non-empty test response. If chat fails but the test passes, look at agent model_names wiring next (not the URL).

---

Common setups

Scenario	OPENAI_API_URL / host	OPENAI_API_VERSION	model_id
Public OpenAI	(leave empty)	(empty)	e.g. gpt-4o
Azure OpenAI classic	https://<resource>.openai.azure.com	e.g. 2024-02-01	Deployment name
Azure AI Foundry	https://<project>.services.ai.azure.com	Value from Azure portal	Deployment name
OpenRouter	https://openrouter.ai/api/v1	(empty)	Gateway model slug
LiteLLM / internal proxy	Your proxy’s documented OpenAI base	(empty)	Model name exposed by proxy
Ollama (OpenAI shim)	e.g. http://ollama.internal:11434/v1	(empty)	Local model name

---

Troubleshooting

Symptom	Likely cause	Fix
“Could not load models” / 401 on add-provider step	Wrong key, URL, or Azure API version	Fix the key or URL; for Azure, set OPENAI_API_VERSION
Models list empty on Azure	No deployments in resource, or wrong resource URL	Confirm deployments in Azure portal; match OPENAI_API_URL to resource endpoint
Chat 404 on completions	Base URL includes /chat/completions or a deployment path	Use resource/gateway base only; Azure deployment comes from model_id
Chat works in UI test but agent mute	Agent model_names not pointing at registered models	Re-apply foundation or fix agent model list (FAQ)
Listing hits api.openai.com instead of proxy	URL only in secret but host empty and secret missing OPENAI_API_URL	Set URL in secret or set provider host
Double /v1/v1 errors	Unusual proxy layout	Match vendor base exactly; see table above

---

aios-foundation module

aios-foundation registers public OpenAI, Anthropic, and Gemini with keys only (no custom URL). For Azure or a proxy, add a parallel sg_secret / sg_llm_provider / sg_llm_model block in your root module (as above) and pass the new model name into agent modules’ model_names. You do not need to fork the foundation module.

---

Platform requirements

• Custom OpenAI-compatible base URLs are supported in Guild releases that include [AIOS-1099] Preserve OpenAI-compatible API base URLs (Guild v0.2.13+).
• Use StackGen provider >= 0.1.33 (see Adopt the repo for current floor).