← All code samples

Test and retire a model

Confirm a model configuration actually works before any agent depends on it, then clean it up when you are done.

  1. ensureProvider / ensureModel register an OpenAI-compatible provider and model (skip this step if you already have one)
  2. testModel runs a small generation call against the provider and returns {success, message} — it does not register or change anything
  3. deleteModel, then deleteProvider, to retire the configuration

When you need this

  • Validate a model id before switching agent traffic to it. Catch a typo’d model_id or an unreachable host with a 400 and a provider error message instead of a failed agent turn in production.
  • Confirm a rotated API key still works. After update_provider / updateProvider sets a new token_reference, testModel proves the new credential is live.
  • Tear down a short-lived evaluation model. deleteModel then deleteProvider — delete the model first; Aiden does not cascade-delete a provider’s models.

How to run

export STACKGEN_URL=https://app.stackgen.com
export STACKGEN_TOKEN=stackgen_…
export STACKGEN_WORKSPACE_ID=<workspace-uuid>

pip install stackgen-sdk==0.4.0
python examples/models/test-and-retire-model.py \
  --host https://api.openai.com/v1 --token-reference secret_ref_… --model-id gpt-4o-mini

Go does not have provider/model create helpers yet — point it at a provider and model that already exist. Python and TypeScript create one with ensureProvider / ensureModel when it is missing.

Minimal

from stackgen import StackgenClient, StackgenConfig

client = StackgenClient(
    StackgenConfig(
        base_url="https://app.stackgen.com",
        api_token="stackgen_…",
        workspace_id="<workspace-uuid>",
    )
)

client.aiden.llm.ensure_provider(
    name="my-openai", host="https://api.openai.com/v1", token_reference="secret_ref_…"
)
client.aiden.llm.ensure_model(
    name="my-model", provider_name="my-openai", model_id="gpt-4o-mini"
)

probe = client.aiden.llm.test_model(provider_name="my-openai", model_id="gpt-4o-mini")
print(probe["success"], probe["message"])

client.aiden.llm.delete_model("my-model")
client.aiden.llm.delete_provider("my-openai")

Notes

  • testModel raises/throws an HttpError(400) with the provider’s own error message on an invalid configuration — it never writes a model row.
  • Pass scope (TENANT / PROJECT) to deleteModel / delete_model when the model was registered at a non-default owner level.
  • Delete models before their provider. Aiden returns 404 for an unknown provider but does not cascade-delete the models registered under it.

Complete scripts

#!/usr/bin/env python3
"""Register an OpenAI-compatible model, probe it with test_model, then
retire it (delete_model + delete_provider).

test_model runs a small generation call against the provider without writing
anything, so it is the right pre-flight check before wiring an agent to a new
model, or after rotating a provider's token_reference.

Usage:
  export STACKGEN_URL=https://app.stackgen.com
  export STACKGEN_TOKEN=stackgen_…
  export STACKGEN_WORKSPACE_ID=<workspace-uuid>
  python examples/models/test-and-retire-model.py \
    --provider-name my-openai --host https://api.openai.com/v1 \
    --token-reference secret_ref_… --model-id gpt-4o-mini
"""

from __future__ import annotations

import argparse
import os

from stackgen import StackgenClient, StackgenConfig


def env(name: str, *fallbacks: str) -> str:
    for key in (name, *fallbacks):
        value = (os.environ.get(key) or "").strip()
        if value:
            return value
    raise SystemExit(f"missing env {name}")


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--provider-name", default="sdk-sample-openai")
    parser.add_argument("--host", required=True, help="e.g. https://api.openai.com/v1")
    parser.add_argument("--token-reference", default="", help="Vault secret_ref id")
    parser.add_argument("--model-name", default="sdk-sample-model")
    parser.add_argument("--model-id", required=True, help="Provider-specific model id")
    parser.add_argument(
        "--keep-after-test",
        action="store_true",
        help="Skip delete_model/delete_provider cleanup at the end",
    )
    args = parser.parse_args()

    client = StackgenClient(
        StackgenConfig(
            base_url=env("STACKGEN_URL"),
            api_token=env("STACKGEN_TOKEN"),
            workspace_id=env("STACKGEN_WORKSPACE_ID", "STACKGEN_PROJECT"),
        )
    )

    provider = client.aiden.llm.ensure_provider(
        name=args.provider_name,
        host=args.host,
        provider_type="openai",
        token_reference=args.token_reference,
    )
    print(f"provider ready name={provider.get('name')} host={provider.get('host')}")

    model = client.aiden.llm.ensure_model(
        name=args.model_name,
        provider_name=args.provider_name,
        model_id=args.model_id,
    )
    print(f"model ready name={model.get('name')} model_id={model.get('model_id')}")

    probe = client.aiden.llm.test_model(
        provider_name=args.provider_name, model_id=args.model_id, max_tokens=16
    )
    print(f"test_model success={probe.get('success')} message={probe.get('message')}")

    if args.keep_after_test:
        print("--keep-after-test set; leaving the model and provider in place")
        return 0

    client.aiden.llm.delete_model(args.model_name)
    print(f"deleted model name={args.model_name}")
    client.aiden.llm.delete_provider(args.provider_name)
    print(f"deleted provider name={args.provider_name}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

See also

  • Create a workflow — wire a tested model into an agent with createAgent({ modelNames: [...] }).