Providers & Models

Providers & Models

Holaryn Agent gets its models exclusively from a provider registry you configure — there is no built-in vendor and no fallback: an unconfigured agent refuses to run and tells you exactly what to set up. This page explains connections, models, defaults, nicknames, capabilities, and switching models mid-conversation.

The provider registry

The registry lives in providers.json in the host state directory (find it under Settings → System → Paths); API keys go in the state directory's write-only secret store, never in the JSON file. Manage everything in Settings → Providers & Models, which edits two levels:

• Connections — a provider endpoint: which adapter it speaks, an optional base URL, and where its API key comes from (an environment variable name, or a key stored in the secret store).
• Credential pools — optional authorized groups of compatible connections with fair scheduling,
  quotas, rate/health state, and drain/revoke controls. Pools contain aliases and opaque connection
  references, never keys.
• Models — the selectable models, each bound to one connection and optionally one credential
  pool, with its capabilities, an optional nickname, and tags.

The registry is validated on load: connection, pool, and model ids must be unique; every model must
reference a known connection; and a model's fallback connection must belong to its selected pool.

Adding a provider

Settings → Providers & Models → Add provider opens a picker over the built-in provider catalog: a filterable, grouped list of curated presets. The groups are:

• Major providers — first-party APIs (OpenAI, Anthropic, Google Gemini, Mistral, DeepSeek, xAI, Groq, and more), each authenticated with an API key.
• Aggregators — many vendors behind one key (OpenRouter, Hugging Face).
• Subscriptions (sign in) — your personal ChatGPT or GitHub Copilot plan via a device-code sign-in (see the caveats below), plus API-key coding plans.
• Local — keyless endpoints on your own machine (Ollama, LM Studio, vLLM, llama.cpp). A local endpoint is just another connection: the preset fills the usual base URL (e.g. http://localhost:11434/v1 for Ollama — change the host to reach it over the LAN), no API key is needed, and a turn routed to it never leaves your network. Two gotchas: Ollama's OpenAI-compatible endpoint defaults to a small context (about 4096 tokens) regardless of what the model supports — bake a larger num_ctx into a model variant with a Modelfile and set the model's declared context window in Holaryn to match, because the agent's compaction math depends on it; and small local models are often unreliable at native tool calling — if a model emits raw tool JSON as text, turn off its native tool calling capability and the adapter falls back to a prompted-JSON tool path instead.

For hardware fit, verified downloads/imports, Ollama and llama.cpp process
supervision, local benchmarks, role assignment, and safe owned-file removal, use
the Local models lifecycle manager. It registers a healthy
endpoint back into this same provider registry.
- Custom — full manual configuration for anything else.

Entries you have already configured are marked, and picking one again simply creates a second connection with a uniquified id. Picking a provider prefills everything — connection id, adapter, endpoint, and the suggested key variable, all still editable under an Advanced disclosure — and the submit button is the whole flow:

• Add & scan (API-key providers) creates the connection, stores the key you paste inline (a Get a key link opens the provider's console), and opens the model scan.
• Add & sign in (subscriptions) creates the connection and starts the device-code sign-in immediately, chaining into the scan the moment you approve.
• A local endpoint goes straight to a keyless scan.

The catalog is a template, not a live binding: creating a connection snapshots the preset's values into your own registry, and later releases update the catalog without ever rewriting an existing connection.

Scanning for models

A scan asks the connection's endpoint for the models it serves and lists them as checkboxes — nothing is ever pre-selected. Filter the list, then use Select all shown (it respects the filter and skips models you already added), Select recommended, or Clear selection, and add the whole selection in one step. Models the catalog recommends for the provider are marked Recommended and sort to the top. Added models get the adapter's default capabilities; edit them afterwards as needed. You can rescan a connection at any time from its Actions → Scan for models… menu.

A few providers (Perplexity, for example) have no model-listing endpoint. For those the scan panel explains that and offers one-click Add buttons that open the manual add-model form prefilled with the provider's typical model ids.

Custom / advanced

The Custom entry (and editing an existing connection) opens the full free-form dialog. Each connection picks one adapter — one per wire protocol, not one per vendor:

After saving a custom connection you can scan it from its Actions menu, exactly like a catalog one.

Credential pools

Credential pools are for organization-approved accounts or endpoints that may share one workload.
They are not a way to combine unrelated users' subscriptions or evade provider limits. All members
must use the same provider wire adapter.

In Settings → Providers & Models → Credential pools:

1. Add a pool and review the generated JSON template.
2. Set a safe alias for each connection, its weight/priority, optional
   tenant/project/profile allowlists and validity window, and pool/member request, token, spend, and
   concurrency limits.
3. Save, then use Preview selection to see which alias is eligible without consuming capacity.
4. Edit a model and select the pool. Its ordinary connection remains a compatible fallback and must
   be one of the pool members.

The pool card shows active leases, aggregate requests/tokens/cost, provider-reported remaining
capacity, circuit/auth/drain state, and recent selection reasons. Drain gracefully stops new work
but lets in-flight work finish. Revoke and cancel stops new work, cancels durable leases, and
discards late model results. Re-activate only after resolving the underlying account condition.

Weights are deterministic: over time a weight-2 member receives roughly twice the selections of a
weight-1 member at the same priority. Higher member priority is considered first. Reserved priority
slots keep part of pool/member concurrency available for requests at or above the configured
threshold. Sticky conversations stay on their prior account when it remains eligible; disabling
account switching makes an unavailable sticky account throttle instead of silently changing billing
accounts.

See the credential-pool ADR and recovery guide for the full state,
quota, failure, privacy, and troubleshooting contract.

Authentication: API key or subscription sign-in

Most connections authenticate with an API key — either the name of an environment variable holding it, or a key stored on this machine (Actions → Set API key, write-only: the UI only ever shows "set" or "not set").

Two connection types can instead sign in with a personal subscription — a short code + URL device flow you approve in a browser. Both use Actions → Sign in with … or, from the terminal, holaryn connection login <id> (with logout / status). Tokens are stored on this machine like any other secret and never shown; the settings page displays only the signed-in state and your account.

In the web UI, pressing Sign in opens your default browser at the provider's page and shows the code with a Copy code button; the panel announces each step in a live region and confirms when you approve. If the browser does not open — a popup blocker, say — the page is still listed as a link and Open sign-in page tries again. The terminal command prints the URL and code for you to open yourself, which is what makes it usable over SSH.

Sign in with ChatGPT — on an OpenAI Responses connection. Requests bill against your ChatGPT Plus/Pro subscription instead of per-token API credits. You approve a short code at https://auth.openai.com/codex/device.

• Turn on device-code login first. OpenAI requires the account to opt in, and sign-in cannot work until you do: for a personal account, ChatGPT → Settings → Security → "Allow device code login"; for a work/team account, an admin enables it under Workspace Settings → Permissions. If it is off, Holaryn reports that device-code login is not enabled and points you here.
• Personal use of your own account only. Do not pool accounts or route several people through one subscription.
• Unofficial and not contractual. It rides an undocumented Codex backend endpoint with no SLA and may stop working without notice.

Sign in with GitHub Copilot — on an OpenAI-compatible connection. Requests use your GitHub Copilot subscription against Copilot's OpenAI-compatible chat endpoint.

• Unofficial, and against GitHub's editor-only terms. It calls GitHub's internal Copilot endpoints outside a supported editor. GitHub does not authorize this, and its API-abuse terms allow discretionary suspension — using it can get your GitHub account suspended. Personal use of your own account only; it may also break without notice. Use it only if you accept that risk.

Anthropic and Google are prohibited and appear in the menu but disabled: their subscription tokens are scoped to their own apps, with account suspensions for violations. Use an API key for Claude and Gemini.

Models, the default, and nicknames

Every model row has:

• Id — the label shown in pickers.
• Capabilities — see below.
• Nickname — an optional friendly name. Nicknames are shown on the agent's replies and double as @-aliases: @bob what's the weather? runs that turn on the model nicknamed bob. A nickname must be a single word, unique, and must not collide with a profile name, coding backend, or paired peer.
• Tags — free-form labels (e.g. local, coding) for your own organization.
• Enabled — disabled models stay manageable in Settings but are hidden from every model chooser.
• Credential pool — optional fair-capacity scheduling across compatible authorized connections;
  leaving it blank preserves single-connection behavior.

One model is the default: it is what a brand-new chat or session uses. Set it in Settings → Providers & Models; the Session status panel shows which model a fresh session would get right now.

Capability awareness

Each model declares a capability descriptor the agent consults before every turn:

• Context window — drives the model-aware compaction that keeps long conversations within budget.
• Native tool calling — whether the model emits real tool calls; when off, the agent emulates tool use with a prompted-JSON protocol.
• Parallel tool calls — whether the model may request several tools at once.
• Vision — whether the model accepts images.
• Prompt cache — whether the provider supports prompt caching. Retention remains off until you
  configure and consent to a per-model prompt cache policy.
• Reasoning effort — whether the model accepts an effort/reasoning knob; only models that opt in enable the effort selector in chat.
• Structured output and output limit — whether typed output is supported and the largest
  requested result the model can accept.
• Reasoning modes — the specific effort modes the model supports.
• Routing metadata — expected quality/latency, operator preference, availability, rate
  capacity, locality, and the current immutable pricing snapshot used by optional adaptive routing.

Switching models per turn

You never have to leave a conversation to try another model:

• The model picker below the composer switches the chat's model for all following turns.
• An @-mention — @<model-id> or @<nickname> as the first token of a message — runs that single turn on the named model, then the chat returns to its selected model.
• The effort selector adjusts the reasoning effort for models whose capability descriptor opts in; it is a no-op elsewhere.

Routing is sticky in the default static mode. The opt-in adaptive router can
choose per task/role under explicit capability, privacy, cost, latency, and pin constraints.
The separate opt-in provider recovery policy can retry a safe
pre-response failure or move to another eligible model; it never replays a consequential tool or
continues automatically after visible partial output.

No silently selected provider

If no connection and default model are configured, the agent fails with instructions rather than
silently assuming a vendor or model. Provider recovery operates only after a real primary model is
configured; it is not a built-in provider. If a run fails with "No runnable provider", the host's
environment is missing the connection's API key—for a Windows service, set it machine-wide, not
per-user. See Troubleshooting.

Related pages

• Settings — the Settings page reference.
• Prompt caching — modes, isolation, diagnostics, provider retention, and
  recovery.
• Provider recovery — retries, fallback chains, circuits, replay safety,
  and diagnostics.
• Web Interface — the model picker and @-mentions in chat.
• Memory — embedding models (a separate choice from chat models).