Adaptive model routing

Adaptive routing can choose a different eligible model for planning, execution, review, summary,
embedding, vision, background work, or delegated coding. It is opt-in; Static remains the
default and preserves the model picker, manual @model choice, profile pin, sticky current model,
and first-suitable behavior.

Before you enable it

Configure at least two enabled models under Settings → Providers & Models. Edit each model and
verify:

capabilities: modality, context window, output limit, tools, vision, structured output, prompt
cache, and reasoning modes;
routing facts: expected quality from 0 to 1, expected latency, preference, availability, rate
capacity, and local/cloud locality;
pricing snapshot, when known, so projected-cost controls are meaningful.

Capability, privacy, cost, and latency limits are hard gates. A high score cannot make an
ineligible model usable. Incorrect model metadata can therefore produce a bad decision; keep it
conservative when facts are unknown.

Preview a route

Open Settings → Providers & Models → Adaptive routing. Choose a preset and limits, then use
Dry-run simulator. The simulator holds the example prompt in memory, invokes no provider, and
shows the intent, eligible set, selected model, component scores, and every rejection reason.

CLI examples:

holaryn routing simulate "Review this patch for correctness" --role reviewer
holaryn routing simulate "Inspect this screenshot" --vision
holaryn routing simulate "Return a typed result" --structured-output --output-tokens 2000
holaryn routing simulate "Use local data only" --local-only --tools
holaryn routing simulate "Plan this change" --model my-model-id

The JSON output resembles:

{
  "simulation": true,
  "configured_mode": "static",
  "decision": {
    "intent": {"role": "reviewer", "signals": ["explicit-role"]},
    "eligible_models": ["balanced", "premium"],
    "selected_model_id": "premium",
    "reason": "adaptive:quality-first",
    "candidates": [
      {"model_id": "economy", "eligible": false, "rejection_reasons": ["tools"]}
    ]
  }
}

The simulator deliberately previews adaptive behavior even while the configured mode is static.

Choose a policy

Quality first weights expected quality and reliability most heavily.
Balanced balances quality, reliability, latency, projected cost, cache opportunity, and
preference.
Economy favors lower marginal cost after all hard gates pass.
Local only rejects every cloud candidate.
Provider pinned restricts selection to one provider connection id.

Expert controls add model allow/deny/preference lists, a maximum projected per-turn cost, maximum
expected latency, role-specific role=model-id routes, and score weights. Supported roles are
planner, executor, reviewer, summarizer, embedding, vision, background, and
delegated-coding.

Allow local-to-cloud routing is off by default. Keep it off when a session begins on a local
model and its data must remain local. A manual or role pin always wins when eligible. An ineligible
pin fails clearly instead of silently selecting another model.

Enable and observe

Select Adaptive, save, and start or continue a session. A decision occurs only before a model
attempt. Activity includes:

route_decided: policy snapshot, classifier signals, candidates, scores, rejection reasons, and
selection;
model_switched: previous/next model, role, reason, and transcript strategy.

The same provider-neutral transcript is rendered by the chosen adapter. Routing never replays a
tool or approval and does not provide provider-failure recovery.

Use holaryn routing show --limit 20 for the active policy and recent content-free decisions.
Decision records contain no prompt/response body. Learn from aggregate outcomes is off by
default; when enabled, only bounded success, latency, and cost aggregates influence future scores.

Reset or roll back

Use the confirmed Settings reset button or:

holaryn routing reset --yes

Reset deletes retained decisions and aggregate outcomes. It does not change model configuration.
To stop adaptive selection, choose Static or set HOLARYN_ROUTING_MODE=static; this immediately
restores established pinned/sticky behavior for newly built runtimes.

Environment configuration

Settings writes validated values to the state directory's config.json. Environment variables
override that file:

HOLARYN_ROUTING_MODE=adaptive
HOLARYN_ROUTING_PRESET=balanced
HOLARYN_ROUTING_ALLOW_LOCAL_TO_CLOUD=false
HOLARYN_ROUTING_LOCAL_ONLY=false
HOLARYN_ROUTING_PROVIDER_PIN=
HOLARYN_ROUTING_MAX_TURN_COST_USD=0.05
HOLARYN_ROUTING_MAX_LATENCY_MS=4000
HOLARYN_ROUTING_ALLOW_MODELS=fast,balanced,premium
HOLARYN_ROUTING_DENY_MODELS=deprecated
HOLARYN_ROUTING_PREFERRED_MODELS=balanced
HOLARYN_ROUTING_ROLE_ROUTES=planner=premium,background=fast
HOLARYN_ROUTING_CONSENTED_LEARNING=false

Optional expert weights use
HOLARYN_ROUTING_QUALITY_WEIGHT, HOLARYN_ROUTING_RELIABILITY_WEIGHT,
HOLARYN_ROUTING_LATENCY_WEIGHT, HOLARYN_ROUTING_COST_WEIGHT, and
HOLARYN_ROUTING_PREFERENCE_WEIGHT, each from 0 to 10. Blank uses the preset.

Troubleshooting

No model satisfies adaptive routing requirements: inspect each candidate's rejection reasons.
Correct the model capability/locality facts or relax only the intended hard limit.
Unknown cost/latency: it is permitted unless a corresponding maximum is set; with a maximum,
the candidate fails closed because the ceiling cannot be proven.
Pinned model is ineligible: the pin is not bypassed. Remove it, make the model eligible, or
deliberately choose another model.
Provider-pinned requires a connection: enter the registry connection id, not a model id.
Routing state could not be opened: verify the state directory and routing.sqlite3 are
writable and not a newer schema. Do not edit the database manually.
Unexpected model selection: run the same input through the simulator and compare the policy
snapshot, metadata, role signal, score components, and aggregate sample counts.

Architecture, threat model, retention, extension rules, benchmark thresholds, and rollback details
are in the adaptive-routing ADR.

Next → Annotations