# LiteLLM Routing — Operational Notes Operational notes + migration log for `litellm-config.yaml` route changes where the rationale isn't obvious from the config diff alone. Keep this file narrow: **only entries where a future reader would otherwise not know why we chose a particular upstream slug or fallback policy.** --- ## Sprint 10 Task 1.4 — DashScope intl primary + OR failover (2026-04-21) **Key provisioning.** A classic DashScope Model Studio API key (`sk-…`, not `sk-ws-…`) was provisioned on 2026-04-21 and is held in two places: - `.env` under `DASHSCOPE_API_KEY=sk-…` (LiteLLM container reads at startup via `os.environ/DASHSCOPE_API_KEY`). - Waggle vault under name `alibaba` (credentialType `api_key`). The key is bound to the **international tenant**, not mainland. Smoke verification (2026-04-21): | Endpoint | Result | |---|---| | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions` | HTTP 200, target `qwen3.6-35b-a3b` returns valid completion with `reasoning_content` populated (thinking default on) | | `https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions` | HTTP 401 `invalid_api_key` — key is intl-bound | **Routing decision.** `litellm-config.yaml` now carries three Qwen3.6 routes: | Alias | Upstream | Role | |---|---|---| | `qwen3.6-35b-a3b` | `openai/qwen3.6-35b-a3b` @ `dashscope-intl.aliyuncs.com/compatible-mode/v1` | **Canonical primary** — real 3.6 model | | `qwen3.6-35b-a3b-via-dashscope` | identical to canonical | **Explicit DashScope pin** — for operator scripts that want to bypass a canonical alias flip | | `qwen3.6-35b-a3b-via-openrouter` | `openrouter/qwen/qwen3.5-35b-a3b` | **Failover** — one-minor regression to 3.5 (OR catalog lacks 3.6-35b-a3b as of 2026-04-21) | **Failover policy.** Caller-side responsibility, not LiteLLM router: 1. First call: `qwen3.6-35b-a3b` (canonical, DashScope-intl). 2. On HTTP 429 rate-limit or 5xx within a reasonable retry window (client-judgment), retry on `qwen3.6-35b-a3b-via-openrouter`. 3. OR path returns Qwen3.5-35B-A3B, so log the model substitution in the caller's trace record so Week-1 / Stage-2 aggregate reports attribute correctly. The `judge-runner.ts` trace already carries `judge_model`, so a failover just writes the failover model name to that field; no schema change needed. 4. Do NOT use `openrouter/auto` as a third-tier failover — it can silently route to an unrelated model and pollute cost accounting. LiteLLM's per-route `fallbacks` field is supported (would let us declare failover in config rather than caller logic) but deliberately not used here: the Week-1 / Stage-2 trace schema relies on precise per-call model attribution, and letting LiteLLM transparently swap models would make traces harder to audit. **Regression gate.** `scripts/smoke-qwen-dual-route.mjs` calls both routes with the same prompt, verifies both return HTTP 200, logs per-route completion, latency, and token counts. Task 1.1 (Qwen stability matrix) uses the canonical alias for its live run — which means the matrix now exercises the real 3.6 model, not the 3.5 OR fallback. Materially improves Stage-2-prep defensibility. **Follow-up for Marko (when available).** If the key later gets mainland provisioning too, a second route `qwen3.6-35b-a3b-via-dashscope-mainland` can be added with the same key pattern — but the intl endpoint is sufficient for Sprint 10 + Stage 2 benchmarks. --- ## Sprint 10 Task 1.2 — Sonnet 4.6 dated-snapshot repair (2026-04-21) **Problem.** The `claude-sonnet-4-6` alias previously routed to `anthropic/claude-sonnet-4-6-20250514`. That dated-snapshot suffix was never a valid Claude API ID for the Sonnet 4.6 family. **Where it was caught.** Sprint 9 Task 4 calibration (commit `f9b98aa`) attempted to exercise the Sonnet route and LiteLLM returned an error consistent with a `model_not_found` response. The Sprint 9 session substituted `claude-opus-4-7` for the calibration gate (10/10 PASS) and queued the Sonnet route repair as Sprint-10 operational work. **Verification.** Re-fetched the Anthropic models overview on 2026-04-21 (docs redirect target: ). Current table confirms: - **Claude Sonnet 4.6** — `Claude API ID: claude-sonnet-4-6`, `Claude API alias: claude-sonnet-4-6`. - No valid dated-snapshot with the `-20250514` suffix exists for the Sonnet 4.6 family. The `-20250514` snapshot belongs to the deprecated Claude **Sonnet 4** (single-digit) family, not Sonnet 4.6, and that family is scheduled for retirement on 2026-06-15 per the public deprecation notice. **Fix.** Repointed the native Anthropic target from `anthropic/claude-sonnet-4-6-20250514` to the plain alias `anthropic/claude-sonnet-4-6`, with matching fixes to the two legacy aliases (`claude-3-5-sonnet-20241022` and `anthropic/claude-sonnet-4-6`) that routed to the same broken target. **Scope discipline.** The adjacent `claude-opus-4-6-20250610` route uses the same dated-snapshot pattern and is therefore a candidate for the same defect. It is **out of scope for Sprint 10 Task 1.2** — Sprint 10 runs judges on `claude-opus-4-7` (verified good in Sprint 9) and `claude-sonnet-4-6` (this PR). Opus 4.6 will be audited separately only if/when a caller needs it. No speculative edits. **Regression gate.** `scripts/smoke-sonnet-route.mjs` is the one-shot HTTP probe that verifies the repaired route returns 200 (not 404 `model_not_found`) on the LiteLLM proxy. Task 1.3 (judge calibration on Sonnet) is the full functional regression — it replays the 10 ground-truth triples through the repaired route and reports match rate vs. PM human labels. **PM review gate (open).** Per Sprint 10 brief §1.2, Marko ratifies the mapping before this PR merges. Open question for the reviewer: do we also want a separate PR to audit the Opus-4.6 dated snapshot, or leave that out until a caller trips on it? --- ## Existing route conventions (carried forward) ### Dated snapshots vs plain aliases Prefer plain family aliases (`claude-opus-4-7`, `claude-sonnet-4-6`) unless a specific dated snapshot is required for reproducibility. Plain aliases automatically follow Anthropic's "current" pointer within a family and survive version bumps without requiring config churn. Dated snapshots are stable indefinitely once live but carry two risks: 1. Anthropic can decommission a dated snapshot without migrating the alias forward — the 2026-04-21 incident (Task 1.2 above) is a canonical example. 2. Dated snapshot IDs are easy to invent with the wrong date format. A 404 on a dated ID always warrants re-verification against the live docs before trusting the old value. ### OpenRouter bridge routes Several routes of the form `--via-openrouter` exist as parallel paths through OpenRouter for providers where the native key is not provisioned or where OR has a more stable upstream (notably Kimi K2.x, Qwen3.x families). Keep the primary alias pointing at the native provider when possible; OR routes are failover, not default. ### `drop_params: true` LiteLLM's global `drop_params` setting strips unsupported params. That includes provider-specific extras like Anthropic's `thinking` or DashScope's `extra_body.enable_thinking`. If a caller needs a param that LiteLLM would otherwise drop, either (a) route around LiteLLM for that specific call (see `scripts/stage-0-query.mjs` for the pattern), or (b) pass the param through a route that uses the provider-native SDK. --- *Maintained as part of Sprint 10 operational queue close-out. Extend this file with a new `##` section per route change that needs operational-memory preservation.*