Stop Sending Everything to the Smartest Model: Policy-Based LLM Routing with Cross-Model Auditing

Production LLM systems make the same decision thousands of times a day; which model gets this task?, and almost always make it implicitly, as a hardcoded model id chosen once under deadline pressure. The result is one of two bad equilibria: overspending 10–30x by sending everything to a frontier model, or absorbing downstream quality failures by sending everything to a cheap one. With serious vendors now spanning a 178x price spread per million output tokens, this invisible decision has become a real, unmanaged engineering economy. This paper presents Switchboard, an open-source LLM model brokerage that makes five implicit things explicit and versionable: the catalog (cost, latency, rated capability per task type), the policy (named quality/cost/latency weights), the triage (task type and complexity inferred with a confidence-gated hybrid classifier), the verification loop (a cross-lab auditor; never the producer, and preferentially from a different provider, since same-lab models share blind spots), and the bill (every attempt costed against a best-model baseline, audits included). Hard gates run before soft scores: no model wins work it is not rated for, however cheap. The system fails closed and degrades upward, is roughly 1,900 lines of dependency-free Python, and reproduces fully offline (216 tests, routing and triage eval harnesses). Results are labeled by realism. Computed at real vendor prices, routing a mixed workload costs 96.7% less than frontier-everywhere; a five-step agentic pipeline routes for an estimated 78% less. Measured live across five providers, cross-lab auditing caught real defects: including a frontier model inventing prices never present in the prompt, and revealed the honest headline: verification, not generation, dominates total cost, with audit overhead reaching 74–95% of spend. Routing solves the generation bill; the verification bill is the next frontier. Capability ratings are labeled estimates; triage accuracy is reported held-out (60%) separately from tuned (100%); all limitations are documented where users encounter them. Every number in the paper traces to a runnable command. Canonical version and interactive playground: https://sheepdog.systemsCode, tests, evals, and full findings: https://github.com/JoaquinDG/switchboard