Make the LLMs Argue: Quorum, a Deliberation Protocol with Dissent Preserved

For high-stakes, ambiguous questions (strategy, pricing, architecture, risk) a single language model produces a confident answer with no visibility into its blind spots, and nothing in its fluency distinguishes a robust conclusion from a well-phrased guess. The disagreement between independent models is the best available signal of genuine uncertainty, and today it is thrown away: merged into consensus summaries that destroy precisely the information worth collecting. This paper presents Quorum, an open-source deliberation protocol that convenes three models from distinct labs to answer a question in enforced isolation, critique each other claim by claim under blind labels, revise, and hand a non-debating arbiter the job of delivering a verdict with dissent preserved. The protocol makes an unusual bet. It does not claim better answers, since the evidence for that is modest and task-dependent, and a three-arm benchmark (council vs. single model vs. self-critique) exists to test rather than assume it. It claims calibration material: the transcript is the product. Anonymization is structural rather than cosmetic. Students exchange rigid answer sheets of numbered claims, not prose, so style signal is never generated in the first place; the critique round has no field for agreement; and position changes are computed by the engine rather than taken from the models' own declarations. Independence constraints live in the type system, sessions fail closed per round, and every session emits a replay-complete JSONL trace from which the entire deliberation (positions, objections, diffs, absences, verdict) can be reconstructed. The engine is complete, with 330 tests and protocol invariants enforced by construction. Live mixed-lab councils (Anthropic, OpenAI, DeepSeek) have been run and measured: a 33% position-change rate inside the 15-60% design band, 13 preserved minority items, and two targets missed and reported as findings, namely 77% claim compliance against a 90% target and a 23.7x session cost against a designed 8x guardrail, with the arbiter measured at 69% of a session's bill. A convening rule that fails closed keeps councils rare: 9.3% of a 43-task mixed workload convenes. The deanonymization probe and the benchmark harness are built and tested but have run only against mocks, which their own output labels as proving nothing about models; the numbers will be published whatever they say. The paper closes with limitations stated where users meet them, and a roadmap that consists of measurement rather than code. Canonical version and illustrated protocol walkthrough: https://sheepdog.systemsCode, evals, committed live traces, and full findings: https://github.com/JoaquinDG/quorum