The AI Recommendation Quotient: A Multi-Turn Instrument for Measuring Whether AI Assistants Recommend a Brand

Read full text
Authors: Paul Sheals, Tim de Rosen
Cited by: 1
References: 2

Abstract

Most AI-visibility measurement asks a single question and counts whether a brand appears in the answer. That is a real signal, but it is not the commercial one: buyers compare, object, and ask a final question that resolves to a purchase, and the brand named at the opening is frequently not the brand recommended at the close. The AI Recommendation Quotient (ARQ) measures the whole conversation to its commercial end.This paper defines the instrument in full: the four-turn probe (unprompted salience, comparison, a sceptical objection, and the recommendation itself); the shared-context rule that holds the opening answer identical across every brand in a category; the LLM-as-judge rubric and the numeric mappings for the three extracted signals; the ARQ composite and its weights; and the derived measures, including the Linkage Gap between being found and being chosen.Two design decisions are documented explicitly. First, retrieval is treated as a controlled variable rather than an artefact of platform choice: the primary axis holds the model constant and varies only whether it may consult the live web, because earlier configurations confounded the two. In one measured category, every brand received some recommendation credit on a retrieving engine against 17 and 19 of 20 on closed-book engines — whether a brand is reachable at all, not merely how it ranks, depends on the channel. Second, the paper discloses a property of its own headline index that is easy to misread: the recommendation measure is a blended score, awarding partial credit where a brand is shortlisted but not chosen, and is therefore not a win rate. Across ten categories and 200 brands measured to date, 102 brands — 51 per cent — never once emerged as the assistant's actual pick, while some carried a blended figure as high as 38.9 per cent. Outright wins must be reported alongside the composite, and a prior publication of the authors' that stated the blended figure as a recommendation rate is corrected here rather than defended.Limitations, sample-size constraints on rank order, judge-model dependence, language and market scope, and the provenance of previously published categories measured under an earlier configuration are all stated. Prompt sets, the judge rubric, transcripts and scored records are available on request. The instrument is published because a ranking that names real companies should be contestable by them, and because a measurement that cannot be interrogated should not be trusted — including this one.Version 1.1 (26 August 2026) — terminology correction. Version 1.0 used the name "Linkage Gap" for the measure Salience@T1 minus Recommend@T4. That usage was ambiguous: "Linkage Gap" names the underlying phenomenon — that being found and being chosen are separate events — which is measured in more than one way within this programme. The measure defined in section 4 is now named the salience–recommendation gap, and is distinguished from the fact-level linkage gap (the share of facts a model holds that go unused when it recommends, 75.7%, n=592). No figure, formula or finding changes; only the name of one derived measure. Earlier citations resolve unchanged via the concept DOI.

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 August 2026
Language
EN