Covering the Query Intent Does Not Predict Rank: Two Blind-Scored Studies of Russian-Language Search Results, the Second One Preregistered

Replication package for two studies measuring whether pages that cover more of a query's intent rank higher in Russian-language Yandex results. For each query, a map of intent components was built from query-side evidence only — the query, sibling queries of the same cluster, Wordstat top phrases and associations — and frozen before any ranked document was opened. Assessors who never saw a document's rank position scored every page against that map on a three-point scale. Study 1 covered 65 queries, 544 pages and 13 clusters; Study 2 replicated the procedure on 97 new queries, 1057 pages and 20 clusters under a protocol registered before the first result list was captured. Result. Intent coverage does not order documents inside a result list. The median within-query Spearman correlation with rank position was +0.033 in Study 1 and −0.047 in Study 2, with a 95% cluster bootstrap interval of [−0.104; +0.019] — inside the preregistered band of practical zero in its strong form. Of the two remaining preregistered hypotheses, both failed: BM25 came out weaker than coverage rather than stronger, and Domain Rating replicated at −0.128 against a predicted −0.35 to −0.15. Seventeen exploratory on-page features separated a higher-ranked page from a lower-ranked one in at most 53.3 pairs out of 100. An exploratory by-product: the single top position covers the query intent worse than any other band of the top 20, and in law and finance it covers roughly half of what positions 11–20 cover, because rank one in those niches is frequently a calculator, a form or a government service page. Contents. The preprint (PDF and LaTeX source), 2655 rows of scored observations as CSV, the frozen intent maps, the computed results, the analysis code including a script that recomputes every preregistered number from the published CSV, and the preregistration with its full deviation log. Not included. Raw HTML of the ranked pages: third-party content, about two gigabytes. Every document is identified by URL, query and rank position. Text, figures and data are released under CC BY 4.0; code under MIT.