Components and methodology of evidence briefs for policy: the need for evaluation tools

Abstract Background Evidence briefs for policy (EBPs) are effective tools for delivering research evidence to policymakers and other stakeholders by highlighting high-priority issues, outlining options and considering implementation strategies. However, policymakers’ demands for evidence and policy-relevant information across different fields have led to variability in the terminology used to describe EBPs, and the methodological quality of these EBPs remains unclear. This study aims to (1) identify organizations whose definitions of EBPs contain the three key components of problem, options and implementation considerations, (2) assess the methodological quality of EBPs that incorporate these three key components and (3) identify existing evaluation/assessment tools of EBPs. Methods A two-stage documentary analysis approach was used. First, we identified documents that were produced by organizations/institutions to inform policymakers and that contained the three key components (Problem, Options and Implementation considerations). Second, the methodological quality of the documents was assessed from the perspectives of the evidence supply side (that is, evidence synthesis) and the evidence demand side (that is, mapping of and engagement between both policymakers and stakeholders). Results In 22 organizations, the term policy brief was the most commonly used, accounting for 50% of organizations, while other terms varied. Issue briefs were used by three organizations (13.6%) and evidence briefs were used by two organizations (9.1%). In total, 50 individual documents from nine different organizations were included to evaluate components and methodology. (1) From the supply-side perspective: 17 (34%) documents described the search resources, 10 (20%) documents described evidence certainty and 15 (30%) assessed the methodological quality of the research evidence. (2) From the demand-side perspective: 30 (60%) documents were developed in response to demand-side needs, while 27 (54%) included both stakeholder mapping and engagement. Conclusions Methodological shortcomings were identified in the EBPs from both the supply-side and demand-side perspectives, highlighting the need to validate and better implement existing tools and to complement existing guidelines.