Is the Search for Grey Literature Worth the Effort?

The gold standard of academic literature is considered reliable based on peer review and expertise. Specifically, publication following peer review ensures that information published is as “truthful, valid, and accurate as possible.”[1] Such assumptions are based on the expertise and rigor of the author(s) and those who have critically (and hopefully, fairly) reviewed it. Such literature is the cornerstone of science and is therefore, the “white” literature, representing information sieved through careful eyes.[2] These are typically “peer-reviewed literature” or “indexed academic literature.” To select the best of the best information, rigorous standards are maintained in peer-reviewing and publication; however, many papers, writings, and articles fail to pass through this scrutiny or might not even be considered for scrutiny. What happens to the information? Does it get lost in time and become futile? This literature, which is not formally peer-reviewed and published, thus becomes the grey literature. This literature is tentative, questionable, imperfect, and perhaps “impure” but still has valuable information. We point out why, when, and how much of this literature should be used to improve our current scientific understanding. We outline common sources of grey literature and position them along a “spectrum of shades of grey” [Figure 1]. At the lighter end of the spectrum, lie sources originating from individuals or institutions with recognized expertise, where the information is subject to some regulation and review.[2] For instance, a student’s dissertation, though not formally published, is guided by mentors and evaluated through academic processes. Therefore, its findings or arguments are relatively reliable and of higher academic quality.Figure 1: The spectrum of grey literatureAt the other end of the spectrum are sources with little or no formal scrutiny, such as comments on social media posts about a published article or blog posts. These remarks may, in some cases, come from genuine experts, but without conventional checks, their reliability is uncertain. Interestingly, if the same expert were to transform that comment into a formal journal commentary, it would pass through editorial and peer-review processes. In doing so, the content gains credibility and is more readily accepted within the broader scientific community. WHY USE GREY LITERATURE? A simple answer is to broaden the scope of our research question, of course, without compromising its scientific rigor.[3] Drawing on grey literature can be particularly useful when the research question is exploratory and the availability of literature is limited. In some cases, sources such as personal communications or group discussions may take precedence, especially when we need to understand the cultural nuances. By contrast, if the aim is to conduct a systematic review on the efficacy of an intervention, grey literature has little role, and only published randomized controlled trials meet the required standards. The “shades” of grey literature shift according to the research question. For example, social media posts might serve as primary data sources if the focus is on youth attitudes toward vaccination. The key point is that grey literature is not a fixed entity; its value depends mainly on how exploratory the research question is. WHEN TO USE GREY LITERATURE? Exploring grey literature becomes essential when “white literature” fails to provide sufficient answers. For instance, if we seek information on the acceptability of a pharmacological agent and no controlled studies exist, we may turn to open-label studies, case series, reports, personal communications, or patient blogs.[3] While the latter two sources may lack formal expert scrutiny, they still contribute valuable insights relevant to the research question. Integrating information from such grey sources can often be more informative than ending up with no data due to the absence of high-quality studies. HOW MUCH OF THE GREY LITERATURE SHOULD BE USED? Grey literature can contribute valuable insights to a research question. Inclusion of the grey literature should not reduce the methodological quality or render the findings overly broad or unreliable.[2,3] For example, when developing guidelines on the efficacy of a new treatment, only “white literature ”– formally published studies – is appropriate, as rigorous peer review and expert validation are essential. Including grey literature in such contexts would not add meaningful value. The most commonly incorporated grey sources in systematic reviews include annual reports, commissioned and organizational reports, conference proceedings or abstracts, discussion papers, editorials, government reports, industry and institutional reports, policymaker consultations, and dissertations.[2] When both grey and white literature are used, distinguishing the results from each source is essential. For instance, systematic reviews should ideally perform sensitivity analyses using only white or grey literature to identify systematic differences. THE FUTURE: GREY MATTERS In recent years, there has been an increase in the use of grey literature in academic articles and in policymaking.[4,5] Rational use and analysis of the information obtained from grey literature improve several aspects of our research question. It shows such trends in science and publication, and nudges toward a more nuanced understanding of a topic, as it encompasses views from several perspectives.[6] Furthermore, artificial intelligence (AI)-assisted automation of grey literature reviews is likely to emerge as a promising approach to enhance the efficiency and comprehensiveness of literature searches.[7] Awareness of the grey literature surrounding our topics of interest helps develop expertise and insight and facilitates our next research question. Universities and institutions should develop a method of providing the list of dissertations that have not been published in periodicals to enable researchers to access them. There is a need to evaluate the quality of grey literature if it is included in systematic reviews. For a scientifically inquisitive brain, the grey matter does include the grey literature. Author contribution statement SBN: Conceptualization, design, and first draft. GD: Conceptualization, design, first draft, and revision. Disclosure of use of AI assistive or generative devices Generative AI has been used to solely improve the manuscript’s grammar and writing style. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.