Evaluating the Instrument to assess the Credibility of Effect Modification Analysis (ICEMAN) - review of citing studies and user-interviews six years after publication

Background: 6 years after publication, many researchers have used and cited the Instrument to Assess the Credibility of Effect Modification Analysis (ICEMAN). Objectives: To assess how researchers applied ICEMAN to various types of studies and to identify possible challenges and user needs. Methods: We will perform two linked studies. In a review of ICEMAN applications, we will use citation tools to retrieve all publications that cited ICEMAN, classify publication types (study protocols, study reports, other) and types of studies (randomized trials, meta-analyses, other). For all protocols and reports, we will describe how researchers referenced and applied ICEMAN. For all reports that provide a completed ICEMAN instrument, we will assess individual items and the overall credibility rating for comprehensiveness, clarity, and appropriateness. In a second study, we will conduct semi-structured interviews with 30 recent ICEMAN users (excluding developers of ICEMAN), to identify practical challenges in applying ICEMAN and suggestions for improvement. Discussion: The studies will document current practice and identify practical challenges in applying ICEMAN, possible misapplications, and the users’ needs and suggestions for improvement. The findings will inform future versions of ICEMAN and may lead to additional instructions and templates to support uptake and appropriate use of ICEMAN.