Principled workflow for meta-analysis of multicategory diagnostic test by a Bayesian multinomial hierarchical model

This resource investigates the validity of a Bayesian hierarchical model in meta-analysis of a multicategory diagnostic test using the example of breast fine needle aspiration biopsy. A published meta-analysis is taken as the basis of this demonstration. The principled workflow includes prior predictive checks, algorithmic calibration by simulation-based calibration, inferential calibration including shrinkage analysis and parameter recovery, convergence checks and posterior predictive checks. Only the relevant portions of the last two have been given as this document deals only with validation of the model and not the final results. The final results along with derived parameters (likelihood ratios and predictive values) will be the focus of a speciality specific publication. In brief, we first propose a model, define the priors and check for reasonableness of priors by a prior predictive check. Once this reasonableness is satisfied, we check the performance of the model on ideal data. The validity/calibration of the model and sampler (NUTS through brms and Stan) is checked across the entire range of priors by 1020 simulation runs (Simulation-based calibration). Then the usefulness/informativeness of the results are checked by examining the shrinkage and parameter recovery from the data already generated in the simulation-based calibration experiment. Doing these two shows the validity of the software, the Bayesian algorithm (hierarchical model using brms and NUTS) and the usefulness of the model in ideal conditions. Following these , we check whether the software has run successfully on the real data. Finally, we check the closeness of the real data to the assumptions of the model by posterior predictive checks.The code for the meta-analysis is also published