Reply to Tang and Sun, and to Deng and Liu

From the authors: We thank Tang, Sun, Deng and Liu for their interest in our recent publication where we show patients with chronic Pseudomonas aeruginosa infections have heterogeneous endotypes, with a post hoc analysis showing these endotypes and geographical region are associated with changes in antibiotic treatment responses in bronchiectasis.1 Tang and Sun have raised a number of important points ­regarding the development of a prognostic or predictive biomarker/model, in particular the dangers of “double-dipping.” We completely agree with these concerns and the impact on the generalizability of any biomarker developed in this way. However, in the context of this paper, we do not believe these concerns apply. We identified a number of features that show promise in terms of prognostic and predictive ability, but we have stopped short of proposing a single biomarker that we believe is ready for external validation or implementation. This paper is inherently hypothesis generating and adds to a large body of previously published work on potential biomarkers.2 Rather than a focus on any of these markers alone, we believe a more valuable next step might be the development of a panel combining multiple markers. In the paper, we explore sources of treatment effect heterogeneity, a longstanding question in the field, through sensitivity analyses. The approach taken is in line with European Medicines Agency guidance on adjusting for baseline covariates in clinical trials3 and, in particular, baseline imbalances observed post hoc, to assess how robust the main analysis is. We are, by the very definition of a sensitivity analysis, deliberately seeking baseline features that are both related to outcome and imbalanced between trial arms, to examine the stability of the efficacy endpoint. Linear discriminant analysis (LDA) is a supervised dimension reduction technique, reducing the dimensionality of the microbiome data while maximizing the separation between the trial arms (and trials). The rationale for this was to focus on the microbiome features that are imbalanced between arms; as microbiome data are complex and variable, we felt there was value in looking at LDA in addition to the unsupervised nonmetric multidimensional scaling dimension reduction, which may capture the biggest sources of variability common to both trials/arms. We reiterate that we do not consider this “double-dipping” in this context, and that we have not proposed the LDA-based approach, nor any other feature, as a biomarker for future use. Indeed, we view complete microbiome profiling as an impractical biomarker in the context of a future global, multicenter, clinical trial. This is about understanding disease, not clinical prediction. We believe the value of these sensitivity analyses are in demonstrating that major sources of heterogeneity in efficacy results are from variation resulting from geographical region and microbiome profile; these points and their implications are discussed further below. Deng and Liu highlighted the importance of geographical differences in the design and analysis of clinical trials. Recent papers have found important geographical difference between patients in the bronchiectasis registries of EMBARC,4 EMBARC-India,5 and BE-China,6 in metagenomic analyses7,8 and this paper,1 which all show 1 consistent message: bronchiectasis severity, outcomes, and the microbiome are highly influenced by geography and this is a key consideration for future trials and trial design. Accounting for geographical region may cover many aspects of variability, and in this setting may act as an easy to measure “surrogate” for key microbiome features. The authors’ suggestion for stratification by geography and biomarkers, and analysis accounting for geographical clustering, is a sensible proposal, and we fully support the principle. However, we are reluctant to be too prescriptive about the exact nature of stratification, as there is still much work to be done to develop validated prognostic/predictive biomarkers that are widely applicable and accepted. Optimal analysis approaches for site adjustment have been discussed extensively elsewhere,9,10 and the exact approach taken may depend on, for example, the level of clustering (site vs country vs geographical region) and computational issues related to sample size per cluster.3 Ultimately, we are excited about the interest and discussion generated by this work. Our results have shown a plausible relationship between geographical regions and the microbiome which, when building on a body of research, are also likely to have a relationship with inflammation, which is known to influence patients’ outcome/response. Considering the biological ­plausibility of these results, they are likely to be generalizable. The best validation of these results will be to apply our learnings from this work, develop effective biomarkers, and apply them to future trials, which gives the potential for improved targeted and effective antibiotic treatments in bronchiectasis. Rebecca C. Hull, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Jamie Stobo, MSc (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Hani Abo-Leyah, MD, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Hollian Richardson, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Daniela Alferes de Lima Headley, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Merete B. Long, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Chandani Hennayake, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Amy Gilmour, BSc (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Emma D. Johnson, MD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Michael Tunney, PhD (School of Pharmacy, Queen’s University Belfast, Belfast, United Kingdom), Alison J. Dicker, PhD (Division of Population Health and Genomics, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Eleanor Kewin, MBiol (Division of Population Health and Genomics, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Jeffrey T.J. Huang, PhD (Division of Cardiovascular Research, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom), Charles S. Haworth, MBChB, MD (Cambridge Centre for Lung Infection, Royal Papworth Hospital and University of Cambridge, Cambridge, United Kingdom), and James D. Chalmers, MBChB, PhD (Division of Respiratory Medicine and Gastroenterology, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom). Supplementary material is available at American Journal of Respiratory and Critical Care Medicine online. Please see the ICMJE disclosure forms, which have been provided as supplementary material.