Coverage Metrics for T-Wise Feature Interactions - Summary

In this work, we investigate 𝑡-wise feature interaction coverage metrics, originally published at the 18th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2025 [Bö25]. Most software systems are typically configurable by means of configuration options (features). As testing every possible configuration of a system is infeasible, 𝑡-wise sampling has been proposed to systematically derive a representative subset of configurations that covers all interactions between at least 𝑡 features. As testing a complete 𝑡-wise sample may still be infeasible due to restricted resources, practitioners started to use smaller samples and aim to optimize their 𝑡-wise coverage to be as close to 100% as possible for a given 𝑡. However, there is no consensus in the literature on how to exactly compute 𝑡-wise coverage for a given sample. We propose the first systematic framework to define coverage metrics for 𝑡-wise feature interactions. These metrics differ in the set of features and set of feature interactions being considered. We found evidence for at least six different metrics in the literature. In an empirical evaluation, we show that for a partial sample the coverage differs up to 21% and for some metrics only half of the feature interactions need to be considered. As a long-term impact, our work may help to improve the efficiency and effectiveness of both, 𝑡-wise sampling and coverage computations.