How do my distributions differ? significance testing for the overlapping index using permutation tests

Abstract Psychological research frequently relies on statistical tests targeting single distributional parameters, typically means, despite empirical data often differing in variance, skewness, or overall shape. We introduce the $$\zeta _{\textrm{ov}}$$ ζ ov test, a permutation-based inferential procedure built on the Overlapping Index, an effect size quantifying similarity between empirical distributions. The proposed approach evaluates global distributional differences without relying on parametric assumptions. Through simulations manipulating mean, variance, skewness, and sample size, we examine the $$\zeta _{\textrm{ov}}$$ ζ ov test alongside commonly used tests (t, Welch, Wilcoxon–Mann–Whitney, Kolmogorov–Smirnov, and variance tests), while acknowledging that these tests address different null hypotheses. Results indicate that the $$\zeta _{\textrm{ov}}$$ ζ ov test maintains adequate Type I error control under the simulated scenarios and shows comparatively high sensitivity to distributional differences, particularly when these involve more than a single parameter. An applied example using reaction-time data shows how distributional overlap detects differences missed by mean-based analyses. Rather than replacing traditional tests, the method provides a theoretically aligned global assessment that encourages distribution-aware inference and integration of visualization and descriptive analysis into statistical workflows. The $$\zeta _{\textrm{ov}}$$ ζ ov framework supports ongoing methodological shifts in the psychological sciences toward robust, assumption-light, and interpretable statistical reasoning.