A Practical Guide to Assess Data Distribution and Normality with Open-Source Software

Introduction In data analysis, choosing the right statistical test is very important.[1] A wrong choice can give misleading results. One key assumption is the distribution of the data. It shows how values are spread on a scale. Different tests need different types of distribution.[2] For example, when comparing a continuous variable between two independent groups, an unpaired t-test is used if the data are normally distributed. If the data are not normally distributed, the Mann–Whitney U test is used instead.[3] If researchers ignore this, the results can be misleading. Many biomedical researchers and residents skip checking data distribution before analysis. This may be partially due to lack of awareness[4] and partially due unavailability of proper software. In this context, we describe the basics of data distribution and developed a free software application that tests the normality of data. Normal or Gaussian Distribution There are several types of probability distributions commonly used in medical research, summarized in Table 1.[5,6]Table 1: Common distribution of data in biomedical researchOne of the most significant and often applied probability distributions in statistics is the Gaussian distribution, known as the normal distribution. It is named after the German mathematician and physicist Carl Friedrich Gauss.[7] The Gaussian distribution is symmetric and forms a bell-shaped curve [Figure 1]. It is symmetrical all the way around its mean, which is also the mode and median of the distribution.Figure 1: Visualization of normal distribution, skewness, and kurtosisDeviation from Normality Deviation from normality can be defined as either skewness or kurtosis. Figure 1 shows the skewness and kurtosis pattern. Skewness (skew: a slant) measures the asymmetry of the data.[8] In simpler terms, it tells researchers whether the data are more concentrated on one side of the average (mean) than the other. It may be of two types – positive or negative. When data are positively skewed, the right-hand tail of the distribution stretches out more. On the other hand, when data are negatively skewed, the left-hand tail extends further. Kurtosis gives idea about how the data are peaked.[9] It tells researchers whether the data have more extreme values (outliers). The distribution may be of three types: leptokurtic, mesokurtic, and platykurtic [Figure 1]. A leptokurtic distribution has a tall central peak, meaning the data are tightly grouped around the mean but also show more extreme values. A mesokurtic distribution looks similar to the normal distribution, with a moderate peak and balanced tails. In contrast, a platykurtic distribution has a flatter peak and lighter tails, showing that the data are more spread out with fewer extreme values. Tests for Checking Normal Distribution Normality can be checked visually (with graphs) or using statistical tests, which give a number and a “P value” to decide if the data are normal or not. In these tests, P < 0.05 indicates that the data are not normally distributed.[10] Some of the tests and their brief descriptions are shown in Table 2.Table 2: Tests for checking the normal distribution of dataVisual methods 1) Histogram: A histogram is a graphical representation of the distribution of numerical (continuous or discrete) data, where values are grouped into contiguous intervals (bins) and the height of each bar represents the frequency (or density) of observations within each interval.[11] By looking at its shape, researchers can get a quick sense of the distribution: A bell-shaped, symmetric histogram suggests that most values are near the average [Figure 2a].Figure 2: An example of a (a) histogram and (b) Q-Q plot of normally distributed data2) Quantile-Quantile Plot: A Q-Q (quantile-quantile) plot is a scatter plot that compares data to a perfectly normal distribution.[12] If the points fall roughly along a straight diagonal line, the data are close to normal [Figure 2b]. Deviations from the line indicate non-normality: Points curving upward at the right suggest a positive skew, and points curving downward at the left suggest a negative skew. Calculation method Some of the calculations are described below, and other tests are shown in Table 2.[10,13] For all tests, P < 0.05 indicates that the data are probably not normal. 1) Shapiro–Wilk Test: The Shapiro–Wilk test is a popular method to check for normality. It compares your data with a perfectly normal distribution and tells you how similar they are. This test works well for small- and medium-sized datasets.[3] 2) Kolmogorov–Smirnov Test: The Kolmogorov–Smirnov (K-S) test is another way to see if the data match a normal pattern. It looks at how far the data are from a standard normal curve. This test is usually used for larger datasets, but it is less sensitive to unusual values at the edges (tails) of the data.[14] 3) Anderson–Darling Test: The Anderson–Darling test is similar to the others but pays more attention to the edges of the data, where extreme values occur.[15] It is useful in biomedical research because sometimes rare high or low measurements (like very high blood sugar) can affect results. A Free Application for Normality Test There are statistical software packages or online services that provide options to test the distribution of data. However, with an aim to create free software specially for normality tests, an R-based Shiny application was developed. NormSight was designed as a freely accessible application code where users can easily perform different types of normality tests, visualize their data through histograms and Q-Q plots, and receive clear interpretations of the results. To use the NormSight app, first, researchers need to ensure that their computer has R, R Tools, and RStudio installed. They are available for free: R: https://cran.r-project.org/bin/windows/base RStudio: https://posit.co/download/rstudio-desktop RTools: https://cran.r-project.org/bin/windows/Rtools Now, researchers need to download a Microsoft Word document (from doi: 10.17632/2jp9j5ytfy.2) that contains the code of the NormSight application. This Word document has commands for installing the required packages and running the software. To open the application, researchers need to first open the RStudio software. On the console, they need to paste the code and click “enter” (in Windows) or “return” (in Macintosh). It will take some time to install the packages and then open the user interface [Figure 3].Figure 3: User interface of the NormSight Shiny application running on RHow to Test Data in NormSight After the app opens, researchers can download [Figure 3] a sample comma-separated value (.csv) file and use it as a template. In this CSV file, they should enter their data and save it. Next, they should click on ‘browse’ and upload the CSV file, select the normality test of their choice, and click on the “Run Full Analysis” button. This will show the result in the “Primary Result” tab. In addition, it will show the central tendency and shape of data distribution, histogram, and Q-Q plot (visuals tab) and compare all test results (all tests summary tab). For using the visualization in a thesis or research paper, they can save the images in JPG or PNG format. Limitations This tool is limited by the normality tests available in existing R packages. Therefore, its functionality and accuracy are constrained by the assumptions, algorithms, and updates of those packages. Furthermore, the tool is intended for univariate assessment only and does not replace expert statistical consultation. In addition, several limitations apply universally to normality tests across all statistical methods. These tests provide evidence against normality rather than definitive proof of a normal distribution, and their performance is influenced by sample size. Hence, p-values should not be interpreted in isolation and results should be interpreted alongside visual methods such as histograms and Q–Q plots. Conclusion This article highlights a basic understanding of data distribution and its analysis methods. Researchers can use a free R-based Shiny application – NormSight to test normality. This open-source tool can help students, clinicians, and researchers make correct analytical decisions with ease. Author contributions We confirm that all authors have read and approved the manuscript for submission. All authors meet authorship criteria, and the manuscript represents honest work. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest. Use of artificial intelligence (AI) We have used ChatGPT to correct language and grammar. We have referred to the publisher’s policy of use of AI as depicted in information for authors and assume full responsibility for the content of our manuscript, even those parts produced by an AI tool, and are thus liable for any breach of publication ethics.