Kruskal–Wallis test

1. Rank all data from all groups together; i.e., rank the data from 1 to N ignoring group membership. Assign any tied values the average of the ranks they would have received had they not been tied.
2. The test statistic is given by

   ${\displaystyle \definecolor {ChromeYellow}{rgb}{1,0.6549019607843137,0.011764705882352941}\definecolor {Green}{rgb}{0,0.5019607843137255,0}\definecolor {green}{rgb}{0,0.5019607843137255,0}\definecolor {Blue}{rgb}{0,0,1}\definecolor {Purple}{rgb}{0.5019607843137255,0,0.5019607843137255}H=({\color {Red}N}-1){\frac {\sum _{i=1}^{\color {Orange}g}{\color {ChromeYellow}n_{i}}({\color {Blue}{\bar {r}}_{i\cdot }}-{\color {Purple}{\bar {r}}})^{2}}{\sum _{i=1}^{\color {Orange}g}\sum _{j=1}^{\color {ChromeYellow}n_{i}}({\color {Green}r_{ij}}-{\color {Purple}{\bar {r}}})^{2}}},}$ where

   • ${\textstyle \color {Red}N}$ is the total number of observations across all groups
   • ${\textstyle \color {Orange}g}$ is the number of groups
   • ${\textstyle \definecolor {ChromeYellow}{rgb}{1,0.6549019607843137,0.011764705882352941}\color {ChromeYellow}n_{i}}$ is the number of observations in group ${\displaystyle i}$
   • ${\displaystyle \definecolor {Green}{rgb}{0,0.5019607843137255,0}\definecolor {green}{rgb}{0,0.5019607843137255,0}\color {Green}r_{ij}}$ is the rank (among all observations) of observation ${\displaystyle j}$ from group ${\displaystyle i}$
   • ${\displaystyle \definecolor {blue}{rgb}{0,0,1}{\color {blue}{\bar {r}}_{i\cdot }}={\frac {\sum _{j=1}^{n_{i}}{r_{ij}}}{n_{i}}}}$ is the average rank of all observations in group ${\displaystyle i}$
   • ${\textstyle \definecolor {Purple}{rgb}{0.5019607843137255,0,0.5019607843137255}{\color {Purple}{\bar {r}}}={\tfrac {1}{2}}(N+1)}$ is the average of all the ${\textstyle \definecolor {Green}{rgb}{0,0.5019607843137255,0}\definecolor {green}{rgb}{0,0.5019607843137255,0}\color {Green}r_{ij}}$ .
3. If the data contain no ties the denominator of the expression for ${\displaystyle H}$ is exactly ${\displaystyle (N-1)N(N+1)/12}$ and ${\displaystyle {\bar {r}}={\tfrac {N+1}{2}}}$ . Thus

   ${\displaystyle {\begin{aligned}H&={\frac {12}{N(N+1)}}\sum _{i=1}^{g}n_{i}\left({\bar {r}}_{i\cdot }-{\frac {N+1}{2}}\right)^{2}\\&={\frac {12}{N(N+1)}}\sum _{i=1}^{g}n_{i}{\bar {r}}_{i\cdot }^{2}-\ 3(N+1)\end{aligned}}}$
   The last formula only contains the squares of the average ranks.

4. A correction for ties if using the short-cut formula described in the previous point can be made by dividing ${\displaystyle H}$ by ${\displaystyle 1-{\frac {\sum _{i=1}^{G}(t_{i}^{3}-t_{i})}{N^{3}-N}}}$ , where G is the number of groupings of different tied ranks, and t_i is the number of tied values within group i that are tied at a particular value. This correction usually makes little difference in the value of H unless there are a large number of ties.
5. When performing multiple sample comparisons, the type I error tends to become inflated. Therefore, Bonferroni procedure is used to adjust the significance level, that is, ${\displaystyle {\bar {a}}={\frac {\alpha }{\Bbbk }}}$ , where ${\displaystyle {\bar {a}}}$ is the adjusted significance level, ${\displaystyle \alpha }$ is the initial significance level, and ${\displaystyle \Bbbk }$ is the number of the contrasts.^[13]
6. Finally, the decision to reject or not the null hypothesis is made by comparing ${\displaystyle H}$ to a critical value ${\displaystyle H_{c}}$ obtained from a table or a software for a given significance or alpha level. If ${\displaystyle H}$ is bigger than ${\displaystyle H_{c}}$ , the null hypothesis is rejected. If possible (no ties, sample not too big) one should compare ${\displaystyle H}$ to the critical value obtained from the exact distribution of ${\displaystyle H}$ . Otherwise, the distribution of H can be approximated by a chi-squared distribution with g-1 degrees of freedom. If some ${\displaystyle n_{i}}$ values are small (i.e., less than 5) the exact probability distribution of ${\displaystyle H}$ can be quite different from this chi-squared distribution. If a table of the chi-squared probability distribution is available, the critical value of chi-squared, ${\displaystyle \chi _{\alpha :g-1}^{2}}$ , can be found by entering the table at g − 1 degrees of freedom and looking under the desired significance or alpha level.^[14]
7. If the statistic is not significant, then there is no evidence of stochastic dominance between the samples. However, if the test is significant then at least one sample stochastically dominates another sample. Therefore, a researcher might use sample contrasts between individual sample pairs, or post hoc tests using Dunn's test, which (1) properly employs the same rankings as the Kruskal–Wallis test, and (2) properly employs the pooled variance implied by the null hypothesis of the Kruskal–Wallis test in order to determine which of the sample pairs are significantly different.^[5] When performing multiple sample contrasts or tests, the Type I error rate tends to become inflated, raising concerns about multiple comparisons.

A large amount of computing resources is required to compute exact probabilities for the Kruskal–Wallis test. Existing software only provides exact probabilities for sample sizes of less than about 30 participants. These software programs rely on the asymptotic approximation for larger sample sizes.

Exact probability values for larger sample sizes are available. Spurrier (2003) published exact probability tables for samples as large as 45 participants.^[15] Meyer and Seaman (2006) produced exact probability distributions for samples as large as 105 participants.^[16]

Choi et al.^[17]made a review of two methods that had been developed to compute the exact distribution of ${\displaystyle H}$ , proposed a new one, and compared the exact distribution to its chi-squared approximation.

Test for differences in ozone levels by month

The following example uses data from Chambers et al.^[18]on daily readings of ozone for May 1 to September 30, 1973, in New York City. The data are in the R data set airquality, and the analysis is included in the documentation for the R function kruskal.test. Boxplots of ozone values by month are shown in the figure.

The Kruskal-Wallis test finds a significant difference (p = 6.901e-06) indicating that ozone differs among the 5 months.

kruskal.test(Ozone ~ Month, data = airquality)Kruskal-Wallis rank sum testdata:  Ozone by MonthKruskal-Wallis chi-squared = 29.267, df = 4, p-value = 6.901e-06

To determine which months differ, post-hoc tests may be performed using a Wilcoxon test for each pair of months, with a Bonferroni (or other) correction for multiple hypothesis testing.

pairwise.wilcox.test(airquality$Ozone, airquality$Month, p.adjust.method = "bonferroni")Pairwise comparisons using Wilcoxon rank sum testdata:  airquality$Ozone and airquality$Month  5      6      7      8     6 1.0000 -      -      -     7 0.0003 0.1414 -      -     8 0.0012 0.2591 1.0000 -     9 1.0000 1.0000 0.0074 0.0325P value adjustment method: bonferroni

The post-hoc tests indicate that, after Bonferroni correction for multiple testing, the following differences are significant (adjusted p < 0.05).

• Month 5 vs Months 7 and 8
• Month 9 vs Months 7 and 8

The Kruskal-Wallis test can be implemented in many programming tools and languages.

• Mathematica implements the test as LocationEquivalenceTest.^[19]
• MATLAB's Statistics Toolbox has kruskalwallis to compute the p-value for a hypothesis test and display ANOVA table.^[20]
• SAS has the "NPAR1WAY" procedure for the test.^[21]
• SPSS implements the test with the "Nonparametric Tests" procedure.^[22]
• Minitab has the implement in the "Nonparametrics" option.^[23]
• In Python's SciPy package, the function scipy.stats.kruskal can return the test result and p-value.^[24]
• R base-package has an implement of this test using kruskal.test.^[25]
• Java has the implement provided by provided by Apache Commons.^[26]
• In Julia, the package HypothesisTests.jl has the function KruskalWallisTest(groups::AbstractVector{<:Real}...) to compute the p-value.^[27]

• One-way ANOVA
• Mann–Whitney U tests
• Bonferroni test
• Friedman test
• Jonckheere's trend test
• Mood's Median test

• Daniel, Wayne W. (1990). "Kruskal–Wallis one-way analysis of variance by ranks". Applied Nonparametric Statistics (2nd ed.). Boston: PWS-Kent. pp. 226–234. ISBN 0-534-91976-6.

• An online version of the test