Friedman Test (Nonparametric Repeated Measures ANOVA)

Compare data where every subject has a value for each condition, such as the same workers using several tools or the same judges rating several prototypes. The condition difference is tested with ranks within each subject, and a post hoc test tells you which conditions differ.

1Data Table

Paste a table copied from Excel into a cell (Ctrl+V). Click a column header (A, B…) before pasting to use the first line as column names. Double-click a header to rename it, right-click to insert/delete rows and columns, and formulas such as =A1*2 work too.

2Friedman Test Options

Data input

How to run a Friedman test

  1. Copy three columns - subject ID, condition and measurement (long format) - or a table with one row per subject and one column per condition (wide format), together with the header.
  2. Choose the data input and select the columns. In wide format, check 3 or more condition columns. If needed, choose a post hoc method (Nemenyi or Conover).
  3. Press Analyze and check the p-value in the test summary and the mean rank of each condition.

Worked example with sample data

The sample data are the times (s) 10 workers took to machine the same part with tools A, B and C (long format: worker, tool, machining time). The workers differ in skill, so the machining times themselves vary a lot.

Machining time by tool for 10 workers - each line is one worker
Machining time by tool for 10 workers - each line is one worker

The test summary shows a Friedman statistic of 16.2 (2 degrees of freedom) and a p-value of 0.0003035, so the machining time differs significantly by tool. The mean ranks are 1.1 for Tool B, 2 for Tool C and 2.9 for Tool A - almost every worker was fastest with Tool B and slowest with Tool A.

In the individual lines chart, the heights of the lines differ widely between workers (45 to 66 s), but most share the same V shape, dropping from A to B and rising a little at C. Because the Friedman test uses only the ranks within each worker, it looks at the tool effect without being thrown off by these individual differences.

Choosing Nemenyi as the post hoc method shows only Tool A-Tool B (p=0.0001685) as significant, while A-C and B-C have p=0.1092 and are not. Tool B is clearly faster than A, and C lies in between, so more data are needed to judge it.

How it is calculated

The conditions are ranked within each subject (row), the Friedman statistic is computed from the rank sums of the conditions, and a tie correction is applied. It is tested against the chi-square distribution with (number of conditions - 1) degrees of freedom (the same as SciPy's friedmanchisquare).

Only subjects with a value for every condition are used.

Post hoc tests

Nemenyi tests the differences in mean ranks with the studentized range distribution, and Conover with the t distribution. For Conover you can choose a Bonferroni, Holm or Benjamini-Hochberg (FDR) correction (the same as scikit-posthocs).

What the result shows

Descriptive statistics by condition (median, mean rank), the test summary, the post hoc table, individual lines, a box plot by condition and the mean ranks with confidence intervals.

How to read the Friedman test result

  • If the p-value is below the significance level, the rank distribution of at least one condition differs. A condition with a small mean rank had small values within subjects.
  • If most individual lines rise and fall in the same pattern, the condition effect is consistent across subjects.
  • To see which pairs of conditions differ, choose Nemenyi or Conover as the post hoc method.

Frequently asked questions

How is it different from repeated measures ANOVA?

Repeated measures ANOVA compares means and assumes normality and sphericity; the Friedman test uses only the ranks within subjects and needs no such assumptions.

What about subjects with missing conditions?

Only subjects with a value for every condition are analyzed, and at least 3 such subjects are needed.

How are subjects named in wide format?

In wide format, subjects are named S1, S2, ... in row order.

Other chart tools