How to run a McNemar test
- Copy the before and after rating columns so that the same subject is on the same row (two kinds of values, e.g. Pass/Fail).
- Select the before rating column and the after rating column, and choose the test method (exact test by default).
- Press Analyze and check the p-value and the counts by direction of change.
Worked example with sample data
The sample data are the before and after training practical test results (Pass, Fail) of 30 workers. Select the before rating column and the after rating column.
The paired table shows 10 workers who went from Fail before training to Pass, and 1 worker who went from Pass to Fail. The exact test p-value of 0.01172 shows that the results changed significantly toward Pass.
The McNemar test uses only the 11 pairs that changed. The 19 workers whose result stayed the same (4 stayed Fail, 15 stayed Pass) are not part of the test. With the chi-square approximation (continuity correction) the statistic is 5.818 and p=0.01586, the same conclusion.
The before/after proportion bar shows the Fail rate dropping from 46.67% to 16.67%, and in the state transition diagram the thick red Fail→Pass line shows this change.
How it is calculated
Only the pairs whose state changed are used (b: state 1→state 2, c: state 2→state 1). The exact test is the two-sided binomial probability of min(b, c) with n = b + c and p = 0.5, and the chi-square approximation tests (|b - c| - 1)² / (b + c) with 1 degree of freedom (without continuity correction, 1 is not subtracted).
The state names are sorted by name and used as state 1 and state 2 (the same as statsmodels mcnemar).
What the result shows
The paired 2×2 table, the counts by direction of change, the test summary, a before/after proportion bar and a state transition diagram.
How to read the McNemar test result
- If the p-value is below the significance level, the state changed more in one direction, and the interpretation tells you which.
- In the state transition diagram, red lines are pairs whose state changed and blue lines are unchanged pairs; thicker lines mean more pairs.
- Unchanged pairs are not used in the test, so also look at the overall change in the before/after proportion bar.
Frequently asked questions
How is it different from the 2-proportion test?
The 2-proportion test compares two different groups; the McNemar test compares paired data where the same subjects were rated twice.
What if there are three kinds of values?
The McNemar test accepts exactly two states. With three or more values, an error message appears.
Should I use the exact test or the chi-square approximation?
The default exact test is accurate even for small samples. With many changed pairs, the two methods give similar results.