How to run Fisher's exact test
- Copy two categorical columns (one observation per row), or pre-counted category and frequency columns, together with the header.
- Select the row category column (e.g. process) and the column category column (e.g. result). Both must have exactly 2 categories.
- Press Analyze and check the exact p-value and the confidence intervals of the odds ratio and relative risk.
Worked example with sample data
The sample data are the results (Defect, Good) of 20 products each from the current and new processes, one product per row. The Process column is the row category and the Result column the column category.
The 2×2 table shows 7 defects (35%) for the current process and 1 defect (5%) for the new process. The exact p-value of 0.0436 is below the significance level of 0.05, so the defect proportions of the two processes differ significantly. 'Defect' is the first column by name, so it is the event.
The odds ratio is 10.23 (95% confidence interval 1.121 to 93.34), which does not contain 1. The relative risk of 7 means the current process has 7 times the defect proportion of the new one, but its confidence interval (0.9459 to 51.8) contains 1, so its conclusion is weaker than the odds ratio's.
The risk difference is 0.3 (confidence interval 0.07017 to 0.5298), a 30 percentage point drop in the defect rate. With only 1 defect in the new process, Fisher's exact test fits better than the chi-square approximation, and the wide proportion intervals in the chart also come from the small sample.
How it is calculated
With the row and column totals fixed, the probability of tables at least as extreme as the observed one is summed exactly from the hypergeometric distribution (the same as SciPy's fisher_exact).
The odds ratio, relative risk and their confidence intervals are calculated like statsmodels Table2x2; a cell with 0 is replaced by 0.5.
Proportions and risk difference
The first column category (by name) is treated as the 'event', and the proportion is calculated for each row. Risk difference = row 1 proportion - row 2 proportion, with a normal (Wald) confidence interval.
What the result shows
The 2×2 table, the test summary, the proportions with confidence intervals by group and an odds ratio forest plot.
How to read the Fisher's exact test result
- If the exact p-value is below the significance level, the event proportions of the two rows differ significantly.
- If the odds ratio confidence interval does not contain 1, the conclusion is the same as a significant difference; above 1 means the first row has higher odds of the event.
- Which category is the 'event' depends on the name order, so check the first column category.
Frequently asked questions
When should I use it instead of the chi-square test?
Use Fisher's exact test when a 2×2 table has a cell with an expected count below 5. It also works with large samples.
Can I use a 2×3 table?
This tool accepts only 2×2 tables. For larger tables, use the chi-square test of association.
What does the one-sided test compare?
'Group 1 > group 2' tests whether the odds ratio is greater than 1, i.e. whether the first row has higher odds of the event.