Fisher's Exact Test

Use it for a 2×2 table, such as defect status for the current and new process, when the counts are too small for the chi-square approximation. Along with the exact p-value, the odds ratio, relative risk and risk difference show the size of the difference.

1Data Table

Paste a table copied from Excel into a cell (Ctrl+V). Click a column header (A, B…) before pasting to use the first line as column names. Double-click a header to rename it, right-click to insert/delete rows and columns, and formulas such as =A1*2 work too.

2Fisher's Exact Test Options

Data input

How to run Fisher's exact test

  1. Copy two categorical columns (one observation per row), or pre-counted category and frequency columns, together with the header.
  2. Select the row category column (e.g. process) and the column category column (e.g. result). Both must have exactly 2 categories.
  3. Press Analyze and check the exact p-value and the confidence intervals of the odds ratio and relative risk.

Worked example with sample data

The sample data are the results (Defect, Good) of 20 products each from the current and new processes, one product per row. The Process column is the row category and the Result column the column category.

Defect proportions of the current and new processes with confidence intervals
Defect proportions of the current and new processes with confidence intervals

The 2×2 table shows 7 defects (35%) for the current process and 1 defect (5%) for the new process. The exact p-value of 0.0436 is below the significance level of 0.05, so the defect proportions of the two processes differ significantly. 'Defect' is the first column by name, so it is the event.

The odds ratio is 10.23 (95% confidence interval 1.121 to 93.34), which does not contain 1. The relative risk of 7 means the current process has 7 times the defect proportion of the new one, but its confidence interval (0.9459 to 51.8) contains 1, so its conclusion is weaker than the odds ratio's.

The risk difference is 0.3 (confidence interval 0.07017 to 0.5298), a 30 percentage point drop in the defect rate. With only 1 defect in the new process, Fisher's exact test fits better than the chi-square approximation, and the wide proportion intervals in the chart also come from the small sample.

How it is calculated

With the row and column totals fixed, the probability of tables at least as extreme as the observed one is summed exactly from the hypergeometric distribution (the same as SciPy's fisher_exact).

The odds ratio, relative risk and their confidence intervals are calculated like statsmodels Table2x2; a cell with 0 is replaced by 0.5.

Proportions and risk difference

The first column category (by name) is treated as the 'event', and the proportion is calculated for each row. Risk difference = row 1 proportion - row 2 proportion, with a normal (Wald) confidence interval.

What the result shows

The 2×2 table, the test summary, the proportions with confidence intervals by group and an odds ratio forest plot.

How to read the Fisher's exact test result

  • If the exact p-value is below the significance level, the event proportions of the two rows differ significantly.
  • If the odds ratio confidence interval does not contain 1, the conclusion is the same as a significant difference; above 1 means the first row has higher odds of the event.
  • Which category is the 'event' depends on the name order, so check the first column category.

Frequently asked questions

When should I use it instead of the chi-square test?

Use Fisher's exact test when a 2×2 table has a cell with an expected count below 5. It also works with large samples.

Can I use a 2×3 table?

This tool accepts only 2×2 tables. For larger tables, use the chi-square test of association.

What does the one-sided test compare?

'Group 1 > group 2' tests whether the odds ratio is greater than 1, i.e. whether the first row has higher odds of the event.

Other chart tools