Printer-friendly version MS Word (.doc)

Fisher’s Exact Test

· Does not rely on Normality assumption.

· Uses “exact” distribution instead of a Normal approximation.

· Use in place of χ² test when any expected cell frequency is less than 5.

Example: caries incidence

	Caries
	yes	no	total
control	10	26	36
intervention	6	62	68
total	16	88	104

The null-hypothesis of Fisher’s Exact test is that there is no relationship between the two categories. Thus, every possible arrangement of observations in the respective cells is equally likely (we assume the row and column totals are fixed).

The p-value is computed by calculating the number of possible arrangements of observations that produce tables that are more extreme than the observed and then dividing this by the total number of possible arrangements of the observations.

Example: Caries incidence

observed table	Caries
	yes	no	total	p̂c- p̂i=	0.190
control	10	26	36
intervention	6	62	68	probability under H0
total	16	88	104	pH0=	0.01048

Tables more extreme (result in greater difference in proportions)
11	25	36	p̂c- p̂i=	0.23	15	21	36	p̂c- p̂i=	0.40
5	63	68			1	67	68
16	88	104	pH0=	0.00236	16	88	104	pH0=	0.00000
12	24	36	p̂c- p̂i=	0.27	16	20	36	p̂c- p̂i=	0.44
4	64	68			0	68	68
16	88	104	pH0=	0.00038	16	88	104	pH0=	0.00000
13	23	36	p̂c- p̂i=	0.32	0	36	36	p̂c- p̂i=	-0.24
3	65	68			16	52	68
16	88	104	pH0=	0.00004	16	88	104	pH0=	0.00055
14	22	36	p̂c- p̂i=	0.36	1	35	36	p̂c- p̂i=	-0.19
2	66	68			15	53	68
16	88	104	pH0=	0.00000	16	88	104	pH0=	0.00602
Total probability of all as or more extreme tables =									0.01985

Formula for Probability of Table in Fisher’s Exact Test

Table
a	b
c	d

Probability

SPSS output

McNemar’s Test for Proportions (Paired Data)

Use for comparing proportions from paired data

Example: Change in plaque index

Fifty-three study participants assessed twice for plaque index (PI), at baseline and 4 weeks later. We wish to assess whether the proportion of patients with high PI changes.

	PI at 4 weeks
PI at baseline	low	high
low	29	1
high	13	10

Incorrect methods:

1. Comparing with using the Z-test: This will not give a valid p-value because it does not compare independent samples. The same 53 people are used in each proportion.

2. Performing a Chi-square or Fisher’s Exact test on the above 2×2 table: These would test whether the proportion of high’s at baseline is related to the proportion of high’s at 4 weeks. They would not test whether or not the proportions are different.

	PI at 4 weeks
PI at baseline	low	high
low	29	1
high	13	10

McNemar’s Test assesses the null hypothesis

H₀: P(PI high at baseline) = P(PI high at 4 week),

by noting that it is equivalent to:

H0:	P(PI changes high to low) = P(PI changes low to high),
	for all discordant pairs.

The discordant pairs are those that have different values for the two observations. Note that each entry in the table is the number of pairs.

The latter H₀ can be evaluated using a one-sample test for proportions with,

H₀: p = 0.50, vs. H₁: p ≠ 0.50,

where p = proportion of discordant pairs that increase.

	PI at 4 weeks
PI at baseline	low	high
low	29	1
high	13	10

· If n > 20 (where n is # of discordant pairs) can use Z-test for proportions (chapter 9.3).

· If n < 20 (as in the current example, n = 14) use the binomial distribution to compute the exact p-value.

Let X = number of discordant pairs that increase, which, under H₀, is binomial(n = 14, p = 0.5).

The two-sided p-value is the probability that we would see a more unbalanced sample of the discordant pairs than 13 vs 1, which is

P(X < 1) + P(X > 13)

= P(X=0) + P(X=1) + P(X=13) + P(X=14)

= 0.0001 + 0.0009 + 0.0009 + 0.0001

= 0.0020
SPSS output

Analysis of Categorical Data Summary

Ø Proportions from two independent samples

Þ Large samples – Z-test for proportions

Þ Small samples – Fisher’s Exact Test

Ø Proportions from > 2 independent samples

Þ Chi-square test

Ø Proportions from paired data

Þ McNemar’s Test

next lecture | course page