Printer-friendly version MS Word (.doc)

Fisher’s Exact Test

·      Does not rely on Normality assumption.

·      Uses “exact” distribution instead of a Normal approximation.

·      Use in place of χ2 test when any expected cell frequency is less than 5.

Example: caries incidence

 Caries yes no total control 10 26 36 intervention 6 62 68 total 16 88 104

The null-hypothesis of Fisher’s Exact test is that there is no relationship between the two categories. Thus, every possible arrangement of observations in the respective cells is equally likely (we assume the row and column totals are fixed).

The p-value is computed by calculating the number of possible arrangements of observations that produce tables that are more extreme than the observed and then dividing this by the total number of possible arrangements of the observations.

Example: Caries incidence

 observed table Caries yes no total p̂c- p̂i= 0.190 control 10 26 36 intervention 6 62 68 probability under H0 total 16 88 104 pH0= 0.01048

 Tables more extreme (result in greater difference in proportions) 11 25 36 p̂c- p̂i= 0.23 15 21 36 p̂c- p̂i= 0.40 5 63 68 1 67 68 16 88 104 pH0= 0.00236 16 88 104 pH0= 0.00000 12 24 36 p̂c- p̂i= 0.27 16 20 36 p̂c- p̂i= 0.44 4 64 68 0 68 68 16 88 104 pH0= 0.00038 16 88 104 pH0= 0.00000 13 23 36 p̂c- p̂i= 0.32 0 36 36 p̂c- p̂i= -0.24 3 65 68 16 52 68 16 88 104 pH0= 0.00004 16 88 104 pH0= 0.00055 14 22 36 p̂c- p̂i= 0.36 1 35 36 p̂c- p̂i= -0.19 2 66 68 15 53 68 16 88 104 pH0= 0.00000 16 88 104 pH0= 0.00602 Total probability of all as or more extreme tables = 0.01985

Formula for Probability of Table in Fisher’s Exact Test

 Table a b c d

Probability

SPSS output

McNemar’s Test for Proportions (Paired Data)

Use for comparing proportions from paired data

Example: Change in plaque index

Fifty-three study participants assessed twice for plaque index (PI), at baseline and 4 weeks later.  We wish to assess whether the proportion of patients with high PI changes.

 PI at 4 weeks PI at baseline low high low 29 1 high 13 10

Incorrect methods:

1.   Comparing  with   using the Z-test:    This will not give a valid p-value because it does not compare independent samples.  The same 53 people are used in each proportion.

2.   Performing a Chi-square or Fisher’s Exact test on the above 2×2 table:  These would test whether the proportion of high’s at baseline is related to the proportion of high’s at 4 weeks.  They would not test whether or not the proportions are different.

 PI at 4 weeks PI at baseline low high low 29 1 high 13 10

McNemar’s Test assesses the null hypothesis

H0:  P(PI high at baseline) = P(PI high at 4 week),

by noting that it is equivalent to:

 H0: P(PI changes high to low) = P(PI changes low to high), for all discordant pairs.

The discordant pairs are those that have different values for the two observations.  Note that each entry in the table is the number of pairs.

The latter H0 can be evaluated using a one-sample test for proportions with,

H0: p = 0.50, vs. H1: p≠ 0.50,

where p = proportion of discordant pairs that increase.

 PI at 4 weeks PI at baseline low high low 29 1 high 13 10

·      If n > 20   (where n is # of discordant pairs) can use Z-test for proportions (chapter 9.3).

·      If n < 20 (as in the current example, n = 14) use the binomial distribution to compute the exact p-value.

Let X = number of discordant pairs that increase, which, under H0, is binomial(n = 14, p = 0.5).

The two-sided p-value is the probability that we would see a more unbalanced sample of the discordant pairs than 13 vs 1, which is

P(X < 1) + P(X > 13)

= P(X=0) + P(X=1) + P(X=13) + P(X=14)

= 0.0001 + 0.0009 + 0.0009 + 0.0001

= 0.0020
SPSS output

Analysis of Categorical Data Summary

Ø Proportions from two independent samples

Þ                       Large samples – Z-test for proportions

Þ                       Small samples – Fisher’s Exact Test

Ø Proportions from > 2 independent samples

Þ                       Chi-square test

Ø Proportions from paired data

Þ                       McNemar’s Test