Printer-friendly version MS Word (.doc)

Fisher’s Exact Test

 

·      Does not rely on Normality assumption.

 

·      Uses “exact” distribution instead of a Normal approximation.  

 

·      Use in place of χ2 test when any expected cell frequency is less than 5.

 

Example: caries incidence

 

Caries

 

 

yes

no

total

control

10

26

36

intervention

6

62

68

total

16

88

104

 

The null-hypothesis of Fisher’s Exact test is that there is no relationship between the two categories. Thus, every possible arrangement of observations in the respective cells is equally likely (we assume the row and column totals are fixed).

 

The p-value is computed by calculating the number of possible arrangements of observations that produce tables that are more extreme than the observed and then dividing this by the total number of possible arrangements of the observations.


Example: Caries incidence

 

observed table

Caries

 

yes

no

total

p̂c- p̂i=

0.190

control

10

26

36

intervention

6

62

68

probability under H0

total

16

88

104

pH0=

0.01048

 

Tables more extreme (result in greater difference in proportions)

11

25

36

p̂c- p̂i=

0.23

15

21

36

p̂c- p̂i=

0.40

5

63

68

1

67

68

16

88

104

pH0=

0.00236

16

88

104

pH0=

0.00000

12

24

36

p̂c- p̂i=

0.27

16

20

36

p̂c- p̂i=

0.44

4

64

68

0

68

68

16

88

104

pH0=

0.00038

16

88

104

pH0=

0.00000

13

23

36

p̂c- p̂i=

0.32

0

36

36

p̂c- p̂i=

-0.24

3

65

68

16

52

68

16

88

104

pH0=

0.00004

16

88

104

pH0=

0.00055

14

22

36

p̂c- p̂i=

0.36

1

35

36

p̂c- p̂i=

-0.19

2

66

68

15

53

68

16

88

104

pH0=

0.00000

16

88

104

pH0=

0.00602

Total probability of all as or more extreme tables =

0.01985


Formula for Probability of Table in Fisher’s Exact Test

 

 

Table

 

 

a

b

 

 

c

d

 

 

 

 

 

Probability

SPSS output

 

 


McNemar’s Test for Proportions (Paired Data)

 

Use for comparing proportions from paired data

 

Example: Change in plaque index

 

Fifty-three study participants assessed twice for plaque index (PI), at baseline and 4 weeks later.  We wish to assess whether the proportion of patients with high PI changes.

 

 

PI at 4 weeks

PI at baseline

low

high

low

29

1

high

13

10

 

Incorrect methods:

1.   Comparing  with   using the Z-test:    This will not give a valid p-value because it does not compare independent samples.  The same 53 people are used in each proportion.

 

2.   Performing a Chi-square or Fisher’s Exact test on the above 2×2 table:  These would test whether the proportion of high’s at baseline is related to the proportion of high’s at 4 weeks.  They would not test whether or not the proportions are different. 

 


 

 

PI at 4 weeks

PI at baseline

low

high

low

29

1

high

13

10

 

McNemar’s Test assesses the null hypothesis

 

H0:  P(PI high at baseline) = P(PI high at 4 week),

 

by noting that it is equivalent to:

 

H0:

P(PI changes high to low) = P(PI changes low to high),

 

for all discordant pairs.

 

The discordant pairs are those that have different values for the two observations.  Note that each entry in the table is the number of pairs.

 

The latter H0 can be evaluated using a one-sample test for proportions with,

 

H0: p = 0.50, vs. H1: p ≠ 0.50,

 

where p = proportion of discordant pairs that increase.


 

 

PI at 4 weeks

PI at baseline

low

high

low

29

1

high

13

10

 

 

·      If n > 20   (where n is # of discordant pairs) can use Z-test for proportions (chapter 9.3).  

 

·      If n < 20 (as in the current example, n = 14) use the binomial distribution to compute the exact p-value.

 

Let X = number of discordant pairs that increase, which, under H0, is binomial(n = 14, p = 0.5). 

 

The two-sided p-value is the probability that we would see a more unbalanced sample of the discordant pairs than 13 vs 1, which is

 

P(X < 1) + P(X > 13)

= P(X=0) + P(X=1) + P(X=13) + P(X=14)

= 0.0001 + 0.0009 + 0.0009 + 0.0001

= 0.0020
SPSS output

 


Analysis of Categorical Data Summary

 

Ø Proportions from two independent samples

Þ                       Large samples – Z-test for proportions

Þ                       Small samples – Fisher’s Exact Test

 

Ø Proportions from > 2 independent samples

Þ                       Chi-square test

 

Ø Proportions from paired data

Þ                       McNemar’s Test

next lecture | course page