17  Mc Nemar’s Test

McNemar test

The McNemar test is a statistical test used to analyze paired nominal data or matched data collected from two related groups or conditions. It assesses whether there is a significant difference in proportions or frequencies of a categorical outcome between paired observations. The McNemar test is particularly useful when dealing with data that are not independent, such as before-and-after measurements on the same subjects or matched pairs in case-control studies.

17.1 Understanding the McNemar Test:

1. Null and Alternative Hypotheses:

  • Null Hypothesis (H0): There is no difference in the proportions or frequencies of the categorical outcome between the two conditions.
  • Alternative Hypothesis (H1): There is a significant difference in the proportions or frequencies of the categorical outcome between the two conditions.

2. Test Statistic:

  • The McNemar test statistic is calculated based on the number of discordant pairs, i.e., pairs where the outcomes differ between the two conditions.
  • It follows a chi-squared distribution with one degree of freedom under the null hypothesis.

3. Calculation of Test Statistic:

  • Let a be the number of discordant pairs where a subject has a positive outcome in the first condition and a negative outcome in the second condition.
  • Let b be the number of discordant pairs in the opposite direction.
  • The uncorrected McNemar test statistic is:

\[\chi^2_{M} = \frac{(b - a)^2}{b + a}\]

  • The corrected McNemar test statistic (Yates’ continuity correction to adjust for the discrete nature of the data)is:

\[\chi^2_{M, corrected} = \frac{(|b - a| - 1)^2}{b + a}\]

4. Interpretation of Results:

  • If the calculated \(\chi^2_M\) value is greater than the critical value from the chi-squared distribution with 1 degree of freedom at the chosen significance level (commonly \(\alpha = 0.05\)), we reject the null hypothesis, indicating a significant difference.
  • The continuity-corrected value is slightly lower than the uncorrected value, but in most practical cases, both lead to the same conclusion.

17.1.1 McNemar Test: Corrected and Uncorrected

The McNemar test is designed to analyze paired nominal data, particularly when the outcomes are dichotomous (e.g., pass/fail). There are two approaches to calculate the test statistic: uncorrected and continuity-corrected.

Uncorrected McNemar Test

  • Calculates the chi-squared statistic directly from the discordant pairs.
  • Formula:

\[ \chi^2_{uncorrected} = \frac{(b - a)^2}{b + a} \]

  • a = number of pairs where the first condition is positive and the second is negative.
  • b = number of pairs where the first condition is negative and the second is positive.
  • The uncorrected test shows the theoretical difference but may slightly overestimate significance with small samples.

Continuity-Corrected McNemar Test

  • Also known as Yates’ correction, adjusts for small sample discrete data.
  • Formula:

\[ \chi^2_{corrected} = \frac{(|b - a| - 1)^2}{b + a} \]

  • This correction reduces the test statistic slightly, making it more conservative.
  • Recommended for small discordant pairs.

Summary: The uncorrected McNemar test is straightforward, while the continuity-corrected version provides a more conservative estimate. Both are valid, but the corrected test is often preferred for small sample sizes or when using standard software implementations.

17.1.2 Considerations:

  • The McNemar test assumes that the data are paired and dichotomous.
  • It is sensitive to small sample sizes, especially when the number of discordant pairs is small.
  • Extensions exist for analyzing categorical data with more than two categories.

Applications of McNemar Test

A. Medical Research:

  • In clinical trials, the McNemar test is used to assess whether there is a significant difference in treatment outcomes between two treatment groups or before and after treatment within the same group.

B. Education:

  • Educational researchers may use the McNemar test to evaluate the effectiveness of teaching methods or interventions by comparing pre-test and post-test scores of students.

C. Epidemiology:

  • Epidemiologists use the McNemar test to analyze matched case-control studies or cohort studies where data are collected from the same subjects at different time points.

D. Psychology and Social Sciences:

  • Researchers in psychology and social sciences utilize the McNemar test to analyze paired survey responses, preferences, or behaviors before and after exposure to certain stimuli or interventions.

17.2 Example problem: McNemar test.

Suppose we’re conducting a study to evaluate the effectiveness of a new teaching method for improving students’ performance in a mathematics exam. We collect data from 50 students who were administered a pre-test (before the teaching method was introduced) and a post-test (after the teaching method was introduced). Each student’s performance is categorized as “pass” or “fail” in both the pre-test and post-test.

Here’s a summary of the data:

  • In the pre-test, 25 students passed and 25 students failed.
  • In the post-test, 35 students passed and 15 students failed.
  • Among the students who passed the pre-test, 20 also passed the post-test.
  • Among the students who failed the pre-test, 15 passed the post-test.

We want to determine if there is a significant difference in the proportions of students passing the exam before and after the teaching method was introduced.

17.2.1 Calculation of McNemar Test (Continuity-Corrected)

  1. Create a Contingency Table:
Passed Post-test (Yes) Passed Post-test (No)
Passed Pre-test (Yes) 20 5
Passed Pre-test (No) 15 10

  1. Calculate the McNemar Test Statistic (Continuity-Corrected):

    • \(a\) = Number of discordant pairs where a student passed the pre-test but failed the post-test = 5
    • \(b\) = Number of discordant pairs where a student failed the pre-test but passed the post-test = 15

    \[\chi^2_{M, corrected} = \frac{(|b - a| - 1)^2}{b + a}\] \[\chi^2_{M, corrected} = \frac{(|15 - 5| - 1)^2}{15 + 5} = \frac{(10 - 1)^2}{20} = \frac{81}{20} = 4.05\]

  2. Determine the Critical Value:

    • With 1 degree of freedom and \(\alpha = 0.05\), the critical value of the chi-squared distribution is approximately 3.84.

  3. Compare the Test Statistic to the Critical Value:

    • Since \(\chi^2_{M, corrected} = 4.05 > 3.84\), we reject the null hypothesis.

Interpretation:

Based on the McNemar test, we conclude that there is a significant difference in the proportions of students passing the exam before and after the teaching method was introduced. In this example, the teaching method appears to have a significant positive effect on students’ performance in the mathematics exam.

McNemar Test in R and Python