Overview
ANOVA (Analysis of Variance) is a statistical method used to test differences between two or more group means. When an ANOVA indicates significant differences, it does not tell which specific groups differ. Post hoc tests are used to conduct pairwise comparisons between group means after a significant ANOVA result.
Purpose
The main purpose of post hoc tests is to control the type I error rate that increases with multiple comparisons. These tests apply various corrections to maintain a more accurate level of statistical significance.
Why Conduct Post-hoc Tests:
Identify Differences: If the ANOVA results are significant, it only tells us that at least one group mean is different from the others. However, it doesn’t specify which groups are different from which. Post-hoc tests are conducted to pinpoint exactly which pairs of groups differ.
Control Type I Error: When making multiple comparisons, the chance of committing a Type I error (false positive) increases. Post-hoc tests adjust for this multiple comparison problem to maintain the overall Type I error rate at the desired level.
Common Post Hoc Tests
Tukey’s Honest Significant Difference (HSD): This is one of the most popular post hoc tests when all groups have equal sample sizes. It controls the family-wise error rate and is robust across a range of scenarios.
Bonferroni Correction: This method adjusts the p-value threshold by dividing it by the number of comparisons. It is very conservative, reducing the power to detect differences when numerous comparisons are made.
Scheffé’s Test: Another conservative test, Scheffé’s test is particularly useful when exploring all possible contrasts among group means, not just pairwise comparisons.
Dunnett’s Test: This test compares a control group against all other groups and is useful in clinical trials.
Fisher’s Least Significant Difference (LSD): This test does not adjust for multiple comparisons, so it has higher power but also a higher risk of type I errors.
Example: Tukey’s HSD Test in a One-Way ANOVA
Scenario
Suppose a botanist wants to compare the growth of plant species in different fertilizers. They have four types of fertilizers and measure growth (in cm) after a set period.
Data
-
Fertilizer A: 15, 14, 16, 14, 15
-
Fertilizer B: 22, 20, 21, 22, 21
-
Fertilizer C: 28, 25, 27, 30, 29
-
Fertilizer D: 15, 13, 14, 15, 14
Post Hoc Tests for one way ANOVA in R
First, we conduct a one-way ANOVA to see if there are significant differences among the means of these groups.
data <- data.frame(
growth = c(15, 14, 16, 14, 15, 22, 20, 21, 22, 21, 28, 25, 27, 30, 29, 15, 13, 14, 15, 14),
fertilizer = factor(rep(c('A', 'B', 'C', 'D'), each = 5))
)
data
growth fertilizer
1 15 A
2 14 A
3 16 A
4 14 A
5 15 A
6 22 B
7 20 B
8 21 B
9 22 B
10 21 B
11 28 C
12 25 C
13 27 C
14 30 C
15 29 C
16 15 D
17 13 D
18 14 D
19 15 D
20 14 D
anova_result <- aov(growth ~ fertilizer, data = data)
summary_anova <- summary(anova_result)
p_value <- summary_anova[[1]]["fertilizer", "Pr(>F)"]
if (p_value < 0.05) {
tukey_results <- TukeyHSD(anova_result)
print(tukey_results)
} else {
cat("No significant differences found among the means.\n")
}
Tukey multiple comparisons of means
95% family-wise confidence level
Fit: aov(formula = growth ~ fertilizer, data = data)
$fertilizer
diff lwr upr p adj
B-A 6.4 4.221112 8.578888 0.0000016
C-A 13.0 10.821112 15.178888 0.0000000
D-A -0.6 -2.778888 1.578888 0.8588883
C-B 6.6 4.421112 8.778888 0.0000011
D-B -7.0 -9.178888 -4.821112 0.0000005
D-C -13.6 -15.778888 -11.421112 0.0000000
To interpret the results from Tukey’s Honest Significant Difference (HSD) test for multiple comparisons of means as you’ve provided, let’s break down each component of the output:
Output Breakdown
diff: The difference in means between the groups being compared. lwr: The lower bound of the 95% confidence interval for the mean difference. upr: The upper bound of the 95% confidence interval for the mean difference. p adj: The p-value adjusted for multiple comparisons.
Interpretation
This test will compare every pair of fertilizers. If the differences in their means are greater than a certain critical value, Tukey’s test will indicate these differences as significant. The output will include confidence intervals for each difference and a p-value for each comparison.
Practical Application
The botanist can use the results to determine which fertilizers significantly enhance growth compared to others, guiding future experimental designs or agricultural practices.
Post hoc tests are crucial for making informed decisions after an ANOVA. They help identify specific differences between groups, allowing researchers to understand deeper nuances beyond the initial ANOVA results. When selecting a post hoc test, consider the balance between controlling type I errors and maintaining statistical power, based on the study design and objectives.