Family-wise Error Rate
Family-wise error rate refers to the rate or probability of committing one type 1 error when performing a family of comparisons.
The more hypotheses and comparisons a researcher performs, the higher chance of at least 1 type 1 error occurring in one of the variables compared1.
“For studies that test multiple hypotheses or make multiple comparisons, the probability of at least 1 Type I error (family-wise error rate; FWER) increases as the number of hypotheses/comparisons increase”1.
Uncorrected for Family-Wise Error
Without correcting for family-wise error rate, the significance level refers to “The chance that random sampling would lead this particular comparison to an incorrect conclusion that the difference is statistically significant when this particular null hypothesis is true”2.
Corrected for Family-Wise Error
When taking into account family-wise error rate, one is looking at multiple comparisons and determining the significance level as the chance of obtaining one or more statistically significant conclusions if all of the null hypotheses in the family are actually true2.
Importance
The purpose of family-wise error rate is to use a stricter threshold to define significance and reduce type 1 error2. When using the standard significance (
To organize
When each comparison is made individually without any correction for multiple comparisons, the traditional 5% significance level applies to each individual comparison2.
What is a family of comparisons
What is a family of comparisons? What exactly is a family of related comparisons? Usually, a family consists of all the comparisons in one experiment or all the comparisons in one major part of an experiment. That definition leaves lots of room for ambiguity. When reading about results corrected for multiple comparisons, ask about how the investigators defined the family of comparisons.2
Controlling Family-wise error
One can use a Bonferroni Correction to control for family-wise error2.
Supplemental Material
This is a modal window.