Classical Multiple Comparison Procedures aim at
controlling the probability of committing even a single type-I error within
the tested family of hypotheses. The main problem with such classical
procedures, which hinder their application in applied research, is that they
tend to have substantially less power than uncorrected procedures. In many
instances, lack of multiplicity control is too permissive; the full
protection resulting from controlling the FWE is too restrictive. This is
the case when the overall conclusion from the various individual inferences
is not necessarily erroneous as soon as one of them is, yet selection effect
is still of concern. Benjamini and Hochberg (1995) introduced the False Discovery Rate
(FDR) - the expected ratio of erroneous rejections to the number of rejected
hypotheses, as an appropriate error rate to control. The FDR is equal to the
family wise error rate when the number of true null hypotheses mo equals the number of all hypotheses under test
m, so in such a situation controlling the FDR
controls the FWE as well. But the FDR criterion is adaptive, in the sense
that when some of the tested hypotheses are not true (i.e. mo < m), the FDR is smaller, and more so when more of the
hypotheses are not true. Hence FDR controlling procedures can be more
powerful than FWE controlling procedures at the same level.
 |
history |