Multiplicity Adjustment

Controlling the probability of falsely rejecting the null hypothesis is critical when there are multiple, simultaneous hypotheses. The most common method is to control the family-wise error rate (FWER) which guarantees the probability of falsely rejecting at least one of the hypotheses to be within a desired level. As the number of hypotheses to be tested grew larger, the Bonferroni correction is too conservative and lacking power. This leads to the introduction of the False Discovery Rate (FDR) which is defined to be the expected proportion of falsely rejected hypotheses out of all rejected hypotheses. Several modern stepwise methods controlling FDR have been proposed to increase power in the presence of too many hypotheses. We give an overview on the classical and modern multiplicity adjustment methods as well as how to run the procedures in R.