Hypothesis Testing

The $\text{null hypothesis}$ ${\mathrm{H}}_0$ states the parameter takes the claimed value, e.g. ${\mathrm{H}}_0:p=0.25$. The $\text{alternative hypothesis}$ ${\mathrm{H}}_1$ states how it might differ: $p>0.25$ or $p<0.25$ (one-tailed), or $p\ne 0.25$ (two-tailed).

The $\text{test statistic}$ is the observed value, such as the number of successes $X$. The $\text{significance level}$, often 5%, is the probability threshold below which a result is considered too unlikely under ${\mathrm{H}}_0$.

The $\text{critical region}$ is the set of values of the test statistic that would lead to rejecting ${\mathrm{H}}_0$; its boundary is the $\text{critical value}$.

Under ${\mathrm{H}}_0$, $X\sim \mathrm{B}(n,p_0)$. For ${\mathrm{H}}_1:p>p_0$, calculate $\mathrm{P}(X\ge x_{\text{obs}})$; for ${\mathrm{H}}_1:p<p_0$, calculate $\mathrm{P}(X\le x_{\text{obs}})$.

If this probability is less than the significance level, reject ${\mathrm{H}}_0$. Otherwise, do not reject it.

To find a critical region for ${\mathrm{H}}_1:p>p_0$, find the smallest $c$ such that $\mathrm{P}(X\ge c)\le$ the significance level. The $\text{actual significance level}$ is $\mathrm{P}(X\ge c)$, usually a little below the nominal level because the binomial is discrete.

For a two-tailed test at 5%, split the significance level: find a critical region in each tail with probability at most 2.5%.

If $X\sim \mathrm{N}(\mu ,{\sigma }^2)$, the mean of a random sample of size $n$ satisfies $\bar{X}\sim \mathrm{N}\left(\mu ,\frac{{\sigma }^2}{n}\right)$. Larger samples give sample means that cluster more tightly around $\mu$.

To test ${\mathrm{H}}_0:\mu ={\mu }_0$, calculate $\mathrm{P}(\bar{X}\ge \bar{x})$ (or $\le$), or compare $z=\frac{\bar{x}-{\mu }_0}{\sigma /\sqrt{n}}$ with the critical $z$-value.