2.05StatisticsStretch

Hypothesis Testing

Q: What is the difference between one-tailed and two-tailed tests?

One-tailed tests look for a change in a specified direction; two-tailed tests look for any change, so the significance level is split between both tails.

Q: Why can’t I accept the null hypothesis?

Failing to find evidence against a claim is not evidence for it. You say there is insufficient evidence to reject H_0.

Q: Why is the actual significance level smaller than 5%?

The binomial distribution is discrete, so tail probabilities jump in steps and rarely equal exactly 5%.

Q: Where does the direction of H_1 come from?

From the question’s wording — "suspects it has increased" gives >, "believes it has changed" gives. Decide it before looking at the data.

A company claims 25% of its customers win a prize. In a sample of 20, nine win. Is the claim wrong, or was the sample just lucky? $\textbf{Hypothesis testing}$ gives a principled way to decide: assume the claim is true, work out how surprising the evidence would be, and reject the claim only if the evidence is surprising enough.

50 min Probability

The big picture

A hypothesis test is an argument by contradiction with a probability threshold. The null hypothesis $\mathrm{H}_{0}$ fixes a parameter at its claimed value; the alternative $\mathrm{H}_{1}$ says how you suspect it differs. You then calculate the probability of a result at least as extreme as the one observed, assuming $\mathrm{H}_{0}$ . If that probability falls below the significance level, the result is too unlikely to blame on chance, and $\mathrm{H}_{0}$ is rejected. OCR tests this with the binomial distribution in Year 1, and with the normal distribution for a sample mean in Year 2. The mathematics is short; what earns the marks is precise language — hypotheses in terms of the parameter, a comparison with the significance level, and a conclusion in context that never claims certainty.

What you'll be able to do

Write null and alternative hypotheses in terms of a population parameter
Carry out one-tailed and two-tailed binomial tests using probabilities or critical regions
Find critical regions and actual significance levels
Test a population mean using the normal distribution of a sample mean
Write conclusions in context with appropriately non-definite language

The language of hypothesis tests

The $\textbf{null hypothesis}$ $\mathrm{H}_{0}$ states the parameter takes the claimed value, e.g. $\mathrm{H}_{0}: p = 0.25$ . The $\textbf{alternative hypothesis}$ $\mathrm{H}_{1}$ states how it might differ: $p > 0.25$ or $p < 0.25$ (one-tailed), or $p \neq 0.25$ (two-tailed).

The $\textbf{test statistic}$ is the observed value, such as the number of successes $X$ . The $\textbf{significance level}$ , often 5%, is the probability threshold below which a result is considered too unlikely under $\mathrm{H}_{0}$ .

The $\textbf{critical region}$ is the set of values of the test statistic that would lead to rejecting $\mathrm{H}_{0}$ ; its boundary is the $\textbf{critical value}$ .

Tip — Hypotheses are always about the population parameter ( $p$ or $\mu$ ), never about the sample result. Writing " $\mathrm{H}_{0}: X = 9$ " earns nothing.

A one-tailed binomial test

Under $\mathrm{H}_{0}$ , $X \sim \mathrm{B}(n, p_{0})$ . For $\mathrm{H}_{1}: p > p_{0}$ , calculate $\mathrm{P}(X \ge x_{\text{obs}})$ ; for $\mathrm{H}_{1}: p < p_{0}$ , calculate $\mathrm{P}(X \le x_{\text{obs}})$ .

If this probability is less than the significance level, reject $\mathrm{H}_{0}$ . Otherwise, do not reject it.

Example 1A company claims 25% of customers win a prize. A journalist suspects the true figure is higher. In a random sample of 20, nine win. Test at the 5% level.

\mathrm{H}_{0}: p = 0.25

\mathrm{H}_{1}: p > 0.25

, where

p

is the proportion of all customers who win.

2Under

\mathrm{H}_{0}

X \sim \mathrm{B}(20, 0.25)

\mathrm{P}(X \ge 9) = 1 - \mathrm{P}(X \le 8) = 0.0409

0.0409 < 0.05

, so the result is significant: reject

\mathrm{H}_{0}

AnswerThere is sufficient evidence at the 5% level to suggest that more than 25% of customers win a prize.

Why this works

The probability is of "9 $\textit{or more}$ ", not "exactly 9". Any single value is fairly unlikely in a spread-out distribution; the question is whether the observed result lies in the extreme tail.

Critical regions and two-tailed tests

To find a critical region for $\mathrm{H}_{1}: p > p_{0}$ , find the smallest $c$ such that $\mathrm{P}(X \ge c) \le$ the significance level. The $\textbf{actual significance level}$ is $\mathrm{P}(X \ge c)$ , usually a little below the nominal level because the binomial is discrete.

For a two-tailed test at 5%, split the significance level: find a critical region in each tail with probability at most 2.5%.

Example 1Find the critical region for testing

\mathrm{H}_{0}: p = 0.4

against

\mathrm{H}_{1}: p \neq 0.4

with

n = 25

at the 5% level, and the actual significance level.

1Lower tail:

\mathrm{P}(X \le 5) = 0.0294 > 0.025

but

\mathrm{P}(X \le 4) = 0.0095 \le 0.025

, so

X \le 4

2Upper tail:

\mathrm{P}(X \ge 15) = 0.0344 > 0.025

but

\mathrm{P}(X \ge 16) = 0.0132 \le 0.025

, so

X \ge 16

3Actual significance level

= 0.0095 + 0.0132

AnswerCritical region

X \le 4

X \ge 16

; actual significance level

2.27\%

Tip — Show the two probabilities either side of each boundary — one just above and one just below the tail threshold. That is what justifies the critical value.

Testing a mean with the normal distribution

If $X \sim \mathrm{N}(\mu, \sigma^{2})$ , the mean of a random sample of size $n$ satisfies $\bar{X} \sim \mathrm{N}\left(\mu, \dfrac{\sigma^{2}}{n}\right)$ . Larger samples give sample means that cluster more tightly around $\mu$ .

To test $\mathrm{H}_{0}: \mu = \mu_{0}$ , calculate $\mathrm{P}(\bar{X} \ge \bar{x})$ (or $\le$ ), or compare $z = \dfrac{\bar{x} - \mu_{0}}{\sigma/\sqrt{n}}$ with the critical $z$ -value.

\bar{X} \sim \mathrm{N}\left(\mu, \dfrac{\sigma^{2}}{n}\right), \qquad z = \dfrac{\bar{x} - \mu_{0}}{\sigma / \sqrt{n}}

Distribution of the sample mean.

Example 1Battery lifetimes are

\mathrm{N}(\mu, 6^{2})

hours. The maker claims

\mu = 50

. A sample of 16 has mean 52.7 hours. Test at the 5% level whether the mean is greater than 50.

\mathrm{H}_{0}: \mu = 50

\mathrm{H}_{1}: \mu > 50

2Under

\mathrm{H}_{0}

\bar{X} \sim \mathrm{N}\left(50, \tfrac{36}{16}\right)

, standard deviation

1.5

z = \dfrac{52.7 - 50}{1.5} = 1.8 > 1.645

; equivalently

\mathrm{P}(\bar{X} \ge 52.7) = 0.0359 < 0.05

AnswerReject

\mathrm{H}_{0}

: there is sufficient evidence at the 5% level that the mean battery life exceeds 50 hours.

Think like an examiner

Define the parameter in words and write hypotheses using it.
State the distribution under $\mathrm{H}_{0}$ .
Compare the probability or test statistic with the significance level explicitly.
Conclude in context with non-definite language: "there is sufficient evidence to suggest...".

Common misconceptions

Calculating

\mathrm{P}(X = 9)

instead of

\mathrm{P}(X \ge 9)

Use the probability of the observed value or more extreme.

Concluding "the claim is proven false".

Tests give evidence, not proof: "sufficient evidence to suggest...".

Using the full 5% in each tail of a two-tailed test.

Split it: 2.5% in each tail.

Using

\sigma

instead of

\dfrac{\sigma}{\sqrt{n}}

for a sample mean.

The sample mean has standard deviation

\sigma/\sqrt{n}

Hypothesis testing

\mathrm{H}_{0}: p = p_{0}

Null hypothesis.

\mathrm{P}(X \ge x_{\text{obs}}) < \alpha \Rightarrow \text{reject } \mathrm{H}_{0}

Upper-tail decision.

\bar{X} \sim \mathrm{N}(\mu, \sigma^{2}/n)

Sample mean.

Stretch yourself

A coin is suspected of being biased towards tails. It is tossed 15 times. Using a 5% significance level, find the critical region for a test of $\mathrm{H}_{0}: p = 0.5$ against $\mathrm{H}_{1}: p < 0.5$ , where $p$ is the probability of heads, and state the actual significance level. Then explain what a result of 4 heads would mean.

Hint — Under $\mathrm{H}_{0}$ , $X \sim \mathrm{B}(15, 0.5)$ . Find the largest $c$ with $\mathrm{P}(X \le c) \le 0.05$ .

Reveal solution

SolutionUnder

\mathrm{H}_{0}

X \sim \mathrm{B}(15, 0.5)

\mathrm{P}(X \le 3) = 0.0176

and

\mathrm{P}(X \le 4) = 0.0592

. Since

0.0176 \le 0.05 < 0.0592

, the critical region is

X \le 3

, with actual significance level

1.76\%

. A result of 4 heads is not in the critical region, so we do not reject

\mathrm{H}_{0}

: there is insufficient evidence at the 5% level to suggest the coin is biased towards tails. This does not show the coin is fair — only that 15 tosses were not enough to detect bias of this size.

\blacksquare

Questions students ask

What is the difference between one-tailed and two-tailed tests?

One-tailed tests look for a change in a specified direction; two-tailed tests look for any change, so the significance level is split between both tails.

Why can’t I accept the null hypothesis?

Failing to find evidence against a claim is not evidence for it. You say there is insufficient evidence to reject

\mathrm{H}_{0}

Why is the actual significance level smaller than 5%?

The binomial distribution is discrete, so tail probabilities jump in steps and rarely equal exactly 5%.

Where does the direction of

\mathrm{H}_{1}

come from?

From the question’s wording — "suspects it has increased" gives

>

, "believes it has changed" gives

\neq

. Decide it before looking at the data.

Key takeaways

Hypotheses are statements about a population parameter.
Assume $\mathrm{H}_{0}$ , find the probability of a result at least as extreme as observed, and compare with the significance level.
Critical regions: the most extreme values with total probability at most the significance level (split for two tails).
For means, use $\bar{X} \sim \mathrm{N}(\mu, \sigma^{2}/n)$ .
Conclude in context, without claiming certainty.