O2StatisticsStretch

Binomial Hypothesis Tests

Q: Why use a tail probability rather than the probability of the exact value?

Because the question is whether the result is surprisingly extreme, and every outcome further out in the same direction is at least as surprising. Individual probabilities are small even for typical values in a large binomial.

Q: Which method should I use?

Either, unless the question specifies. Use critical regions when asked for the region or the actual significance level; the probability method is quicker for a single decision.

Q: Why is the actual significance level lower than the nominal one?

The binomial is discrete, so you take the largest critical region whose probability does not exceed. That usually falls short of it.

Q: What if the tail probability exactly equals?

It lies in the critical region, since the condition is, so you reject H_0. Exact equality is rare with a discrete distribution.

The first test you meet applies the logic of the previous lesson to a population $\textbf{proportion}$ . The observed count of successes is compared against what a binomial with the hypothesised $p$ would produce, and the question is whether the result is too extreme to be comfortable.

40 min Hypothesis Testing

The big picture

There are two routes to the same conclusion and it is worth being fluent in both. The $\textbf{probability method}$ asks how likely a result at least as extreme as the observed one would be, and compares that with $\alpha$ . The $\textbf{critical region method}$ works out in advance which outcomes would trigger rejection, then checks whether the observation lands there. They always agree, but questions ask for them by name — and the critical region method is the only practical one when a question asks for the actual significance level or for the region itself.

What you'll be able to do

Carry out a one-tailed binomial test using the probability method
Find critical regions and the actual significance level
Carry out a two-tailed test, splitting the significance level correctly
Write conclusions in context with the comparison shown

The probability method

State the hypotheses, define the distribution under $H_{0}$ , then compute the probability of a result $\textbf{at least as extreme}$ as the one observed, in the direction of $H_{1}$ .

For $H_{1}: p < p_{0}$ with an observed count $x$ , that is $P(X \leq x)$ . For $H_{1}: p > p_{0}$ , it is $P(X \geq x) = 1 - P(X \leq x-1)$ .

Compare with $\alpha$ . If the probability is smaller, the result is unlikely enough under $H_{0}$ to reject it; if larger, there is insufficient evidence.

The phrase "at least as extreme" matters. Using $P(X = x)$ rather than the tail probability is a standard error — a single outcome can be unlikely without the result being surprising in the relevant direction.

H_{1}: p < p_{0} \Rightarrow \text{find } P(X \leq x)

Lower tail.

H_{1}: p > p_{0} \Rightarrow \text{find } P(X \geq x)

Upper tail.

Example 1A seed company claims a 70% germination rate. In 20 seeds, 10 germinate. Test at

5\%

whether the rate is lower than claimed.

H_{0}: p = 0.7

;

H_{1}: p < 0.7

. Under

H_{0}

X \sim B(20, 0.7)

2Observed 10, and lower values are more extreme: find

P(X \leq 10)

3From cumulative binomial,

P(X \leq 10) = 0.0480

4Compare:

0.0480 < 0.05

AnswerReject

H_{0}

. There is sufficient evidence at the

5\%

level to suggest the germination rate is below 70%.

Tip — Write the comparison line explicitly — " $0.0480 < 0.05$ , so reject $H_{0}$ ". It is where the decision mark lives.

Critical regions and actual significance

The critical region is worked out before looking at the observation. For a lower-tailed test, find the largest $c$ with $P(X \leq c) \leq \alpha$ ; the region is $X \leq c$ .

Because the binomial is discrete you cannot usually hit $\alpha$ exactly, so you take the largest region that does not exceed it. The probability of that region is the $\textbf{actual significance level}$ , and it will be below the nominal figure.

For an upper-tailed test the same logic applies from the top: find the smallest $c$ with $P(X \geq c) \leq \alpha$ .

P(X \leq c) \leq \alpha

Largest such

c

gives the lower critical region.

\text{actual } \alpha = P(\text{critical region})

Below the nominal level for a discrete distribution.

Example 1For

X \sim B(20, 0.7)

and

H_{1}: p < 0.7

5\%

, find the critical region and the actual significance level. Given

P(X\leq10) = 0.0480

and

P(X\leq11) = 0.1133

P(X\leq11) = 0.1133 > 0.05

, so 11 is not in the region.

P(X\leq10) = 0.0480 \leq 0.05

, so 10 is.

3The critical region is

X \leq 10

AnswerCritical region

X \leq 10

; actual significance level

4.80\%

Why this works

Notice the jump from $4.80\%$ to $11.33\%$ between consecutive values. Discreteness means the achievable significance levels come in steps, which is exactly why the actual level rarely equals the nominal one.

Two-tailed tests

A two-tailed test splits the significance level, putting $\tfrac{\alpha}{2}$ in each tail. At $5\%$ that means $2.5\%$ at each end.

Using the probability method, find the tail probability in the direction the observation actually fell and compare it with $\tfrac{\alpha}{2}$ — not with $\alpha$ . Comparing against the full level is the commonest error in two-tailed questions.

Using critical regions, both tails must be found: the largest $c_{1}$ with $P(X \leq c_{1}) \leq \tfrac{\alpha}{2}$ and the smallest $c_{2}$ with $P(X \geq c_{2}) \leq \tfrac{\alpha}{2}$ . The actual significance level is then the sum of the two tail probabilities.

\text{compare with } \tfrac{\alpha}{2}

Two-tailed, probability method.

\text{actual } \alpha = P(X\leq c_{1}) + P(X \geq c_{2})

Sum of both tails.

Example 1A coin is tossed 20 times giving 15 heads. Test at

5\%

whether the coin is biased. Given

P(X \geq 15) = 0.0207

for

X \sim B(20,0.5)

H_{0}: p = 0.5

;

H_{1}: p \neq 0.5

— "biased" is directionless, so two-tailed.

2The observation is in the upper tail, so use

P(X \geq 15) = 0.0207

3Compare with

\tfrac{0.05}{2} = 0.025

0.0207 < 0.025

AnswerReject

H_{0}

. There is sufficient evidence at the

5\%

level to suggest the coin is biased.

Tip — Against the full $5\%$ this result would also be significant, but the two-tailed comparison is the correct one and questions are often set so the two answers differ.

Setting out a complete test

Every binomial test should show the same five stages, and the marks are distributed across them rather than concentrated in the arithmetic.

First define the parameter in words. Second state $H_{0}$ and $H_{1}$ in symbols. Third state the distribution assumed under $H_{0}$ . Fourth compute the tail probability or critical region and show the comparison. Fifth conclude in context, tentatively.

Missing the definition or the context sentence loses marks even when the calculation is perfect, which is why the layout is worth practising as a routine.

Example 1A treatment historically works for 40% of patients. In a trial of 25, it works for 15. Test at

5\%

whether it has improved. Given

P(X \geq 15) = 0.0344

1Let

p

be the proportion of all patients for whom the treatment works.

H_{0}: p = 0.4

;

H_{1}: p > 0.4

. Under

H_{0}

X \sim B(25, 0.4)

3"Improved" is one-tailed upper, so find

P(X \geq 15) = 0.0344

0.0344 < 0.05

AnswerReject

H_{0}

. There is sufficient evidence at the

5\%

level to suggest the treatment’s success rate has improved.

Why this works

Note the conclusion says "suggest", not "prove". With a $5\%$ level, results this extreme arise by chance one time in twenty even when nothing has changed — the tentativeness is an accurate description of what the test establishes.

Think like an examiner

Define the parameter in words before writing the hypotheses.
State the distribution under $H_{0}$ explicitly: "Under $H_{0}$ , $X \sim B(20, 0.7)$ ."
Use the tail probability, not $P(X = x)$ , and show the comparison against the level.
Halve the significance level for two-tailed tests, and sum both tails for the actual level.

Common misconceptions

Computing

P(X = x)

rather than the tail probability.

The test asks how likely a result at least as extreme is, so use

P(X \leq x)

P(X \geq x)

Comparing against

\alpha

in a two-tailed test.

Compare the single-tail probability against

\tfrac{\alpha}{2}

Quoting the nominal level when asked for the actual significance level.

The actual level is the probability of the critical region, usually lower.

Concluding "the rate has improved".

Say there is sufficient evidence to suggest it, at the stated level.

Binomial testing

X \sim B(n, p_{0}) \text{ under } H_{0}

The assumed distribution.

P(X \geq x) = 1 - P(X\leq x-1)

Upper tail.

P(X\leq c) \leq \alpha

Critical region condition.

\text{two-tailed: use } \tfrac{\alpha}{2}

Split the level.

Stretch yourself

A manufacturer claims at most 5% of its components are faulty. A sample of 50 contains 6 faulty components. Test the claim at the $5\%$ significance level, given $P(X \geq 6) = 0.1036$ and $P(X \geq 7) = 0.0378$ for $X \sim B(50, 0.05)$ . Also state the critical region.

Hint — The claim is "at most 5%", so the suspicion is that the rate is higher. That fixes the tail.

Reveal solution

SolutionLet

p

be the proportion of all components that are faulty.

H_{0}: p = 0.05

against

H_{1}: p > 0.05

, one-tailed since the doubt is that the rate exceeds the claim. Under

H_{0}

X \sim B(50, 0.05)

. The observed value is 6, so the tail probability is

P(X \geq 6) = 0.1036

. Comparing,

0.1036 > 0.05

, so do

\textbf{not}

reject

H_{0}

— there is insufficient evidence at the

5\%

level to suggest more than 5% of components are faulty. For the critical region, we need the smallest

c

with

P(X \geq c) \leq 0.05

: since

P(X\geq6) = 0.1036 > 0.05

but

P(X \geq 7) = 0.0378 \leq 0.05

, the critical region is

X \geq 7

, with actual significance level

3.78\%

. Six faulty components sits just outside it — a useful reminder that a result can look bad without being statistically significant.

\blacksquare

Questions students ask

Why use a tail probability rather than the probability of the exact value?

Because the question is whether the result is surprisingly extreme, and every outcome further out in the same direction is at least as surprising. Individual probabilities are small even for typical values in a large binomial.

Which method should I use?

Either, unless the question specifies. Use critical regions when asked for the region or the actual significance level; the probability method is quicker for a single decision.

Why is the actual significance level lower than the nominal one?

The binomial is discrete, so you take the largest critical region whose probability does not exceed

\alpha

. That usually falls short of it.

What if the tail probability exactly equals

\alpha

It lies in the critical region, since the condition is

\leq \alpha

, so you reject

H_{0}

. Exact equality is rare with a discrete distribution.

Key takeaways

Under $H_{0}$ the count follows $B(n, p_{0})$ ; compare a tail probability with the level.
Use $P(X \leq x)$ for a lower-tailed test and $P(X \geq x)$ for an upper-tailed one.
The critical region is the largest set of extreme outcomes with probability at most $\alpha$ .
Its probability is the actual significance level, which is below the nominal one.
Two-tailed tests compare against $\tfrac{\alpha}{2}$ in the relevant tail.