L2StatisticsStretch

Correlation and Regression

Q: Why can I not use the regression line in reverse?

The regression of y on x minimises vertical deviations, treating x as known. Predicting x from y requires minimising horizontal deviations, which gives a different line.

Bivariate data pairs two measurements from each subject — height and weight, revision hours and marks. A scatter diagram shows whether they move together, the $\textbf{correlation coefficient}$ measures how strongly, and the $\textbf{regression line}$ lets you predict one from the other.

35 min Data Presentation

The big picture

Two ideas here carry more weight than the arithmetic. The first is that correlation never establishes causation: a $\textbf{confounding}$ variable can drive both quantities, or the relationship can be coincidence in a small sample. The second is that a regression line is only trustworthy inside the range of data used to build it — pushing beyond that range assumes a pattern continues when nothing in the data says it does. Both points appear in exam questions as "comment on the validity" marks, and both are the difference between using statistics and misusing it.

What you'll be able to do

Interpret scatter diagrams and describe correlation in context
Interpret the product moment correlation coefficient $r$
Use the regression line $y = a + bx$ to make predictions
Distinguish interpolation from extrapolation and judge reliability

Scatter diagrams and describing correlation

Plot the $\textbf{explanatory}$ (independent) variable on the horizontal axis and the $\textbf{response}$ (dependent) variable on the vertical. The pattern of points shows the relationship.

Describe correlation with three words: its $\textbf{direction}$ (positive or negative), its $\textbf{strength}$ (weak, moderate, strong) and its $\textbf{form}$ (linear or not). Then say what it means for the variables in question — the context sentence is usually a separate mark.

A scatter diagram also reveals things a single number cannot: a curved relationship, two distinct clusters, or a single point driving an apparent trend.

Why this works

Always look at the scatter before trusting a correlation coefficient. A strong curved relationship can produce an $r$ near zero, because $r$ measures $\textit{linear}$ association only — the number says "no straight-line pattern", not "no relationship".

The product moment correlation coefficient

The PMCC, written $r$ , measures the strength and direction of a $\textbf{linear}$ relationship. It always lies between $-1$ and $1$ .

$r = 1$ is perfect positive linear correlation with every point exactly on an upward line; $r = -1$ is the perfect downward equivalent; $r = 0$ means no linear relationship. Values around $\pm0.8$ or beyond are usually described as strong, and around $\pm0.3$ as weak.

$r$ has no units and is unaffected by linear coding of either variable, which is why converting units cannot change it.

-1 \leq r \leq 1

The range of the PMCC.

r = \pm 1 \Rightarrow \text{points exactly on a line}

Perfect linear correlation.

Example 1Interpret

r = -0.87

for hours of television watched and exam mark.

1The sign is negative, so as one increases the other tends to decrease.

2The size, close to 1, indicates a strong relationship.

3State it in context, and stop short of causation.

AnswerStrong negative linear correlation: students who watched more television tended to score lower. This does not show that television causes lower marks.

Tip — Never write " $r$ shows that $x$ causes $y$ ". The available mark is for saying the opposite — that correlation alone cannot establish causation.

The regression line

The least-squares regression line $y = a + bx$ is the straight line minimising the sum of the squared vertical distances from the points. $b$ is the gradient — the change in $y$ per unit increase in $x$ — and $a$ is the intercept.

Interpreting the coefficients in context is routinely examined. If $y$ is cost in pounds and $x$ is items produced, $b$ is the cost per additional item and $a$ is the fixed cost when nothing is produced — though only if $x = 0$ is within the sensible range.

The line is $\textbf{asymmetric}$ : the regression of $y$ on $x$ minimises vertical distances and is used to predict $y$ from $x$ . Using it backwards to predict $x$ from $y$ is not valid, because a different line would be needed.

y = a + bx

Regression of

y

x

b = \text{change in } y \text{ per unit } x

Interpret in context.

Example 1For revision hours

x

and mark

y

, the regression line is

y = 22 + 4.6x

. Interpret both coefficients.

1Gradient 4.6: each extra hour of revision is associated with about 4.6 more marks.

2Intercept 22: the predicted mark with no revision — meaningful only if

x = 0

lies within the data range.

AnswerEach additional hour is associated with roughly 4.6 extra marks; 22 is the predicted mark at zero hours.

Why this works

Note the wording "is associated with" rather than "causes". Students who revise more may also differ in other ways, and the regression line cannot separate those effects.

Interpolation, extrapolation and validity

$\textbf{Interpolation}$ predicts within the range of the observed $x$ values and is reasonably reliable, because the line was fitted to data covering that region.

$\textbf{Extrapolation}$ predicts outside that range and is unreliable. Nothing in the data supports the assumption that the linear pattern continues, and relationships frequently bend, level off or reverse beyond the observed range.

When asked to comment on a prediction, check first whether the $x$ value lies inside the data range, and say so explicitly. That is where the mark is.

Example 1A regression line is built from revision hours between 2 and 12. Comment on using it to predict the mark for 25 hours.

125 lies well outside the observed range of 2 to 12.

2This is extrapolation, so the linear pattern is assumed rather than evidenced.

3Realistically marks are capped and returns diminish, so the prediction may exceed the maximum possible.

AnswerUnreliable — it extrapolates far beyond the data, and the relationship is unlikely to remain linear at 25 hours.

Tip — State the data range in your answer. "25 is outside the range 2–12 used to fit the line" is a concrete justification; "it is extrapolation" alone is thinner.

Think like an examiner

Describe correlation with direction, strength and context — three elements, often three marks.
Interpret $a$ and $b$ using the units of the variables, not abstractly.
Always check whether a prediction interpolates or extrapolates and quote the data range.
Explicitly refuse causal claims: correlation may arise from a confounding variable.

Common misconceptions

Concluding that

x

causes

y

from a high value of

r

Correlation is association only. A confounding variable or coincidence may explain it.

Using the regression of

y

x

to predict

x

from

y

The line minimises vertical distances only; predicting

x

needs the other regression line.

Reporting

r = 0

as "no relationship".

It means no

\textit{linear}

relationship. A strong curved pattern can give

r \approx 0

Interpreting the intercept when

x = 0

is far outside the data.

It is then an extrapolation and may be meaningless — say so.

Bivariate data

-1 \leq r \leq 1

PMCC range.

y = a+bx

Regression of

y

x

b: \Delta y \text{ per unit } \Delta x

Gradient in context.

\text{within range} \Rightarrow \text{interpolation}

The reliable case.

Stretch yourself

Across a country’s towns, ice cream sales and drowning incidents are strongly positively correlated, $r = 0.82$ . A newspaper concludes that ice cream causes drowning. Explain the flaw, name the likely mechanism, and state what evidence would be needed for a causal claim.

Hint — Ask what third variable might independently raise both quantities.

Reveal solution

SolutionThe flaw is inferring causation from correlation. The likely mechanism is a

\textbf{confounding variable}

: hot weather. High temperatures independently increase ice cream sales and increase swimming, which raises exposure to drowning risk. Neither quantity influences the other; both respond to a third. The correlation is real but the causal claim is not supported by it. Establishing causation would require either a controlled experiment — manipulating ice cream availability while holding temperature and swimming behaviour fixed, which is impractical and unethical here — or an observational analysis that statistically controls for temperature and shows the association persists. If the correlation vanishes once temperature is accounted for, the confounder explains it entirely.

\blacksquare

Questions students ask

Does

r = 0

mean the variables are unrelated?

No. It means there is no linear relationship. A strong symmetric curve can produce

r

near zero, which is why the scatter diagram should always be inspected.

Why can I not use the regression line in reverse?

The regression of

y

x

minimises vertical deviations, treating

x

as known. Predicting

x

from

y

requires minimising horizontal deviations, which gives a different line.

How large must

r

be to count as strong?

There is no fixed threshold and it depends on the field. Around

\pm0.8

or beyond is usually described as strong at A-Level; describe it in words rather than claiming a cutoff.

When is extrapolation ever acceptable?

Only just beyond the data range, and only with a stated reason to expect the relationship to continue. Even then the prediction should be flagged as less reliable.

Key takeaways

Scatter diagrams show direction, strength and form, and reveal patterns a single number hides.
$r$ measures linear correlation only and lies between $-1$ and $1$ .
The regression line $y = a+bx$ predicts $y$ from $x$ and cannot be used in reverse.
Interpret $a$ and $b$ in the units of the problem.
Interpolation is reliable; extrapolation assumes a pattern the data does not evidence.