Loading...
An is a value far from the rest of the data. Finding one is arithmetic; deciding what to do with it is not. A recording error should be removed — but a genuine extreme value may be the most important observation you have.
The big picture
This is the point in the statistics course where judgement is examined directly rather than calculation. Deleting inconvenient data is the oldest way to make a result look better than it is, so the discipline is to justify any removal with a reason external to the number itself — an impossible value, a known instrument fault, a duplicated entry. "It spoiled my correlation" is not a reason. The habit matters beyond the exam: the difference between cleaning data and manipulating it is exactly whether the justification would survive someone else reading it.
What you'll be able to do
The flags any value more than below or above . It pairs with the median and IQR and is unaffected by the outlier it is testing for, which makes it the more robust choice.
The flags values more than a stated number of standard deviations from the mean — usually two or three. It is quicker when you already have and , but it has a circularity: a large outlier inflates , which widens the boundary, which can hide the outlier that caused it.
Exam questions specify which rule to use. Apply the one given, show the boundary calculation, and compare each candidate value against it explicitly.
The two rules disagree more often than students expect. A value can be an outlier by one and not the other, which is why the question always tells you which to apply — and why quoting the rule you used is part of the answer.
The moves, sometimes a great deal, because every value contributes to it. The barely moves, because only position matters and one extra extreme value shifts the middle by at most one place.
The is affected most of all. It squares deviations, so a value far from the mean contributes disproportionately — a single outlier can inflate noticeably. The is untouched, since it discards the outer quarters entirely.
That pattern is why median and IQR are described as statistics, and why they are the right summary for data known to contain extremes.
Tip — If a question asks which average is "more appropriate" for data with an outlier, the answer is the median — but the mark is for saying : the mean is distorted by the extreme value.
Remove an outlier when there is evidence it is : a physically impossible value, a known equipment failure, an obvious transcription slip such as a decimal point in the wrong place, or a duplicate record.
Retain it when it appears to be a genuine measurement. Unusual is not the same as wrong, and genuine extremes are often the most informative points in a data set — the unusually hot day, the exceptional performance, the fault you are trying to detect.
Whichever you do, . An analysis that silently drops values cannot be checked by anyone else, and in an exam the justification is where the marks are.
A useful discipline is to report the analysis both ways when a removal is debatable — with and without the value. If the conclusion holds either way it is robust; if it does not, that fact is itself worth knowing.
Outliers are the examined case, but real cleaning covers more. are excluded with the reduced sample size stated, never replaced by zero. are removed, since they double-weight one observation.
must be converted before combining — a column mixing metres and centimetres will produce nonsense silently. , such as a negative age or a percentage above 100, indicate errors regardless of how far they sit from the mean.
The general principle throughout: document every change so the analysis can be reproduced by someone else.
Tip — A value can be an error without being an outlier. A misrecorded 45 among values from 40 to 50 fails no outlier test — which is why cleaning also means checking that values are possible.
Think like an examiner
Common misconceptions
Outliers
Stretch yourself
A set of eight reaction times (seconds) is: . Use the quartile rule to test the largest value, then advise whether to remove it, considering two possible explanations.
Hint — Find the quartiles from the ordered list first. Then think about what could physically produce a 1.90 s reaction time.
Questions students ask
Key takeaways
How this fits the course
Test yourself
Ready to lock in Outliers and Cleaning Data? Pick a mode and earn XP & Dobloons.
Real past-paper questions on Outliers and Cleaning Data, marked mark-by-mark. How you do feeds straight into your weak-topic list, so your revision keeps targeting what actually needs work.