1 Boring Old Man

The effect of regression to the mean in epidemiologic and clinical studies.

by Davis CE.

American Journal of Epidemiology. 1976 104[5]:493-498.

"Regression to the mean is the phrase used to identify the phenomenon that a variable that is extreme on its first measurement will tend to be closer to the center of the distribution for a later measurement. In studies based on biological measurements, this variability can be attributed to both the inherent variation in the phenomenon being measured and the variability of the measurement itself. The concept of regression to the mean is an important consideration in studies where subjects are chosen because of a biological variable above or below a specified level…

It’s counter-intuitive. It means if you give a metric like the HAM-D to a group, and then pick out one subject who scores in the shaded area [above a set cut-off value], the subsequent value from that subject will tend to be lower – moving down towards the mean. Likewise, a subject with a score at the lower end’s next value will tend to be higher – again moving up towards the mean. Why don’t they stay the same? Perhaps that’s why regression to the mean is so hard to hold onto, because it’s the essence of what’s different about statistics and our everyday dealings with numbers. We’re working with a trend rather than a certainty, with values that move when no operation has been applied except resampling. And since the speed with which repeated samples tend to creep towards the mean slows down as it gets closer, it tends to to create a graph that looks familiar [often misinterpreted]. And when we go hunting for the reasons, we find explanations like this…

… and maybe that’s another reason it doesn’t stick – who’s going to carry that around in their mind? Apparently, not me. Probably you won’t either [if you even read it]. So since what we’re aiming for isn’t the why? of regression to the mean, but the why? it won’t stay in mind, maybe we should come at it from a different angle – base it on something we already know. We all seem to accept that if we repeatedly sample something in nature, we’ll get a range of answers, and if we look at the frequency in that range, it’ll look like the curve on the right. Then we can find a Mean [μ] that we accept as the true value [though they’re really all true]. And we can find a Standard Deviation [σ] that represents the variability. We even call it the normal distribution – a testimony to the fact that’s it’s what we normally expect to find in nature.

If you think about it, the shape of the normal distribution is nothing more than an example of regression to the mean. Our repeated samples move towards and collect around the Mean [μ] as if pulled there by some invisible force – a gravity [but as Einstein pointed out about Newton’s force of gravity, it’s not really a force – it’s just in the nature of things]. So regression to the mean is simply a part of sampling nature. And why? should we expect that some extreme value would do anything but trend towards the mean of the distribution it’s part of? Duh! It’s trying to go towards home or maybe home is calling [if you go for anthropomorphic metaphors].

Probably a better way to think about why? we forget about regression to the mean, or why? we expect that some extreme value would do anything but migrate toward the mean, would be to think about our thinker. We invented a mathematics of arithmetic and algebra to fit our minds, so we’re the exceptions. And we try to pull nature into the way our minds work. It’s a bit like our building things based on straight lines and right angles, whereas nature builds with an infinite series of curves [the beauty of nature we so admire]. What we call statistics is our attempt to see things as distributions rather than singularities, as trends rather than certainties, and it’s often hard going because the mind keeps trying to pull things back into our right-angle-straight-line frameworks [more anthropomorphism]. But what I call the land of sometimes is actually closer to the real world of nature than our precision mathematics.

• Placebo effects are weak:
  regression to the mean is the main reason ineffective treatments appear to work
• How much of the Placebo ‘Effect’ is really Statistical Regression?
• Francis Galton and regression to the mean
• Regression towards mediocrity in hereditary stature (1886)