When my son was born, friends and family embarked on a furious attempt to answer the most important question of all: who does he look like? The kid has his own combination of features, but people would be strongly convinced he looks just like one of us. And on average, these estimations were random! Like any new mom, this prompted me to think about median splits.
In neuroscience, it’s not unusual to take a neural correlate of a continuous variable (such as height), then split the data across the median of this variable, ending up with two groups to compare (short vs. tall people). A forced dichotomisation, like my son’s looks.
Statistically, this is unwise: when we do it, we throw away precious (and precise) continuous data, and introduce noise instead. We work against ourselves, because we decrease the sensitivity of our statistical test and with it our chances of finding an effect. So why do we do it?
I ask this question whenever I come across a median split. People tend to react with a “but, but, but…”, and start offering answers that are so diverse that none of them can be right. The statistical argument is trivial, the people in question are trained to follow their finely honed research instincts, and they generally welcome having their ideas searched for inconsistencies. So what gives with the median split?
My best guess is that what we’re observing here is a trade-off between statistical precision and conceptual clarity. Even though it’s methodologically suboptimal, it feels right at the level of the inferences we are prepared to make. We know that we’re not able to reach down to the level of the individual and draw our conclusions there. We instead compare groups of events or groups of people, and bring new knowledge about the average behaviour of the nervous system or cognitive apparatus. The median split fits this well. Our data sometimes don’t.
Personally, I tend to side with the statistical argument – but there are definite merits to keeping to a consistent framework within one’s decision space. More research theory is needed.
(He looks just like me, of course.)