It was when Kate Button started her PhD, that she began to appreciate the disconnect between the methods teaching she had prior to the PhD, and the knowledge she needed for actual research. Her early statistics lessons were more about determining which line to read in the SPSS output than about the inferential process. Little … [Read more…]
Say that you are new to research. You’ve read all the textbooks, passed all your exams, and are now beginning to perform empirical investigations. Everyone who has stepped into the lab like this is familiar with the explosion of new elements to master, things that seemed straightforward until you sat down to do them yourself. … [Read more…]
They ride in at dawn while the world quietly slumbers, cocooned in innocence, unsuspecting of the storm that is already under way. Immensely powerful, they gallop in bringing chaos. They are the four horsemen of the reproducibility apocalypse: HARKing, low power, p-hacking and publication bias. Who will resist and who will succumb? Who will be … [Read more…]
Here is an opportunity to view a series of timely talks on reproducibility, online. This September we held the Oxford Reproducibility School, a meeting aimed at discussing current problems in empirical science as well as best research practices. Spurred into action by Kia Nobre, organized by Dorothy Bishop, side-kicked by Caroline Nettekoven, Verena Heise and … [Read more…]
The New York Times recently published Susan Dominus’ piece, When the Revolution Came for Amy Cuddy. The article describes the replication crisis from the vantage point of its refraction through Amy Cuddy’s professional and personal life. The writing is incredible. Dominus talks about the changing rules of science, about a professional life that surged and … [Read more…]
Things I’ve learned today: make an empty room recording before piloting on people. Check if my experimental effects come out in the absence of a brain in the helmet. Then pilot. I usually use brief, pure tones in my experiments, and I usually do MEG. There is a stimulus computer outside the MEG room, that … [Read more…]
This is going to be one of those annoying posts where I tell you to first go and read something else before coming back. Sometimes, when research results are too good to be true, people start thinking there might be something fishy going on. Jens Foerster, for instance, was called out on the excessive linearity of his … [Read more…]
The rules of science are changing, to the exhilaration of some and apprehension of others. The problem is by now well defined: when we run an analysis on an unstable effect using a small sample, we can get a variety of different statistical outcomes. If only some of those outcomes are acceptable by journal standards, the literature will produce a skewed … [Read more…]
These thoughts about plotting were largely inspired by Guillaume Rousselet’s blog. Let’s start with this combined scatter ERP/ERF plot: These are real data from an early paper of mine, picked for convenience. The blue and red lines are the signal recorded by a set of temporal MEG sensors, to a tone click played twice (under two different conditions, called Blue … [Read more…]
I read a few papers and wanted to share my thoughts with the cognitive science community. If phrases like predictive coding, mismatch negativity, or gamma band don’t mean much to you, the rest of the text is likely to be mind-numbingly boring. Mismatch negativity If we hear a train of identical tones, occasionally interrupted by … [Read more…]