Earlier this month, the New York Times Magazine published “The Brain That Couldn’t Remember,” an excerpt from the upcoming Patient H.M.: A Story of Memory, Madness, and Family Secrets by Luke Dittrich. The article made a splash in the cognitive neuroscience world, at least among those of us who study memory in one form or another. It is highly critical of Dr. Suzanne Corkin, an influential neuropsychologist. The general scientific consensus, as evidenced by a Letter to the Editor and a more detailed response on M.I.T.’s website, seems to be that the article amounted to a sensationalist hit-piece on a recently deceased woman who is a hero to many in the field.
My goal here is not to challenge that characterization as a whole. However, the piece did seem to touch a nerve with people outside of the scientific community and for that reason I think it is worth exploring the merit of some of its claims. I am going to focus on the treatment of the data collected on H.M. over the span of decades. This seems to be the only part of the story backed up with quotations, and I think it has the most relevance to psychology and neuroscience as a whole. Fairness and taste of the Dittrich piece aside, is there anything for psychologists and neuroscientists to learn from it?
From Dr. Corkin herself:
Corkin: Yeah, well, the tests are gone. The test data. The data sheets are gone. Because the stuff is published. Most of it is published. Or a lot of it is published.
Me: But not all of it.
Corkin: Well, the things that aren’t published are, you know, experiments that just didn’t … [another long pause] go right. Didn’t. You know, there was a problem. He had a seizure or something like that.
Corkin: Yeah, but it’s not peer-reviewed, for one thing. That’s important. The stuff that’s published is good stuff. Peer-reviewed. You can believe it. Things that, you know, experiments that might not have been good experiments, there might have been inadequate control groups. … There are all sorts of things that can go wrong with experiments. Not every experiment is publishable.
I think much of the response to this article has missed a major point: the attitude and behavior towards data that are apparent in Dr. Corkin’s quotes have historically been the norm in the field. And we are currently paying for it.
We are living through a period of uncertainty when it comes to interpreting the scientific record. It is becoming clearer and clearer that the traditional ways of analyzing and interpreting data in psychology and neuroscience are not up to the challenges we put in front of them. Some of the more problematic aspects of our current (but hopefully changing!) behavior are proprietary treatment of data, a ritualistic approach to hypothesis testing, and overconfidence in published results, paired with publication bias and post-hoc interpretations.
Back to the NYTM. Suzanne Corkin’s comments make it clear that:
- She felt the data collected from H.M. were hers to share or not as she deemed appropriate
- Not all of the experiments conducted yielded results she saw as worth reporting on
- The published data are in some sense “true-er” than unpublished data
To me, these sound a lot like the problems many of us have with the way psychology has been practiced generally. The data is ours to find the story in, we will report the results our methods declare significant, and the overarching theoretical narrative will come from patterns in the published record. In that sense, her behavior seems no different from what many scientists were trained to do, and she should not be singled out for it. However, I imagine most non-scientists would find this general pattern of scientific reporting more shocking than the idea of an individual neuropsychologist scheming to destroy any data that does not fit her story.
I think we still have a lot of work to do cleaning house in terms of analysis and reporting practices. To that end, we would all do well to acknowledge these issues when they appear, either in our current practices or in the historical record. Even if we disagree vehemently with the person pointing them out.