The social side of psychology
I’ve been talking about the replication crisis in pretty abstract terms, but it’s important to remember that there are people behind all the theories, hypothesis testing, and failed replications. For them, this crisis involves their passions, their friends, and maybe even their livelihood. When you’re attuned to the human side of the replication crisis, the whole thing can start to look like a big turf war (if you think I’m exaggerating– a past president of the APS accused the replicators of methodological terrorism). Yikes. The debates are no longer about which theories are true– they’re about proving the other team wrong. At least that’s one, fairly cynical, way to see social side of the replication crisis. Another perspective, given to us by the insightful Helen Longino, sees this social side of science as the very thing that insures science’s claim to truth.
What’s the social side of science?
Imagine you’re a researcher and someone in your lab noticed they just couldn’t bring themselves to do the dishes after a long day of grading papers. You’ve got some extra research funds (this is how you know this is a fictional account), so you study whether exercising self-control in one context leads people to have less of it in other areas of their life. After some fiddling around with the experiment, that’s what you find! Tada, now you’ve got the ego-depletion theory of self-control. Except, not really. In order for the idea of ego-depletion to have any influence, it has to be presented at conferences, submitted to peer-review, published, critiqued, repeated, taught, etc. That whole vetting process is thoroughly social. And when a group of researchers gets together, reanalyzes all that published data, and tries to repeat an experiment without success, well that’s social too.
For all the pristine and objective knowledge that we applaud science for delivering, the whole process depends on social interactions. The social construction of science has been recognized by philosophers for quite some time and there are basically two takes on it.
One reaction is– So what? Of course the whole thing is social, but that doesn’t matter. The special part about science is that it all comes down to evidence. The vast majority of practicing scientists fall into this camp. If you talk to them about this social stuff at conferences, their eyes glaze over and they quickly remember that they were supposed to be somewhere else.
The other reaction is– Aha! We knew that science wasn’t special– what really matters is who holds the power and what sorts of theories benefit them. These scholars see the social side of science as proof that scientific claims to truth are no different than any other group activity that arrives at a consensus. You might argue with them, “well, what about evidence?” But they’ll respond, “Evidence, shmevidence, even that has to be socially agreed upon.” And they’re right– the standards of what counts as appropriate evidence are indeed established through social processes (especially when you can’t directly point at things like self-control or the Higgs bosons). Taken to this extreme, science become little more than a big word game played by the world’s weirdest club.
In the 90s, these two opposing views clashed in a public argument that became known as the science wars. Sexy right? While the hot war petered out by 2000, the conflict never quite resolved. Anytime you hear someone griping about postmodernism or scientism, it’s evidence of the cold science wars still raging beneath the surface.
Ok, but who’s right?
Well, both of them.
The science wars could’ve been avoided if these folks had just read Longino’s work. To vastly simplify her argument– the fact that science is the product of people working together is the very thing that lets us know it’s trustworthy. Rather than undermining science, its social dimension can give us good reason to trust it.
The science wars were fought over the wrong question. Instead of asking whether or not the socialness of science means it’s all hogwash, we should be asking about the different characteristics of this social process. In particular– what are the characteristics that make scientific knowledge trustworthy or not?
For Longino, trustworthy science is that which is open to transformative criticism. This is criticism that doesn’t just challenge a particular theory or hypothesis, it also has the power to change the way personal or social values may have shaped inquiry. These values are the key thing that skeptics appeal to as distorting the objectivity of science. Getting into the role of values in science is way beyond the scope here (check out the SEP entry if you’re curious). The point is that there are certain features of the social process of science that help to guarantee this type of criticism.
In particular, trustworthy science must have:
- Evidence that is open and available to anyone interested
- Recognized avenues for criticism
- Shared standards that critics can invoke in their debates
- A community of researchers that responds to such criticism
- Intellectual authority be shared among qualified researchers
If the social processes of a research program meet these criteria, then by Longino’s account we have good reason to trust the knowledge it produces. Put more strongly, these social features of inquiry are the very things that underwrite science’s claim to objectivity.
What does all this have to do with the replication crisis?
Remember the overriding question here– can we trust psychology? As I noted at the beginning, there’s a roiling social side to the replication crisis. This social dimension can make the replication crisis seem like a big melee. But, even if there’s a bunch of intergroup conflict, these skirmishes don’t undermine the reliability of psychology unless they undermine these criteria. So, let’s see if any of the five points are radically changed or abandoned.
Starting with the availability of evidence– one of the most apparent outcomes of the replication crisis is a push towards open science, with public datasets and transparent analyses. The internet has made this possible in a way that was unimaginable twenty years ago. If anything, this standard is getting better.
How about the shared avenues for criticism? This is one of the biggest points of contention within the replication crisis. Traditionally these shared avenues would have been peer-reviewed journals or conferences. Now though, the debates are playing out on social media, blogs, and pre-prints. Some argue that these are inappropriate forums because they’re unmoderated by peer-review. Others argue that they’re necessary because traditional avenues, like journals, were being used to thwart disagreements.
Where this particular debate will land is an open question, but it’s hard to argue that blogs are irrelevant forums when they’re full of sound critiques. Either way, one consequence that seems to be emerging from this shift is that researchers are assessing quality by paying attention to other standards (e.g., was it pre-registered, how sound are the analyses, what are the reactions of other colleagues, etc.) instead of solely relying on the reputation of journals or peer-review as assurances of sound research. Clearly neither of those prevented the replication crisis in the first place.
Let’s move to Longino’s third criterion for trustworthiness– does the community of psychologists still have shared standards by which to weigh arguments? This is the less contentious, but more fundamental, shift within the replication crisis. Prior to the replication crisis, the magical statistical threshold for getting a study published was to get your p-value less than .05 (this is an indicator that your finding wasn’t just the result of chance). While this form of statistical test (null hypothesis significance testing) is still prevalent, more and more researchers are urging that statistical standards become much more rigorous, with many arguing that the tests for statistical significance, which used to be the standard, be abandoned. In short, the standards that researchers can invoke are shifting, but they are changing to become more rigorous not less. While this change in standards may pull the rug out from past research, there’s wide agreement (indeed 800 prominent psychologists endorse these changes) about these changes.
This fairly widespread agreement on shifting standards is a good sign that the fourth marker of trustworthy science is being upheld. The community of researchers is clearly responsive. Pre-registration is becoming more of a norm. Journal standards are shifting. And at conferences, replication is a prominent topic. Clearly the discipline of psychology is taking the replication crisis seriously and is attempting to respond. The open question here is whether this response will really address some of the fundamental problems underlying the replication crisis (more on that next time).
Finally, in order for the social processes of science to ensure trustworthiness, intellectual authority must be shared among qualified researchers. This criterion is linked to the second one (shared avenues of criticism) because both involve a shifting sense of authority. Traditionally, intellectual authority was roughly indexed by professional status– a tenured professor with a long track record of publishing on a topic had more of it. Now, it doesn’t matter if you’re a graduate student with no publications or a full professor with hundreds of citations, intellectual authority goes to the researcher with the better experimental and statistical skills. At least that’s the aim.
Much like the shifting standards for what are viable forums for critique, this dismantling of our traditional sense of authority is a slow process that will probably continue to meet with resistance. And perhaps that resistance is warranted– as I noted in the previous post, throwing out entire theories for one failed replication is foolish. But, so is preserving authority that came from building research programs on sand. From the perspective of Longino’s criteria, these pushes towards the democratization of science will only help to ensure its trustworthiness in the long-run.
So, can we still trust psychology? Watching all of the squabbling about replication can lead to the impression that the answer is a resounding no. But, pause and focus in on the characteristics of that social messiness that Longino highlights. You’ll see that these conflicts are how this community is restructuring itself in response to the new modes of interaction that the internet allows and the new insights that statistics provides. While there’s plenty of disagreement about how this restructuring should be done, the whole reason they’re arguing is to ensure that it’s done in a way that preserves the integrity of psychology. In other words, all that social messiness is the very thing that is ensuring that we can trust psychology.
Leave a Reply