Skip to 0 minutes and 0 secondsOpenness, integrity, transparency are really at the heart of Merton's articulation. And as you read this, you'll see that the free communication of findings and the sharing of data and the ability to verify findings is absolutely central to the ethos of science. So the question is, how close are we to these ideal standards. What I'm going to go through next is a couple of real world cases and data that suggest we're pretty far from these ideals right now, and then we'll think about how we can sort of maybe get there if we want to. So the first piece is Anderson, and it's actually a nice short article.
Skip to 0 minutes and 36 secondsIt's really just based on some survey data of US researchers to try to understand how strongly they identify with the Mertonian norms, A. B, how close their own behavior is to the norms– self-assessed behavior. And third, what they think of others in their field. Those are the three things that Anderson et al look at. It's based on a nationally representative sample of US researchers funded by NIH– so, National Institutes of Health. They fund a lot of biomedical research and they fund a lot of social scientists actually as well. They fund lab scientists, they fund people who do field work. So there's a broad range of researchers.
Skip to 1 minute and 17 secondsOf course not in even most research fields, but in a pretty broad range of research fields here. So they have a few thousand researchers. There's two different groups of people in the sample. The first are what Anderson et al call mid-career folks– people who got their first RO1 grant. So for people outside of the health field, the RO1 is like the standard five-year research grant, typically five-year research grant that you would get to really sustain a lab for a certain period of time, but it's really a mark of being an established researcher.
Skip to 1 minute and 46 secondsSo these would be assistant professors or associate professors getting their first RO1, which sort of means like, okay, you've kind of made it, you can sustain a lab for five years, or a research group for five years. I am in this group. I got an RO1 in 2001. I don't remember filling in this survey. I don't know if that's because I have a bad memory, which I probably do, or because I didn't fill it in or something. But it was interesting to me to read this paper and think maybe I'm one of these people. The early career folks are people who receive post-doctoral training grants in the same period, so they're earlier in the career cycle.
Skip to 2 minutes and 21 secondsSeveral years earlier, maybe five years earlier, seven years earlier than the mid-career folks, but these are basically relatively young scholars at the time somewhere in their 20s to 40s on average and there'll be some range. Some of the post-docs would be on the younger side, just to give you a sense. The response rate was only 50 percent, so it's not ideal. So let's just look at this as illustrative of some broad patterns and not really worry about selection bias. We could get into all that, but let's just enjoy these patterns for what they are.
Skip to 2 minutes and 54 secondsThey ran a survey where they asked explicitly about Merton's four norms, but then also counter-norms, which people have identified as also existing in the research sphere. On the one hand you have Universalism. The other is Particularism, a very specific view of right and wrong, a sort of lack of maybe openness to different types of people or researchers. Communality versus Secrecy– we already talked about that. How strongly do you believe you should be sharing all your data and sharing all your findings versus keeping things secretive in a maybe self-promoting way. Disinterestedness versus Self-Interestedness. Would you sort of not report something if it went against your interests?
Skip to 3 minutes and 41 secondsOrganized Skepticism versus Organized Dogmatism. So are you really evaluating claims critically or are you kind of attached to some claims? So for those of us in economics, we know there are certain folks who are very attached to certain types of classic theory, say, or maybe attached to certain forms of behavioral theory, or whatever it is, who don't really welcome a direct attack on their theory.
Skip to 4 minutes and 6 secondsThere are a lot of researchers, we fear, maybe– we'll see the survey evidence – who would welcome a kind of bad empirical study that confirms their priors, to a bad empirical study that goes against them, which they savage. Again, that's the difference between Organized Skepticism and Dogmatism. They also add two other norms, which between Merton in the 40s and today, people have identified a number of researchers as being central to the scientific ethos. One is this notion of Self-Governance. The notion that scientists and the scientific community and peers should really be the ones governing the direction of science, that it's the community that leads the effort versus some sort of administration. It gets a little bit to Anne's point.
Skip to 4 minutes and 49 secondsShould we be making the decisions ourselves about what should be funded, what the key issues are in our field? Or should a government official or a Gates Foundation official be telling us what to study. This goes against the ethos. And then this last one I'm not as sure about, but it's listed and it's in the data so I'll mention it, which is quality of research versus quantity. Should we care about making some fundamental contribution? Some very high-quality insight? Or just producing a lot of papers? So these are the six norms and the six counter-norms. This is how Anderson puts them together.
Skip to 5 minutes and 25 secondsI've already gone through these, but the question is, how much weight, how much support do you have for these versus these? And to the point of the question that was just asked, what they're going to look at next is sort of the difference between your answers to these– like how different they are – and that's how they're going to judge whether you're more in this camp or the other. They're going to try to classify people as being strongly in this camp, strongly in this camp, and there's going to be a third group that's roughly equal– people that on the scale of 0, 1 and 2 put a 1 on both.
Skip to 5 minutes and 51 seconds"Yeah, I believe in openness, but I think sometimes secrecy is okay." That person would be in that middle group. So there are three things again. My own belief – sorry, my own values, my subscription, how much I subscribe to these values in the abstract, what I do and what I think others do. Let me explain this to you. This is for the mid-career and early-career scholars funded by NIH. So, from a lot of the fields of people represented in this room, this grey means I strongly agree with the norm, the openness, communality, those norms. The checkered one here means I have sort of equal weight on the norm and the counter-norm.
Skip to 6 minutes and 34 secondsAnd the black is I generally agree with the counter-norm. So in terms of what researchers say, researchers who've never read Merton, like none of us had read Merton, when you just ask them these kind of classic questions, almost all researchers, active researchers, will say, yeah, I believe in openness and sharing and all these universal values of the scientific enterprise, yes. I do believe in that. About 90 percent agree and then another 7 or 8 percent sort of have mixed views. Very few people openly will just consistently be like, "No, I believe in secrecy. I'm totally self-interested in my research. I'm totally dogmatic."
Skip to 7 minutes and 13 secondsSo 90 percent. That's what people say. What do they do? Or what do they say they do? This is what they say they do. In terms of what they say they do, about 70 percent still say, "Yes, I generally live up to these ideals." And for the mid-career, maybe a bit more than the early career." I don't know if they've learned to be dishonest. I don't know if they're more secure, so they somehow feel like they can be more open about their research or who knows what it is. A lot of people are in this sort of morally ambiguous– or sort of, let's say, ethos-wise ambiguous area, and say, "You know what?
Skip to 7 minutes and 50 secondsSometimes I really do need to be secretive or self-interested." But still, you might look at this and say, well, two-thirds, 70 percent, are sort of conforming with the norm. The most interesting plot here is researchers' beliefs about other researchers. So that's the last one. Here, in terms of researchers' beliefs about other researchers, only about 10 percent actually believe other researchers tend to follow the norms, and then a small number are mixed. Most researchers think the vast majority of other researchers, and especially the early-career ones, almost 80 percent, do not follow these norms, in general. This is pretty striking.
Skip to 8 minutes and 35 secondsThe punch line from this figure, as Anderson et al interpret it is there really appears to be what they call Normative Dissonance. People believe in certain norms but they really don't feel like their fields are living up to these norms, and it's widespread. The majority of scholars believe others are not living up to the norms. You might say, well, look, maybe the survey questions aren't that reliable. Maybe everybody hears about one or two cases of bad apples and so they sort of condemn the field, but I think the numbers are pretty striking.
Skip to 9 minutes and 7 secondsThe other thing we've learned, if we've learned anything about original data collection and development economics the last 15 years is when you want to ask a sensitive question, you don't ask people what they do, you ask them what they think people like them do, and that's how you're going to get a more convincing answer. In fact, there's a whole branch of literature, of research, on corruption, that is literally based on that, saying "I know you don't break the law, you would never pay a bribe, but a firm exactly like your firm in this town – not yours – but exactly like yours – how many bribes did they pay in the last month?"
Skip to 9 minutes and 43 secondsPeople find that data much more convincing than, "Did you pay a bribe?" You might actually think that the most reliable measure of norm following is that bottom panel, if we're a bit cynical.
Skip to 9 minutes and 57 secondsWhat were some of the breakdown in patterns? We talked about field. There is some interesting breakdown. They have researchers that are at private for-profit firms who get NIH grants, and researchers in academic or non-profit institutions. And they don't give all the raw data here, they just sort of plot that there's a significant difference, so there are some limitations to this study. But they find there's significantly more norm-following, stated norm-following among the academic researchers than the researchers at the for-profit firms. Maybe that's not surprising. For-profit firms are often developing new technologies that they want to patent. There's proprietary data that they're actually hired to generate. So it may not be surprising but it's a striking difference.
Skip to 10 minutes and 41 secondsAlso, they asked a bunch of questions about how competitive people find their fields. In fields where people feel like their field is more competitive they're less likely to conform to the norms. There was another pattern, I don't know what to make of this. Among those getting NIH grants, those from non-US institutions are more likely to state support for the counter-norms. I don't know what to make of this at all but that's what they say. There's slightly higher stated subscription to the norms among female researchers but it isn't very large.
Skip to 11 minutes and 16 secondsSo there are a couple of patterns in the data but it's a short article, it would be great to get the raw data here and try to figure out exactly what's going on. Their bottom line was a bit depressing. They said there's this "persistent mismatch" between what researchers want to be doing and what they're actually doing, it's going to create all this stress, it's going to create this terrible work environment. And maybe people will reduce the dissonance by just conforming with the counter-norms. That's sort of one possibility.
Skip to 11 minutes and 45 secondsIt just raises this question: Is there a way to bring scientific practice back to the norms? If we think that openness and sharing and sort of this communal effort is actually useful. Now, from the point of view of economics, you might say maybe there's something good about all this competition. Maybe the drive
Skip to 12 minutes and 6 secondsto be first and be secretive and whatnot, that sort of self-interested drive causes people to work harder and generate more good science. That isn't sort of in the Mertonian framework but it certainly would be a kind of standard view, an incentive theory. So I don't know, these are a set of ideas one could imagine doing some work on as well.
How closely do we conform to Merton's norms?
Merton’s norms speak to an idealized scientific ethos whereby researchers prioritize openness, integrity, and transparency. But how closely do researchers really conform to Merton’s norms? To find out, Dr. Melissa Anderson and her colleagues, in a study funded by the National Institutes of Health (NIH), surveyed early- and mid-career researchers on (1) their level of agreement with Merton’s norms, (2) the degree to which they followed those norms themselves, and (3) their perception of other scientists’ conformity to the norms. The study revealed a striking cognitive dissonance within the scientific community.
If you have permission to read the original article (through your academic institution or journal subscription), it can be found here. If you don’t the following articles are good alternatives:
“Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling.” by Leslie K. John, et al. This article can be found at https://www.cmu.edu/dietrich/sds/docs/loewenstein/MeasPrevalQuestTruthTelling.pdf.
“How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data” by Daniele Fanelli This article can be found at http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005738.
© Center for Effective Global Action