.. or put differently: ALWAYS read the original study, and look at the variables, before basing your own research on past findings! Surprisingly often, authors make hefty claims based on their findings.. which may not always have as strong a basis. I have recently found an example of this in my own field, where the dependent variable is named so that it seems a valid and important outcome. However, looking at the instruments which underpin the variable, it becomes clear that 1 in 4 of the questions are basically the same as the independent variables. This is problematic.

I just heard a podcast, at “You are not so smart”, that looks at much of the same: it looks at the “backfire effect”. This finding from political science, basically states that some people can become more entrenched after having been corrected for erroneous beliefs. The finding got a lot of interest, to the extent that Oatmeal made a cartoon showing the effect; http://theoatmeal.com/comics/believe ; and the paper has been cited 800 times since 2010.

The question used in the study has been popularized in the media as: “Do you believe there were Weapons of Mass Destruction in Iraq”; which people who believed it from before, continued saying yes to, despite being corrected. This is an eyeopening effect, and easy to understand.

.

.

However, there is an alternative explanation, namely that the question was so complex that many respondents glazed over, chose not to think about it, and just stuck to what they had answered before. To understand this, lets look at the original question, from the paper:

Study 1 (WMD): Dependent variable
Immediately before the U.S. invasion, Iraq had an active weapons of mass destruction program, the ability to produce these weapons, and large stockpiles of WMD, but Saddam Hussein was able to hide or destroy these weapons right before U.S. forces arrived.
-Strongly disagree [1]
-Somewhat disagree [2]
-Neither agree nor disagree [3]
-Somewhat agree [4]
-Strongly agree [5]
This is clearly a different question that the one popularized, and which spawned a lot of research. Only recently, was a replication study conducted, and failed: The Elusive Backfire Effect: Mass Attitudes’ Steadfast Factual Adherence.
All to say, read and understand the instruments used in research you are basing your own studies on!.
Share This