As a method of collecting data, surveys have been widely criticized for the biases that are included in the data. When the source of biases are the same for both the dependent and the independent variable, (as in, what is used to explain, and what we are seeking to explain) there is the danger of Common Method Bias. The “rater effect” is a major source, such as how one tends to answer a scale, how a word is understood and underlying motives to name a few. Do see Podasakoff for more on common method bias:
Podsakoff, Philip M., Scott B. MacKenzie, Jeong-Yeon Lee, and Nathan P. Podsakoff. “Common method biases in behavioral research: a critical review of the literature and recommended remedies.” Journal of applied psychology88, no. 5 (2003): 879.
Specific cases also come up, such as HR data; as explained in this article. The takeaway is the reference to three studies showing that 71% – 55% of the variance in responses were explained by the individual, rather than what was under study. (The same results showed that what under study represented 20% of the variance; making the survey results all but useless)
Most HR Data Is Bad Data
Create a FREE account to: Get 15 free articles per month* Access to personalized content Save articles and create shareable folders in your personal HBR library Get 20% off your first order using code HBRORGREG3** *not including articles that are exclusively for Harvard Business Review magazine subscribers **does not include