I’ve been reading about the so-called common method bias and how it can taint study results, but so far I have not found an article that explains with any level of clarity why exactly it’s a problem, for example, to ask one person both about an independent variable and a dependent variable in a study.
Can someone explain? Is this a cognitive issue or a mathematical/statistical one? Please explain the nature of the problem in a way that makes intuitive sense. Help!
The way I understand it, this comes about when trying to compare two variables for a possible connection e.g. poverty and drunkedness, fish oil and IQ. . A common bias comes about when using the same instrument/measuring method for both. A classic example is the use of internet surveys. Not only are you pulling in only a small subset of people, people may more likely lie on the internet. Using one interviewer to interview people to assess the connection between intelligence and good looks means the good looks may also affect their judgment on the interviewees intelligence.
You’re making it sound like it’s a problem with the way people respond, but the way it was explained to me, it sounded like the problem was not in the way that people respond, but that the responses were too correlated to be accurate. Or something like that. I don’t really get it.
Another example is poorly constructed questionaires e.g.
oh and here is a list of common biases from http://www.usq.edu.au/users/patrick/PAPERS/Common%20Method%20Variance.pdf
Summary of Potential Sources of Common Method Biases
Common rater effects
Refer to any artifactual covariance between the predictor and criterion variable produced by the fact that the respondent providing the measure of these variables is the same
Consistency motif
Refers to the propensity for respondents to try to maintain consistency in their responses to questions.
Implicit theories (and illusory correlations)
Refer to respondents’ beliefs about the covariation among particular traits, behaviors, and/or outcomes.
Social desirability
Refers to the tendency of some people to respond to items more as a result of their social acceptability than their true feelings.
Leniency biases
Refer to the propensity for respondents to attribute socially desirable traits, attitudes, and/or behaviors to someone they know and like than to someone they dislike.
Acquiescence biases (yea-saying and nay-saying)
Refer to the propensity for respondents to agree (or disagree) with questionnaire items independent of their content.
Mood state (positive or negative affectivity; positive or negative
emotionality)
Refers to the propensity of respondents to view themselves and the world around them in generally negative terms (negative affectivity) or the propensity of respondents to view themselves and the world around them in generally positive terms (positive affectivity).
Transient mood state
Refers to the impact of relatively recent mood-inducing events to influence the manner in which respondents view themselves and the world around them.
Item characteristic effects
Refer to any artifactual covariance that is caused by the influence or interpretation that a respondent might ascribe to an item solely because of specific properties or characteristics the item possesses.
Item social desirability
Refers to the fact that items may be written in such a way as to reflect more socially desirable attitudes, behaviors, or perceptions.
Item demand characteristics
Refer to the fact that items may convey hidden cues as to how to respond to them.
Item ambiguity
Refers to the fact that items that are ambiguous allow respondents to respond to them systematically using their own heuristic or respond to them randomly.
Common scale formats
Refer to artifactual covariation produced by the use of the same scale format (e.g., Likert scales, semantic
differential scales, “faces” scales) on a questionnaire. Common scale anchors
Refer to the repeated use of the same anchor points (e.g., extremely, always, never) on a questionnaire.
Positive and negative item wording
Refers to the fact that the use of positively (negatively) worded items may produce artifactual relationships on the questionnaire.
Item context effects
Refer to any influence or interpretation that a respondent might ascribe to an item solely because of its relation to the other items making up an instrument (Wainer & Kiely, 1987).
Item priming effects
Refer to the fact that the positioning of the predictor (or criterion) variable on the questionnaire can make that variable more salient to the respondent and imply a causal relationship with other variables.
Item embeddedness
Refers to the fact that neutral items embedded in the context of either positively or negatively worded items will take on the evaluative properties of those items.
Context-induced mood
Refers to when the first question (or set of questions) encountered on the questionnaire induces a mood for responding to the remainder of the questionnaire.
Scale length
Refers to the fact that if scales have fewer items, responses to previous items are more likely to be accessible in short-term memory and to be recalled when responding to other items.
Intermixing (or grouping) of items or constructs on the questionnaire
Refers to the fact that items from different constructs that are grouped together may decrease intraconstruct correlations and increase interconstruct correlations.
Measurement context effects
Refer to any artifactual covariation produced from the context in which the measures are obtained.
Predictor and criterion variables measured at the same point in time
Refers to the fact that measures of different constructs measured at the same point in time may produce artifactual covariance independent of the content of the constructs themselves.
Predictor and criterion variables measured in the same location
Refers to the fact that measures of different constructs measured in the same location may produce artifactual covariance independent of the content of the constructs themselves.
Its a bit of both -CMB doesnt always produce a high correlation - it can produce the reverse. The way people respond can be influenced by the test.
E.g in the internet online survey example, if one was testing for links between geekery and porn watching one might find a very high correlation as internet uses tend toward the tech side and have ready access to porn. However if one was to do the test world wide amongst internet and noninternet users, one might find no connection between geeks and porn.
thanks! that paper should prove useful.