Download data and study materials from OSF
Principal investigator:
Brendan Nyhan
Dartmouth College
Email: nyhan@dartmouth.edu
Homepage: http://www.dartmouth.edu/~nyhan/
Sample size: 514
Field period: 02/2005
Can institutions encourage citizens citizens to consider new factual evidence? Lupia and McCubbins (1998) and Lupia (2004) find that factual claims can be more persuasive if there is a threat that the claims will be verified and penalties for lying assessed. However, this theory has only been tested in laboratory experiments concerning predictions of the outcomes of rolls of a die and a survey experiment focused on predictions of outcomes on a non-salient issue. I extend their concept (a formal result that only applies under certain conditions) to test whether verification threat can increase the persuasiveness of empirical claims in a more general sense.
Statements made under verification threat will induce greater opinion change than otherwise identical statements in less credible contexts. Thus, verification threat will increase the persuasiveness of empirical claims about policy issues and correspondingly increase (or decrease) support for the policies in question.
In the experiment, I assess the effect of verification threat on personal opinions using issue frames that include an empirical claim.
To maximize realism and comparability to previous research, I consider both the context of a single frame and competing frames. The experiment was thus a 2x2 between-subjects design for each of four issues. At the beginning of the experiment, each respondent was assigned to a control group with a 20 percent probability. Controls did not receive any messages and were simply asked for issue opinions. For non-controls, the order of questions was randomly varied to avoid any systematic question order effects and respondents were randomly assigned to one of the four experimental conditions on a question-by-question basis.
Subjects in treatment conditions received prompts on four issues:
Social Security private accounts, limitations on medical malpractice lawsuits, allowing the reimportation of prescription drugs, and withdrawing troops from Iraq. Each prompt begins with a description of a political figure or group advocating a policy based on some empirical rationale. In the one-sided treatments, subjects are told that the rationale was expressed either in a press release (no verification threat) or in congressional testimony (verification threat). In the two-sided treatments, subjects are also told that an opposing partisan or non-partisan expert has expressed a contradictory view in a press release or congressional testimony. The proposer's statement is always made in a press release in the two- sided treatments in order to limit the number of conditions and to generate a contrast with the counter-argument expressed under verification threat.
Subject opinions on the four policy proposals were measured on a four- point Likert scale ranging from strongly disagree to strongly agree.
Across both sets of conditions, we find only one case in which the overall treatment effect is in the predicted direction (medical malpractice, two frames). We also observe movement in the predicted direction for subjects with a partisan predisposition to do so in two other cases (prescription drug, one frame; Social Security, two frames). All other effects are insignificant.
These results indicate that the effect of verification threat on policy opinion about salient political issues is limited and may be concentrated among sympathetic partisans.
The design of these experiments departs from prevailing practice in three important respects. First, the treatments attribute messages to parties or political figures because the media usually do so in their reporting. I also present facts to subjects as contested, which again follows standard practice in news reports. Finally, while Lupia (2004) represented verification threat using court testimony, I instead use congressional testimony, which is much more common in national politics, and state that a political figure "testified before Congress." Finally, due to a typographical error, the two- sided verification threat condition was invalidated for the question about US withdrawal from Iraq (subjects in that condition were dropped from all results).
"The High Price of Constraint? Comparing the relative impact of verification threat and counter-framing in partisan debate.” Paper presented at the annual meeting of the Midwest Political Science Association, Chicago, IL, April 20-23, 2006.