Gender and Non-Attitudes on Foreign Policy
Sample size: 2800
Field period: 06/30/2012-09/28/2012
Public opinion surveys on American foreign policy often elicit many "don't know" or "no opinion" responses. Researchers have paid surprisingly little attention to these responses, typically dropping them from their analyses or suppressing them in the survey design. We argue that these practices lead to potentially misleading conclusions about both the level of support for particular foreign policies and the determinants of individual attitudes (especially gender, education, and party identification). We demonstrate these problems using an original survey experiment testing the effect of including a DKNO option on three common questions about international trade, the use of force, and isolationism. Our findings also suggest that taking DKNO responses more seriously in our analyses provides a richer sense of the process through which important covariates actually influence attitudes.
1. How does the inclusion of DKNO response options affect estimates of the level of support for major foreign policy issues?
2. How does the treatment of DKNO responses affect inferences about the effect of commonly discussed predictors of foreign policy attitudes?
Half of the respondents received an explicit DKNO option in answering several questions about American foreign policy. As the ANES has sometimes done, we added the phrase "or haven't you though much about this issue?" to the end of the question for these respondents. The other half of the sample received no DKNO option and no prompt at the end of the question, which was otherwise identical.
The survey includes five foreign policy questions. These concerned:
1. Increasing trade with other nations
2. Using military force to solver international problems
3. Allowing foreign investors to purchase a controlling share in US companies and banks
4. Increasing the share of the federal budget spent on foreign aid
5. Taking an active part in world affairs
First, we find that the inclusion of the DKNO response option greatly affects estimates of the level of support for major foreign policy issues. There is substantial acquiescence bias when the DKNO option is omitted. This was greatest on the question concerning the use of force, where respondents who would have selected the DKNO response were significantly more likely to express support for the use of force when this response was not available. Second, we find that the inclusion or exclusion of the DKNO prompt substantially affects estimates of the impact of gender, education, and party identification. Both the exclusion of this response category, and also the practice of treating these responses as missing data influence estimates of these effects.
Kleinberg, Katja, and Benjamin Fordham. Don't Know Much about Foreign Policy: Assessing the Impact of "Don't Know" and "No Opinion" Responses on Inferences about Foreign Policy Attitudes. Manuscript under review.