Dealing with measurement error in list experiments: Choosing the right control list design

• List experiments are widely used in the social sciences to elicit truthful responses to sensitive questions. Yet, the research design commonly suffers from the problem of measurement error in the form of non-strategic respondent error, where some inattentive participants might provide random responses. This type of error can result in severely biased estimates. A recently proposed solution is the use of a necessarily false placebo item to equalize the length of the treatment and control lists in order to alleviate concerns about respondent error. In this paper we show theoretically that placebo items do not in general eliminate bias caused by non-strategic respondent error. We introduce a new option, the mixed control list, and show how researchers can choose between different control list designs to minimize the problems caused by inattentive respondents. We provide researchers with practical guidance to think carefully about the bias that inattentive respondents might cause in a given application of the list experiment. We also report results from a large novel list experiment fielded to over 4900 respondents, specifically designed to illustrate our theoretical argument and recommendations. © The Author(s) 2021.

Subject headings

SAMHÄLLSVETENSKAP  -- Statsvetenskap (hsv//swe)

SOCIAL SCIENCES  -- Political Science (hsv//eng)

Keyword

control list design

List experiment

non-strategic respondent error

placebo item

sensitive questions

Publication and Content Type

ref (subject category)

art (subject category)