Privacy and Measurement Error in Phone Surveys: The Case of Women’s Agency

Abstract

There is a growing literature focusing on measurement error in self-reported data. This paper studies the case of women’s agency, a key outcome in economic development, and introduces a simple and cost-effective corrective measure to enhance the reliability of data collection around sensitive topics. The study draws from a survey experiment in rural Guatemala where 1,183 women received the same questions about their agency as part of a phone survey, but the privacy of their responses was varied. Women in the control group answered verbally. The first treatment group used code words to conceal answers from potential bystanders. The second treatment arm used the phone keypad to conceal answers from bystanders and the enumerator alike. The paper finds that women answering under higher levels of privacy report lower levels of agency, by 0.19 standard deviations on an aggregate index. The effects of the two privacy treatments cannot be statistically distinguished. The effect is concentrated among women with higher social standing, and qualitative interviews indicate that agency is socially valued in this setting. Both patterns point to social desirability bias as the most likely explanation for why reported agency responds to privacy.