Incentives and Survey Length: Does Offering a Choice Result in Higher Response Rates? (Presentation)

We tested whether offering a choice of completing just the first section of a survey or the full version would raise response for the first section, compared to a request to complete the full survey with no choice. We randomly assigned respondents to 3 experimental groups, varying their response options and associated incentive amounts for survey completion. We found that offering a choice in survey length reduced response rates as compared with no choice offered.

When the beginning of a survey includes priority questions (or a screener), what is the optimal messaging to encourage people to complete the priority section? We tested this question using the 2017 Comprehensive Primary Care Plus (CPC+) Practice Survey. CPC+ is a Centers for Medicare & Medicaid Services program designed to improve primary care delivery. The first survey section (10 items) was critical to selecting the comparison group for the evaluation. Because some respondents have limited time, we tested whether offering a choice of completing the short version (first section) or full version of the survey would raise response for the first section, compared to a request to complete the full survey with no choice. Based on anchoring theory, which suggests the first piece of information is the most critical reference point in decision making, we developed two alternative choice messages, which first emphasized: (1) minimal time burden or (2) a higher incentive. We randomly assigned practice managers in 8,245 practices to three messaging groups:·No choice: “You will receive $100 for completing this 30-minute survey.”·Choice 1: “If you only have 10 minutes available, you can complete the short version of this survey for $20, or you can receive $100 for completing the 30-minute version.”·Choice 2: “You will receive $100 for completing this 30-minute survey, but if you only have 10 minutes available, you can complete the short version for $20.”Results showed that choice reduced response rates for the first section—about 37 percent responded to each choice condition, whereas about 40 percent responded to the no-choice message (p = 0.03) (there was a similar effect for the full survey). According to social exchange theory, people assess the rewards versus costs of survey participation, and choice messaging may have made this assessment more difficult, lowering response.