Incentives in the Ethnic Minority British Election Study and National Travel Attitudes Survey

Part I: Push-to-Web and mixed incentives in the Ethnic Minority British Election Study 2021 Pilot 

The 2021 Census revealed a growing population of people from ethnic minority backgrounds. 

Yet, the largest representative study of political and social attitudes of Britain’s ethnic minority population, the 2010 Ethnic Minority British Election Study (EMBES) – a face-to-face probability study – is over a decade old. 

Building on the successful transition of BES2019/2020 to push-to-web as an immediate response to Covid, the EMBES 2021 Pilot investigated the feasibility of conducting research among this population using a push-to-web methodology.

While push-to-web has been assessed to produce reasonably representative results for studies of general population, we address its suitability for recruiting a new representative sample of ethnic minorities. 

We adapted the British Election Study questionnaire to incorporate an initial screener questionnaire, with different incentives depending on eligibility for the main questionnaire. 

We found that response rates were lower than expected. Leveraging the fact that we used two recruitment approaches, we suggest that this more complicated incentive structure reduced response rates. 

Further analysis however suggests poor response rates were not limited specifically to ethnic minority respondents, or to the questionnaire with the screener. 

This indicates that general population samples recruited through push-to-web methodologies may under-represent geographies where minorities usually concentrate. 

Our results therefore have important lessons not just for ethnic minority surveys, but also generalist surveys which want to make use of the cheaper push-to-web methodology in recruiting a probability sample.

Part II: Every penny counts? Exploring the impact of incentives on response, sample composition, and data quality in the National Travel Attitudes Survey panel

Over the past few decades, surveys have consistently seen declining response rates globally.

The use of incentives has proven to be one of the most effective strategies to counteract this trend. 

Extensive research indicates that incentives have a small to moderate positive effect on response rates and can reduce field efforts, leading to lower survey costs.

However, the impact of incentives varies based on the research design, survey mode, the type and value of the incentive, and the timing of the offer. 

While incentives are known to positively impact response rates, their effect on non-response bias and data quality remains less clear. 

In this study, we present the results of a randomised experiment embedded in wave 8 (2023) of the National Travel Attitudes Survey (NTAS), a mixed-mode panel based on respondents to the English National Travel Survey. 

Sample cases were randomly assigned to receive one of three conditional incentives: £5, £10, or £15 Love2Shop vouchers. Informed by the Total Survey framework, our analysis examines the impact of incentives on three primary outcomes: (1) response (i.e., response rates, share of web responses, and time to completion); (2) non-response bias (comparing the achieved samples with benchmark data); and (3) data quality (including non-substantive responses, item endorsement, validity, internal consistency, and acquiesce). 

The findings indicate that higher-value incentives increased response rates without having a negative impact on sample composition or data quality.