ESRA 2025 Preliminary Program

Monday 14 July    Tuesday 15 July    Wednesday 16 July    Thursday 17 July    Friday 18 July   

All time references are in CEST

The use of differential incentives in surveys
Session Organisers	Dr Alessandra Gaia (University College London) Ms Gerry Nicolaas (NatCen)
Time	Thursday 17 July, 13:30 - 15:00
Room	Ruppert Wit - 0.52

There is increasing interest in the use of differential incentives – i.e. offering varying levels or types of incentive to different sample members – to attempt to improve survey response rates, minimise non-response bias, and increase cost-effectiveness. For example, higher value incentives may be targeted (i.e. “targeted incentives”) to specific population subgroups which are typically under-represented or of particular research interest (e.g. low-income respondents) or which are expected to be more responsive to incentives (e.g. in longitudinal studies, prior wave non-respondents). Researchers may also offer higher value conditional monetary incentives for survey participation within a certain timeframe (i.e. “early-bird” incentives) or at the end of fieldwork, as a tool to convert unproductive cases. Interviewers may also be instructed to use incentives “discretionally”, where other persuasion strategies are not effective.
While the promise of differential incentives is to efficiently allocate the survey budget and improve data quality, several aspects remain unresolved, particularly concerning their effectiveness, logistical aspects and ethical issues. These include, for example, considerations on the fairness of offering incentives of different amounts/type for the completion of the same task, presence/absence of a requirement to disclose the differential design to participants, and the risk of identifying/stigmatising sample members as belonging to hard-to-reach groups.
We welcome empirical and theoretical submissions on:
- The effect of differential incentives on response rates, representativeness, item non-response, measurement and other aspects of data quality;
- Applications of differential incentives in cross-sectional studies, where limited information on sample members is available for targeting;
- Costs-effectiveness;
- Ethical issues and considerations around acceptability for sample members, interviewers, funders and ethical bodies on the use of differential incentives;
- Operational and technological barriers in the implementation of differential incentives, especially in mixed-modes contexts.

Keywords: Differential incentives, targeted incentives, representativity, inclusiveness, non-response bias, push to web, longitudinal studies, non-response.

Papers

Differential incentives in a multi-mode feasibility study for a new UK birth cohort

Dr Erica Wong (Centre for Longitudinal Studies, University College London) - Presenting Author
Professor Lisa Calderwood (Centre for Longitudinal Studies, University College London)
Professor Alissa Goodman (Centre for Longitudinal Studies, University College London)
Professor Pasco Fearon (University College London and University of Cambridge)
Dr Alyce Raybould (Centre for Longitudinal Studies, University College London)
Ms Karen Dennison (Centre for Longitudinal Studies, University College London)
Dr Charlotte Booth (Centre for Longitudinal Studies, University College London)

Incentives are often used to boost response rates; differential incentives are less common in the UK than in the US but are increasingly being used to strategically and cost-efficiently target harder-to-reach groups. However, evidence is mixed as to whether incentives increase participation of those typically underrepresented in surveys, such as lower SES families, young people, and ethnic minority groups.
In this paper, we present findings on an incentive experiment conducted in the Early Life Cohort Feasibility Study (ELC-FS) which tested a combination of unconditional and conditional incentives. The ELC-FS collected information about several thousand babies aged between 8-12 months old and their families in 2023-2024 to test the feasibility of conducting a new UK-wide birth cohort study. It is funded by Economic and Social Research Council and led by the Centre for Longitudinal Studies at University College London with interviewing carried out by Ipsos.
Parent interviews of 30-60 minutes in length were carried out using web, phone and video-interviewing as well as face-to-face, depending on the informant type, and mothers and fathers (including those living in their own households) were recruited directly and separately. The survey included a self-completion (CASI) section, data linkage requests, and for a subsample of families, saliva collection. The main survey was followed up by a non-response web survey.
The experiment tested six different incentive conditions, comprising of two different kinds of unconditional incentives (a £5 cash note or baby bib or no incentive) and two levels of conditional incentive (£10 or £20 voucher). An additional incentive was given to those providing a saliva sample, and targeted differential incentives were offered in Scotland and Northern Ireland for completing the non-response survey. We assess the effectiveness of these incentive strategies across countries,

The use of differential incentives in surveys

Dr Alessandra Gaia (Centre for Longitudinal Studies, UCL Social Research Institute, University College London) - Presenting Author
Mr Matt Brown (Centre for Longitudinal Studies, UCL Social Research Institute, University College London)
Professor Lisa Calderwood (Centre for Longitudinal Studies, UCL Social Research Institute, University College London)
Mrs Gerry Nicolaas (NatCen Social Research)
Mr Curtis Jessop (NatCen Social Research)

Targeting higher value incentives at specific population subgroups is a potentially promising approach for improving response rates and efficiently allocating survey budgets and as surveys increasingly rely on the web, with no interviewers to encourage participation, the strategic use of targeted incentives could prove even more useful. Targeted incentives could also be effectively at boosting participation amongst under-represented and harder to reach groups, increasing inclusivity and reducing non-response bias.

Despite their potential, the application of targeted incentives is not well documented and evidence on their effectiveness is limited. Most experimental evidence comes from the US and is often limited to specific cohorts or timeframes. Additionally, most experimental research compares groups of participants approached using a larger incentive budget with a group approached with a less expensive budget and so cannot identify how best to allocate fixed resources. Significant gaps in our knowledge, along with practical and ethical challenges, still need to be addressed.

In this paper we first present initial results from a literature and evidence review on the use of targeted incentives in social surveys, focusing on their application, effectiveness and ethical and logistical barriers to their implementation. Second, we describe the design of an experiment investigating the effectiveness of targeting higher incentives at residents in more deprived areas in Britain. The experiment is implemented in the 2025 British Social Attitudes survey, a mixed-mode push-to-web large-scale repeated cross-sectional general population survey. Finally, we will discuss our plans for producing a code of ethics aimed at understanding alignment with existing ethical standards, supporting cost-benefits evaluations (e.g., equality versus equity), identifying aspects to be brought to the attention of ethical bodies, funders, study participants, and providing practitioners and study's Principal Investigators with a “toolkit” for the ethical implementation of targeted incentives.

Targeted incentives and the National Diabetes Experience Survey: Engaging seldom heard groups living with diabetes

Ms Eileen Irvin (Ipsos) - Presenting Author

The 2024 National Diabetes Experience Survey (NDES) offered a unique opportunity to examine the impact of conditional incentives on survey response rates and data quality. This presentation focuses on the incentive experiment conducted as part of the survey.

The NDES, commissioned by NHS England and administered by Ipsos, gathered data on the experiences of adults (18+) living with type 1 or type 2 diabetes in England for at least 12 months. The survey employed a push-to-web design, incorporating postal mailouts, SMS reminders, online completion, and paper questionnaires to maximize participation. Data was collected between March 18th and July 16th, 2024, and reported at both national and Integrated Care System (ICS) levels.

To investigate the effectiveness of targeted incentives, the NDES offered conditional £5 vouchers to subgroups less likely to respond: younger people, those in deprived areas (based on the Index of Multiple Deprivation - IMD), and those from ethnic minority groups. This experiment aimed to assess the influence of these incentives on response rates and data quality across different demographic groups.

Analysis of the incentive experiment will explore its impact on overall response rates, non-response bias, and potential differences in responses between incentivised and non-incentivised participants. Findings will inform future survey strategies, particularly for engaging seldom heard populations in health research, and contribute to the broader discussion on the ethical and practical considerations of differential incentives in survey research.

Targeting incentives based on geodemographics and participant behaviour: Evidence from a push-to-web survey in the UK

Mr Curtis Jessop (The National Centre for Social Research) - Presenting Author

Higher incentives have been shown to increase survey response rates and potentially improve the representativeness of the participating sample. Where both the marginal impact on response rates and the costs of issuing a larger sample (e.g. due to high postage costs) are high, it can also be relatively cost-effective. However, universally increasing incentives may not be the most efficient use of resources. Instead, targeting higher incentives at those unlikely to participate (and not those willing to participate for a lower incentive) may offer greater value for money. However, in the UK, where the sample is drawn from the Postcode Address File, relatively little is known about the sample, so effective targeting can be difficult.

In 2023 and 2024, the British Social Attitudes study (BSA) tested the impact on participation of offering a conditional £10 or £15 incentive. In-line with previous evidence, the higher incentive resulted in a higher response rate. This paper will report on further analysis, exploring the impact of different incentives in different geographies. It focuses on the impact on areas with lower response rates (e.g. areas with higher levels of deprivation) to provide evidence on how an incentive strategy targeted based on geography may impact overall response rates, sample quality, and cost-effectiveness.

An alternative approach to targeting incentives to an address-based sample is using participant behaviour. In 2024 an additional arm of the BSA experiment initially offered all sampled participants a £10 incentive, and then increased this to £15 if they had not participated after the initial invitation and reminder. We explore the impact of this relative to offering a £10 or £15 incentive on response rates, sample quality, and cost-effectiveness.

These findings will provide evidence on how targeting incentives may improve quality and efficiency in push-to-web surveys