ESRA 2025 Preliminary Program
All time references are in CEST
Increasing data quality in online surveys via contact and questionnaire design strategies |
| Session Organisers |
Mrs Vanessa Schmieja (Forschungszentrum Jülich)
Mr Hawal Shamon (Forschungszentrum Jülich)
|
| Time | Thursday 17 July, 15:30 - 17:00 |
| Room |
Ruppert 111 |
Online surveys, attractive because they are inexpensive to conduct, are being used by various actors from science, business and civil society by applying different levels of quality standards. The increasing number of requests to participate in online surveys as well as spillover effects from poorly designed online surveys can lead to survey fatigue and decreasing willingness to participate in online surveys among target persons.
The seemingly inflationary number of online surveys may also erode respondents' appreciation of each individual online survey, which is likely to be associated with a lower baseline motivation to optimize during the survey. Higher rates of dropout, item non-response and careless responding may be the consequence. One leverage to overcome the challenge of data quality loss, is to tailor the questionnaire to survey participants’ preferences and needs.
This session focuses on contact and questionnaire design strategies to improve data quality. Contributions may relate, but are not limited, to innovative approaches to increase
o survey participation rates in cross-sectional and longitudinal online surveys (e. g. incentives that go beyond monetary rewards, adaptive contact strategies),
o survey participants’ motivation, attention and satisfaction with online surveys (e. g. use of gamification or multimedia content, diversified design of the survey, personalized online surveys).
Keywords: online surveys, data quality, questionnaire design
Papers
Optimising Online Time Use Surveys: Balancing Quality, Efficiency, and Inclusivity
Ms Marta Mezzanzanica (The National Centre for Social Research)
Mr Chujan Sivathasan (The National Centre for Social Research)
Mr Curtis Jessop (The National Centre for Social Research) - Presenting Author
Time-use surveys provide invaluable insights into how individuals allocate their time across various activities. Traditionally, paper-based diaries, often administered by face-to-face interviewers, have been the primary method for gathering such data in the UK. More recently, advances in technology have enabled the development of online diary tools, but while these offer advantages like reduced costs, cleaner data, and increased flexibility, they also raise new challenges related to data quality and inclusivity.
This study assesses the impact of design choices on the quality and efficiency of online time-use surveys in two UK time use studies, and analyses eight waves of time-use surveys conducted in the UK between 2020 and 2023 using sample from the NatCen's probability-based panel.
A desk review of the online diary tools, participant materials, and analysis of available data (response rates, sample profiles, split-sample experiments, paradata, and respondent feedback) was conducted to assess the performance of the fieldwork design and diary tools, and evaluate the respondent journey. By examining factors such as incentivisation, completion mode, use of support materials, and user interface, we identify strategies for minimising respondent burden while maximising data quality and response rates to help inform future designs.
EU survey on gender gaps in unpaid care, individual and social activities
Mr Vytautas Peciukonis (European Institute for Gender Equality)
Dr Jolanta Reingardė (European Institute for Gender Equality)
Dr Irene Rioboo Leston (European Institute for Gender Equality) - Presenting Author
In 2022, EIGE conducted an EU-wide online survey to analyse women’s and men’s engagement in unpaid care, individual, and social activities, incorporating gender equality and intersectional perspectives.
The survey aims to support EU and national policy development and monitor gender equality through EIGE’s Gender Equality Index. Its results will contribute to tracking progress on political commitments in the Council Recommendations on the new Barcelona targets for early childhood education and care and access to affordable, high-quality long-term care, adopted on 8 December 2022. It also informs the EU Gender Equality Strategy 2020–2025 and the European Care Strategy, where closing the gender care gap is a core objective.
Progress towards gender equality is hindered by the unequal distribution of caregiving responsibilities, particularly in an ageing population. Prior to this survey, EU data on unpaid care lacked gender sensitivity, depth, recency, and comparability across Member States.
EIGE’s survey addresses these gaps through a comprehensive questionnaire covering informal long-term care, childcare, household tasks, leisure, and volunteering. Additional sections explored topics like transport use, digital tools, and eco-friendly behaviors in informal care. Background data on individual and household characteristics and attitudes toward gender roles was also collected.
The survey included over 60,000 respondents across the EU-27. Most data were collected via CAWIs, except in Luxembourg and Malta, where CATIs were used due to limited online access panels. Respondents were aged 16–74, with exceptions in Luxembourg (16+), Malta (16+), and Romania (16–64). Fieldwork occurred from August to November 2022.
In 2024, EIGE run a second wave of the survey, maintaining the methodology and gathering over 60,000 responses. Fieldwork took place between October and December, with initial results expected in 2025.
This presentation outlines the survey methodology and highlights.
The effect of smartphone first questionnaire design on data quality
Ms Deirdre Giesen (Statistics Netherlands) - Presenting Author
Dr Maaike Kompier (Statistics Netherlands)
Professor Jan van den Brakel (Statistics Netherlands/Maastricht University)
Online household surveys nowadays are mixed device surveys. At the end of 2024, over 45% of first logins in Statistics Netherlands’ household surveys used a smartphone. This means that the questionnaire design should ensure user-friendliness across screens sizes and input modes. Furthermore, to maintain comparability across respondents and over time, it is crucial that respondents experience the same questionnaires consistently, regardless of the device used.
In response to these requirements, along with considerations related to accessibility and corporate identity, Statistics Netherlands has developed a new questionnaire stylesheet with a “smartphone first” approach.
In 2023 a large-scaled field experiment was conducted with a sample of 12060 persons of the general Dutch population aged 16 and older. Sample units were randomly assigned to either the current stylesheet (n=2824, 40% of the sample) or the experimental smartphone first stylesheet (n=7236, 60% of the sample). Within the experimental stylesheet also other design elements were tested: various grid designs, the use of smileys/icons and the use of a help text encouraging respondents to use speech-to-text entry for open questions on smartphones.
In this presentation we examine how both the device type and the various questionnaire characteristics affect data quality. Our analysis controls for respondent background characteristics, survey attitude, and reported features of the questionnaire completion process (e.g. concurrent activities, location and presence of others).
