All time references are in CEST
Increasing data quality in online surveys via contact and questionnaire design strategies |
|
| Session Organisers | Mrs Vanessa Schmieja (Forschungszentrum Jülich) Mr Hawal Shamon (Forschungszentrum Jülich) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
Online surveys, attractive because they are inexpensive to conduct, are being used by various actors from science, business and civil society by applying different levels of quality standards. The increasing number of requests to participate in online surveys as well as spillover effects from poorly designed online surveys can lead to survey fatigue and decreasing willingness to participate in online surveys among target persons.
The seemingly inflationary number of online surveys may also erode respondents' appreciation of each individual online survey, which is likely to be associated with a lower baseline motivation to optimize during the survey. Higher rates of dropout, item non-response and careless responding may be the consequence. One leverage to overcome the challenge of data quality loss, is to tailor the questionnaire to survey participants’ preferences and needs.
This session focuses on contact and questionnaire design strategies to improve data quality. Contributions may relate, but are not limited, to innovative approaches to increase
o survey participation rates in cross-sectional and longitudinal online surveys (e. g. incentives that go beyond monetary rewards, adaptive contact strategies),
o survey participants’ motivation, attention and satisfaction with online surveys (e. g. use of gamification or multimedia content, diversified design of the survey, personalized online surveys).
Keywords: online surveys, data quality, questionnaire design
Ms Marta Mezzanzanica (The National Centre for Social Research)
Mr Chujan Sivathasan (The National Centre for Social Research)
Mr Curtis Jessop (The National Centre for Social Research) - Presenting Author
Time-use surveys provide invaluable insights into how individuals allocate their time across various activities. Traditionally, paper-based diaries, often administered by face-to-face interviewers, have been the primary method for gathering such data in the UK. More recently, advances in technology have enabled the development of online diary tools, but while these offer advantages like reduced costs, cleaner data, and increased flexibility, they also raise new challenges related to data quality and inclusivity.
This study assesses the impact of design choices on the quality and efficiency of online time-use surveys in two UK time use studies, and analyses eight waves of time-use surveys conducted in the UK between 2020 and 2023 using sample from the NatCen's probability-based panel.
A desk review of the online diary tools, participant materials, and analysis of available data (response rates, sample profiles, split-sample experiments, paradata, and respondent feedback) was conducted to assess the performance of the fieldwork design and diary tools, and evaluate the respondent journey. By examining factors such as incentivisation, completion mode, use of support materials, and user interface, we identify strategies for minimising respondent burden while maximising data quality and response rates to help inform future designs.
Ms Deirdre Giesen (Statistics Netherlands) - Presenting Author
Dr Maaike Kompier (Statistics Netherlands)
Professor Jan van den Brakel (Statistics Netherlands/Maastricht University)
Online household surveys nowadays are mixed device surveys. At the end of 2024, over 45% of first logins in Statistics Netherlands’ household surveys used a smartphone. This means that the questionnaire design should ensure user-friendliness across screens sizes and input modes. Furthermore, to maintain comparability across respondents and over time, it is crucial that respondents experience the same questionnaires consistently, regardless of the device used.
In response to these requirements, along with considerations related to accessibility and corporate identity, Statistics Netherlands has developed a new questionnaire stylesheet with a “smartphone first” approach.
In 2023 a large-scaled field experiment was conducted with a sample of 12060 persons of the general Dutch population aged 16 and older. Sample units were randomly assigned to either the current stylesheet (n=2824, 40% of the sample) or the experimental smartphone first stylesheet (n=7236, 60% of the sample). Within the experimental stylesheet also other design elements were tested: various grid designs, the use of smileys/icons and the use of a help text encouraging respondents to use speech-to-text entry for open questions on smartphones.
In this presentation we examine how both the device type and the various questionnaire characteristics affect data quality. Our analysis controls for respondent background characteristics, survey attitude, and reported features of the questionnaire completion process (e.g. concurrent activities, location and presence of others).