ESRA 2025 Preliminary Program

Monday 14 July    Tuesday 15 July    Wednesday 16 July    Thursday 17 July    Friday 18 July   

All time references are in CEST

Adapting survey mode in a changing survey landscape: Experiences from repeat cross-national, cross-sectional, and general social surveys
Session Organisers	Dr Gijs van Houten (Eurofound) Dr René Bautista (NORC at the University of Chicago) Professor Rory Fitzgerald (ESS HQ; City St Georges, University of London) Mr Tim Hanson (ESS HQ; City St Georges, University of London) Mr Nathan Reece (ESS HQ; City St Georges, University of London) Ms Daphne Ahrendt (Eurofound) Ms Jodie Smylie (NORC at the University of Chicago)
Time	Tuesday 15 July, 13:30 - 15:00
Room	Ruppert Blauw - 0.53

Studies to measure attitudes, opinions, and behaviors are critical to understanding societies around the world. In the face of social developments, changing trends in respondent recruitment methods, budget constraints, national infrastructure disruptions, and public health concerns, many repeat cross-sectional social surveys are experimenting with self-completion and mixed-mode approaches. The European Social Survey (launched 2001), United States’ General Social Survey (launched 1972), and the European Quality of Life Surveys (launched 2003) are examples of longstanding studies collecting data to inform research on changes over time and now exploring and transitioning to new modes. This session brings together cross-sectional social surveys to share experiences in survey mode transition.

The session's aims include: (1) Share results and lessons from recent mode experiments and mixed-mode applications by general social studies, and potential ways to improve methods. (2) Highlight how different cross-sectional studies have recently modified survey protocols to adapt to changing public conditions. (3) Provide space for data creators, data users, and survey practitioners to discuss methodological and statistical challenges for cross-sectional studies considering such moves. (4) Discuss integrity and comparability of data collected using new data collection methods with the existing time-series. (5) Explore applications of emergent technologies to new modes.

We invite submissions from those involved in transitioning repeat, cross-sectional, and cross-national social surveys to new data collection approaches. Topics of interest include: results from pilots or feasibility studies based on self-completion or mixed-mode approaches; findings from experimental research testing aspects of self-completion/mixed-mode designs (e.g., incentive and mailing strategies, survey length adaptations, sequential vs. concurrent designs); impacts of mode switches on measurement and survey time series; and discussions of experiences and challenges with adapting cross-sectional surveys to new modes across different cultural/national contexts.

Keywords: general surveys, survey methodology, data collection, data collection modes, mixed mode, self administration

Papers

The effect of questionnaire duration on break-off rates: findings from the 2024 European Working Conditions Survey

Miss Hajar GAD (Verian) - Presenting Author
Mrs Tanja Kimova (Verian)
Mr Gijs van Houten (Eurofound)

The European Foundations for the Improvement of Living and Working Conditions (Eurofound) entrusted Verian with conducting the eighth edition of the European Working Conditions Survey (EWCS) in Spring 2024. To future-proof its surveys, Eurofound conducted the EWCS 2024 as a parallel run study, combining face-to-face and online data collection in all 29 European countries, using telephone and postal push-to-web recruitment methods.
The online component featured various test, including questionnaire duration. Respondents were randomly assigned to one of two conditions: a full questionnaire (40–45 minutes) or an abridged version (20–25 minutes). To maximize completions of the full questionnaire, those assigned to the abridged version (M1) were invited to complete an additional module (M2), comprising the questions omitted from the full questionnaire version.
When implementing a self-completion survey without an interviewer to ensure respondents complete the survey in one sitting, it is crucial to understand how questionnaire duration affects break-off rates and identify specific questions where respondents are more likely to drop. For this analysis, we define a respondent as having "started the questionnaire" if they completed the screener and the first two substantive questions. Break-off rates refer to respondents who began the survey but stopped partway through, rather than those who failed to engage at all.
To investigate the impact of questionnaire length on break-off rates, we will compare rates in the full-length online questionnaire to those in the M1 questionnaire (excluding break-offs from respondents who started the M2 questionnaire). Additionally, we will analyze break-off rates within modules of the short questionnaire. We anticipate that respondents who opt to proceed to M2 are less likely to break off compared to those completing the equivalent section in the full-length questionnaire.
This modular comparison will provide insights into

The effect of questionnaire duration on yield, sample composition and data quality: findings from the 2024 European Working Conditions Survey

Dr Gijs van Houten (Eurofound) - Presenting Author
Dr Tanja Kimova (Verian)
Ms Hajar Gad (Verian)
Mr Christopher White (Eurofound)
Mr Jamie Burnett (Verian)

The European Foundation for the Improvement of Living and Working Conditions (Eurofound) commissioned Verian to conduct the eighth edition of the European Working Conditions Survey (EWCS) in Spring 2024 in 37 countries. As part of Eurofound’s strategy for future-proofing its surveys, the EWCS 2024 was conducted both face-to-face and online in all EU Member States (using a telephone push-to-web or a postal push-to-web approach), and the implementation of the online component included a range of test elements.
A key test element is the duration of the online questionnaire. The questionnaire for the face-to-face segment of the EWCS takes around 45 minutes to complete. Ideally, respondents completing the survey online would complete the same questionnaire. However, there are concerns about the impact of questionnaire duration on survey yield, sample performance, and data quality.
To assess these impacts, respondents in the online segment of the survey were randomly allocated to one of two conditions in terms of questionnaire duration: full (ca 40-45 minutes) or abridged (ca 25-30 minutes). The sample split is informed by the results of a pilot test carried out in all countries, with a view to maximise the likelihood of achieving an even split in the net sample.
In this paper we will discuss the results of this test, assessing the differences between the questionnaire duration conditions in terms of yield, sample composition, and data quality. The pilot test suggested a negative impact of questionnaire duration on yield in most countries. Analysis of the mainstage data will allow assessing this overall reduction of yield, and the impact on the cost per item, against the impact on response bias and data quality.

Examining Differences in Face-to-Face and Self-Administered Mixed-Mode Surveys: Insights from a General Social Survey

Mrs Alexandra Asimov (GESIS) - Presenting Author

General social surveys are traditionally conducted in face-to-face and have long time series to monitor trends in the public opinion. To maintain comparability, these surveys generally minimize changes to its design over time. However, face-to-face surveys have suffered from declining response rates and rising survey costs in recent decades. As a result, self-administered mixed-mode surveys (mail, web) are gaining popularity in general social research due to their ability to circumvent these challenges. However, changing the mode of data collection is a significant methodological shift that could affect estimates of public opinion. Therefore, it is important to examine whether differences in estimates of public opinion are due to changes in actual opinion and/or changes in the data collection mode.
One method that enables a comprehensive understanding of the causes of changes in public opinion estimates is to conduct two subsamples simultaneously: the default face-to-face design and the self-administered mixed-mode design. In 2023, the German General Social Survey (ALLBUS) embedded an experiment where cases were randomly assigned in one of three protocols: (1) face-to-face interviews (2) sequential self-administered mixed-mode (mail and web) (3) concurrent self-administered mixed-mode (mail and web).
This presentation examines the differences in measurement and selection between the face-to-face and the two self-administered mixed-mode designs of ALLBUS 2023. Selection bias is examined through variations in response propensities and sample composition, while measurement bias is examined at the variable level, which allows us to examine whether differences are systematic across question characteristics (use of showcards in the face-to-face interview, sensitivity of questions, and type of question (attitudinal vs. factual)).

Comparing Face-to-Face and Self-Completion Surveys: findings from the 2024 European Working Conditions Survey

Mr Jamie Burnett (Verian) - Presenting Author
Miss Alexander Cronberg (Verian)
Mr Gijs van Houten (Eurofound)

Despite ever-growing reliance on survey data and an increasing shift to self-completion formats, little research exists on mode effects between face-to-face and self-completion surveys that use probability-based offline modes of recruitment with the exception of the work done in round 10 of the European Social Survey. We present results from a nationally representative, population-based mode experiment across the EU27 countries. The European Foundations for the Improvement of Living and Working Conditions (Eurofound) commissioned Verian to conduct the eighth edition of the European Working Conditions Survey (EWCS) in Spring 2024. As part of Eurofound’s strategy for future-proofing its surveys, the EWCS 2024 was conducted both face-to-face and online in all EU Member States (using either a telephone push-to-web or a postal push-to-web approach).
In this paper we will discuss the results of this experiment, investigating the impact of a mode switch for response rate, sample composition (e.g. gender, age, level of education, occupation and industry) and data quality (e.g., item nonresponse, filter errors, response patterns). The results of this experiment will help inform researchers and users of research on the future development of viable push to web designs across Europe.

Comparing an ‘ask first’ vs ‘ask last’ approach to recruitment for a pan-European online survey: findings from the 2024 European Working Conditions Survey.

Mr Jamie Burnett (Verian) - Presenting Author
Mr Gijs van Houten (Eurofound)
Miss Alexander Cronberg (Verian)

The European Foundations for the Improvement of Living and Working Conditions (Eurofound) commissioned Verian to conduct the eighth edition of the European Working Conditions Survey (EWCS) in Spring 2024 in 37 countries. As part of Eurofound’s strategy for future-proofing its surveys, the EWCS 2024 was conducted both face-to-face and online in all 27 EU Member States (using either a telephone push-to-web or a postal push-to-web approach), and the implementation of the online component included a range of test elements. We will focus on the experiment that looked at when to ask individuals whether they would like to take part in an online survey.
For many countries in Europe accessing an address or person level register for a push to web survey design is not feasible without a sponsor’s support (e.g. government department, academic partner). If you want to maintain a probability-based approach to recruitment for an online survey then an RDD telephone sample frame is often the only viable option. However recruitment costs by phone often dwarf those using an address or named register and postal letter, it’s therefore important to look at what measures can be taken to reduce these costs. In this paper we examine the response rates and associated costs with an ‘ask first’ vs ‘ask last’ design for the recruitment of the working population to an online survey.