ESRA 2025 Preliminary Program

Monday 14 July    Tuesday 15 July    Wednesday 16 July    Thursday 17 July    Friday 18 July   

All time references are in CEST

Innovative Uses of Multimode and Multidata in Surveys: Challenges, Success, and Error Trade-offs
Session Organisers	Dr Ting Yan (NORC at the University of Chicago) Dr Leah Christian (NORC at the University of Chicago)
Time	Friday 18 July, 09:00 - 10:30
Room	Ruppert paars - 0.44

Declining response rates and increasing cost of data collection continue to challenge survey practitioners, survey researchers, and government agencies. At the same time, there is a growing need for more data and for more data to be collected more frequently, timely, and cost effectively. As a result, surveys are increasingly employing multiple modes both to contact sampled persons and to collect data from them. For instance, sampled persons may be mailed postal letters and also visited by field interviewers inviting them to participate in a survey. They may be also offered to participate in the survey online or by telephone. Furthermore, data from multiple sources are utilized to assist survey data collection, and to supplement and complement data from traditional surveys for inferences. For instance, satellite images are used to prescreen for buildings with certain desired characteristics. Administrative data can be linked to obtain additional information without burdening survey respondents.

This session will be exploring the innovative uses of multimodes and multidata sources to assist survey data collection, reduce burden, improve survey quality, and reduce cost of data collection. This session focuses on challenges, success, and error trade-offs in the multimode and multidata environment. Researchers are invited to submit papers on any of the following topics:

• Challenges, success, and error trade-offs of using multiple modes to contact/reach sampled persons
• Challenges, success, and error trade-offs of using multiple modes to collect data from sampled persons
• Challenges, success, and error trade-offs of using data from different sources to improve sampling efficiency
• Challenges, success, and error trade-offs of using data from different sources to assist and supplement data collection
• Challenges and success of combining survey data across multiple modes and from multiple sources
• Evaluating error trade-offs arising from combining data from multiple modes and/or multiple

Papers

Multi-national Initiatives on Enhancing Alternatives to Interviewer-Based Surveys

Dr Paul Beatty (US Census Bureau) - Presenting Author
Ms Emma Farrell (Australian Bureau of Statistics)
Ms Fiona James (UK Office for National Statistics)

The rising costs of interviewer-based surveys require statistical agencies to consider less expensive alternatives, such as adaptation to self-administration and use of auxiliary data. Of course, survey sponsors also seek assurances that such alternatives produce comparable estimates that meet data quality standards. Representatives of several national statistical agencies met in Geelong, Australia in December 2022 to discuss potential approaches to address these problems. This paper highlights subsequent efforts to carry forward a research agenda at three of these agencies: the US Census Bureau, the Australian Bureau of Statistics, and the UK Office for National Statistics. A substantial effort for all three agencies includes evaluating self-administered alternatives to interviewer-based questions. Examples including replacement of open-ended questions that require interviewer probing and follow-up with simpler, self-contained questions; reduction of instructions and navigational complexities; modification of help features from interviewer-based to respondent-based; creation of visual aids for both motivation and clarification; and the use of AI to assist with complex classifications usually performed by interviewers. Follow up studies (involving experiments when possible, but also including qualitative evaluation, paradata and response metrics) evaluate how well alternative versions produce comparable results, and exploring data quality tradeoffs produced by the alternative versions. Also, while some agencies are using web-first administration, others are exploring whether response can be optimized by pushing only subsets of the sample to self-administration. Finally, agencies increasingly explore the quality of alternative data sources to augment or replace survey items, as well as respondent willingness to provide access to these data. The presentation provides examples of each of these initiatives spanning the three agencies, noting both commonalities across research programs as well as distinctive emphases.

The Impact of Time-invariant and Time-varying Factors on Survey Participation and Sample Representativeness in Multimode Surveys

Dr Hanyu Sun (Westat) - Presenting Author
Dr Ting Yan (NORC at the University of Chicago )
Dr David Cantor (Westat)

Multimode data collection has become particularly popular in the recent years. The literature has shown that using multiple modes of contact can increase response rates over a single contact mode (Dillman et al. 2014). Most of the previous research has examined the impact of time-invariant factors on survey response such as mode of data collection, contact mode, and respondent characteristics (Coffey, Maslovskaya, McPhee 2024). A few studies have examined how time-varying factors affect response patterns such as date and time of contact (e.g., Fang et al. 2021). There is little research explores the impact of time-varying factors on sample representativeness and the interaction between time-invariant and time-varying factors on survey participation. In this presentation, we will examine whether and how time-varying and time-invariant factors affect survey participation and sample representativeness in four waves of a multimode panel study. Specifically, we will study the effects of time-varying factors (days in the field, day of a week, contact mode) and time-invariant factors (e.g., demographic characteristics) on respondents’ participatory decision, using discrete-time survival analysis to model the effects of time-varying and time-invariant factors on the conditional odds of a person responding to the web survey (given that they have not responded yet). We will also examine the sample representativeness by contact and by wave utilizing sampling frame variables. In addition, we will discuss the implications of the findings and recommendations for future research.

A Meta-Analysis of Web and Mail Mixed-Mode Survey Experiments on Response Rates and Web Participation

Dr Kristen Olson (University of Nebraska-Lincoln) - Presenting Author
Dr Jolene Smyth (University of Nebraska-Lincoln)
Dr Rebecca Medway (AIR)
Dr Ting Yan (NORC)

Mixed-mode surveys provide tools to survey researchers to balance survey costs and survey errors. When mixing self-administered web and mail survey modes, an open question is whether to offer survey respondents both mail and web surveys concurrently or to order them sequentially. Although initial research on mixed-mode surveys indicated that concurrent mixed-mode designs yield lower response rates than mail-only surveys (Medway and Fulton 2012), more recent work raises questions about whether this finding still holds (Olson, et al. 2021). In this paper, we use meta-analytic methods to examine response rates and mode selection in mail-only, web-only, concurrent mixed-mode, and sequential mixed-mode surveys. We examine 83 published and unpublished studies from 2001 to 2023. Initial findings indicate that web-only response rates are about 14 percentage points lower than mail-only surveys. For early studies (before 2012), we replicate the finding that mail-only surveys have higher response rates than concurrent mixed-mode designs, but the difference in response rates between concurrent mixed-mode designs and mail-only surveys has disappeared in more recent studies. We also find that sequential mixed-mode designs have lower response rates than mail-only studies, on average. We find that the mix and sequence of modes affects the proportion of respondents who participate by web, with about 25% participating by web in a concurrent mixed-mode design – a rate that has increased over time – and over half participating by web in a sequential web-to-mail design. We examine moderating factors at the study and treatment level, including whether the study is longitudinal or cross-sectional, whether email is used as a contact method, and use of incentives, among others. We conclude with implications for survey practice.

Mixing data collection modes to achieve response rates above 70% - Results of a mixed-mode experiment at the Hungarian Central Statistical Office

Mr Mátyás Gerencsér (Hungarian Central Statistical Office) - Presenting Author

It is widely accepted that mixing data collection modes is an effective way to reduce survey nonresponse. This is because different data collection methods may engage different respondent groups, thus improving coverage, reducing the rate of nonresponse errors, and making data collection more cost-effective. However, mode effects may also occur.
Due to declining response rates (RRs), the Hungarian Central Statistical Office designed a field experiment in which a probability sample, selected from the Hungarian registry of addresses, was randomly divided into three groups with different sequential mix-mode data collection protocols. CAWI was the first mode in each group; in Group A it was followed by CATI, then CAPI, in Group B the order of interviewer-administered modes was reversed, whereas in Group C only a single CAPI mode followed. Nonrespondents from all groups were re-invited for a final CAWI phase.
The aim was to reveal how the order of data collection modes may affect the RR, respondents’ characteristics and response quality. Additionally, it was also examined to what extent nonrespondents can be converted by reintroducing a post-fieldwork CAWI option. It was also analyzed how mode effects differ during the second CAWI round from the responses collected in the first CAWI round.
Different sequences led to notable differences in settlement type and income. Other results were generally not significant. In the final phase, when CAWI was reintroduced, 25% of all previous nonrespondents were successfully converted. It contributed greatly to achieving a high overall RR of 72.37%. Group- and mode-specific results of the experiment will also be presented and discussed.

Transitioning ELSA to a Multimode Fieldwork Design: Opportunities, Challenges, and Trade-offs

Mrs Lina Lloyd (National Centre for Social Research) - Presenting Author

The English Longitudinal Study of Ageing (ELSA) follows people aged 50 and over. It is a pivotal study helping understand ageing dynamics in the UK for over two decades. With 11 waves of data collected primarily through face-to-face interviews and paper-based self-completions, ELSA is transitioning to a multimode design in Wave 12. This shift is driven by the need for cost-efficiency, declining response rates, and a growing demand for timely, frequent, and high-quality data.

This session explores the use of multimode design in ELSA to improve making contact and following longitudinal survey respondents while addressing the challenges and trade-offs associated with such transition. There is a specific concern with transitioning ELSA to web-first because of the elderly target population for this survey. Because of this the communication strategy incorporates tailored approaches, such as offering flexible methods for updating contact details, direct face-to-face invitations for participants with cognitive impairments, low digital skills or limited internet access. These strategies have been assessed in the planning stages. This includes a dress rehearsal, cognitive and usability testing, focus groups with participants, and literature reviews. Key findings and preliminary results from the 2025 dress rehearsal will be shared.

Particular focus will be given to the challenges of transitioning specific survey components online, such as cognitive assessments and health visits, which are particularly prone to mode effects. To mitigate these, a hybrid approach was adopted: cognitive tasks will be administered via telephone for those completing the main survey online. Health visits will continue to be administered face-to-face every other wave. Keeping some interviewer contact elements in the survey will ensure data consistency while maintaining respondent engagement.

By addressing challenges and sharing lessons learned, this session offers valuable insights into the future of longitudinal surveys.