All time references are in CEST
Challenges and Consequences of Switching Face-to-Face Population Surveys to Self-Completion (Mixed-Mode) Designs 1 |
|
| Session Organisers |
Dr Joss Roßmann (GESIS - Leibniz Institute for the Social Sciences, Germany) Mr Michael Blohm (GESIS - Leibniz Institute for the Social Sciences, Germany) Ms Caroline Hahn (GESIS - Leibniz Institute for the Social Sciences, Germany) Dr Oshrat Hochman (GESIS - Leibniz Institute for the Social Sciences, Germany) Dr Jan-Lucas Schanze (GESIS - Leibniz Institute for the Social Sciences, Germany) |
| Time | Thursday 20 July, 09:00 - 10:30 |
| Room | U6-03 |
The public health regulations that were introduced across Europe (and beyond) since Spring 2020 due to the Corona pandemic impeded the realization of personal face-to-face survey data collection or made it even temporarily impossible – potentially with long ranging consequences for this data collection method. For this reason, surveys of the general population, or important subgroups of the population such as voters, chose to switch from face-to-face interviewing to a mix of self-completion survey modes to secure survey responses from all groups of the target population (e.g., respondents without Internet access or those unwilling to respond in a certain mode).
The Corona pandemic, thus, strongly accelerated a continuing trend of moving surveys from interviewer-administration to self-completion modes. With respect to the general trend and the sustained consequences of the Coronavirus in particular, we welcome submissions that present insights on the methodological and practical challenges of switching population surveys from face-to-face to (mixes of) self-completion modes, and on the effects of doing so on the quality of the data.
Contributions may cover but are not limited to the following research topics:
- Approaches to overcome methodological and practical challenges of switching modes (e.g., regarding sampling and coverage)
- Conception and implementation of self-completion mixed-mode designs for general population surveys (e.g., adaptations in questionnaire design)
- Experimental testing of survey designs to maintain or enhance response and quality of measurement (e.g., use of conditional and unconditional incentives, split questionnaire designs)
- Consequences of switches to self-completion on data quality and total survey error
- Implications of mixing self-completion data collection modes on measurement invariance and possible remedies
- Impact of switching data collection modes on time series in repeated cross-section and panel population surveys
Keywords: Self-completion mixed-mode surveys, mode switch, total survey error, data quality
Ms Katie O'Doherty (NORC at the University of Chicago) - Presenting Author
Ms Anna Wiencrot (NORC at the University of Chicago)
Mr Stephen Smith (NORC at the University of Chicago)
Dr Colm O'Muircheartaigh (University of Chicago)
The National Social Life, Health, and Aging Project (NSHAP) is a longitudinal, population-based study of older adults. It has been conducted face-to-face since its first round in 2005. For the most recent round, NSHAP was developing ways for self-collection of a complex questionnaire and biomeasures to reduce costs, allow for participation from respondents who prefer remote modes, and to evaluate the potential of a hybrid remote/face-to-face approach for future work. Due to the pandemic, the need to adapt data collection from face-to-face to remote was strongly accelerated.
In the first three rounds of NSHAP, an interviewer visited the respondent’s home to administer a questionnaire collecting data on their social network, cognition, physical health, and other aspects of wellbeing. Interviewers also collected biological specimens, anthropometrics, and physical and sensory measures.
To collect questionnaire data remotely, NSHAP adapted the instrument to be self-administered by web or paper, or interviewer-administered by phone. For biomeasure collection, NSHAP developed a kit of supplies and instructions for respondents to self-collect their own biomeasures and send back for analysis. Approximately half of the NSHAP respondents were eligible for remote data collection in 2021. The other half, as well as non-interview respondents were approached face-to-face in 2022-2023.
We will discuss the challenges of balancing methodological rigor and minimizing respondent burden when adapting a lengthy, complicated longitudinal survey for remote administration. We examine overall data quality and item-level nonresponse comparing data from remote and face-to-face data collection. Our results show the promise of self-administration for collecting complex questionnaire data and biomeasures. These findings provide valuable lessons learned and context for future work given the need for remote data collection is likely to increase in importance after the pandemic.
Dr Joss Roßmann (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
In 2021, the German Longitudinal Election Study (GLES) decided to switch the data collection mode of its pre-election survey from face-to-face interviewing to a self-completion mixed-mode design with web and mail questionnaires due to the Corona pandemic. As the fieldwork period of the survey was ultimately limited by the day of the federal election on September 26, a particular challenge was implementing a fieldwork protocol that promised to meet two objectives: First, achieving a desirable distribution of interviews across the fieldwork period, which included collecting substantive numbers of responses within the two weeks before the election; and second, facilitating overall survey response. Two survey design features were implemented to tackle these challenges: First, the sample for the survey was randomly split into three equally sized tranches, which were fielded at intervals of about a week within the four-week fieldwork period. Second, while all invited sample members received an unconditional 5 Euro prepaid incentive, 80 percent of the sample were randomly assigned to be offered a delayed conditional 10 Euro incentive for their timely participation with the second reminder letter.
The present contribution investigates the effects of the two design features on survey response and data quality in the pre-election survey of the GLES 2021. Preliminary results show that fielding the sample tranches at intervals of about a week facilitated the desired spreading of interviews across the fieldwork period. Thereby, a considerable number of interviews were collected in the two weeks before the election. Offering a delayed conditional 10 Euro incentive to reluctant respondents with the second reminder letter increased the response rate by more than two percentage points in each sample tranche. The final analyses will include more detailed assessments of the effects of the design features on sample composition and data quality.
Dr Davide Girardi (IUSVE) - Presenting Author
Dr Beatrice Bartoli (Demetra opinioni.net srl)
The dynamics involving the Italian youth population continue to be an area widely investigated by the sociological research. Within this framework, an increasing number of surveys is carried out through the CAWI mode (Computer Assisted Web Interviewing). They allow to limit the costs and are based on user-friendly tools. Moreover, the spread of digital ecosystem through the population gives them a central role compared to other modes, e.g. CATI (and CAMI) and even more F2F. However, these strengths are offset by some limitations: the response rates; for the CAWI mode in particular, the demographic traits of panels frequently used to carry out online surveys, leading to potential coverage errors. In addition, there are some difficulties related to the “digital divide” of specific segments of population. Nevertheless, it is possible to identify some strategies to improve the recruitment of the “target population” sample units, pointed out in this paper. The data from two surveys on 14-18 years old students (one national-based and the other local-based) are considered, both conducted according to a two-stage sampling design: by schools (first stage sampling) and then involving the students (second stage sampling). The field meta-data analysis shows the effects of “external” variables in relation to the response rate. More specifically, using the same tool and the same sample design, the response rate improves significantly where the survey benefits from a wide institutional coverage and an active networking with the nodes giving access to the target population. This evidence can provide useful insights to improve surveys on youth population to achieve the established research objectives.