All time references are in CEST
Challenges and Consequences of Switching Face-to-Face Population Surveys to Self-Completion (Mixed-Mode) Designs 2 |
|
| Session Organisers |
Dr Joss Roßmann (GESIS - Leibniz Institute for the Social Sciences, Germany) Mr Michael Blohm (GESIS - Leibniz Institute for the Social Sciences, Germany) Ms Caroline Hahn (GESIS - Leibniz Institute for the Social Sciences, Germany) Dr Oshrat Hochman (GESIS - Leibniz Institute for the Social Sciences, Germany) Dr Jan-Lucas Schanze (GESIS - Leibniz Institute for the Social Sciences, Germany) |
| Time | Thursday 20 July, 14:00 - 15:30 |
| Room | U6-08 |
The public health regulations that were introduced across Europe (and beyond) since Spring 2020 due to the Corona pandemic impeded the realization of personal face-to-face survey data collection or made it even temporarily impossible – potentially with long ranging consequences for this data collection method. For this reason, surveys of the general population, or important subgroups of the population such as voters, chose to switch from face-to-face interviewing to a mix of self-completion survey modes to secure survey responses from all groups of the target population (e.g., respondents without Internet access or those unwilling to respond in a certain mode).
The Corona pandemic, thus, strongly accelerated a continuing trend of moving surveys from interviewer-administration to self-completion modes. With respect to the general trend and the sustained consequences of the Coronavirus in particular, we welcome submissions that present insights on the methodological and practical challenges of switching population surveys from face-to-face to (mixes of) self-completion modes, and on the effects of doing so on the quality of the data.
Contributions may cover but are not limited to the following research topics:
- Approaches to overcome methodological and practical challenges of switching modes (e.g., regarding sampling and coverage)
- Conception and implementation of self-completion mixed-mode designs for general population surveys (e.g., adaptations in questionnaire design)
- Experimental testing of survey designs to maintain or enhance response and quality of measurement (e.g., use of conditional and unconditional incentives, split questionnaire designs)
- Consequences of switches to self-completion on data quality and total survey error
- Implications of mixing self-completion data collection modes on measurement invariance and possible remedies
- Impact of switching data collection modes on time series in repeated cross-section and panel population surveys
Keywords: Self-completion mixed-mode surveys, mode switch, total survey error, data quality
Dr Mare Ainsaar (University off Tartu, Tallinn Strategy Centre) - Presenting Author
Miss Hanna Kerstina Kartau (University of Tartu)
During COVID19 pandemic European Social survey experienced new modes for data collection. While the mixed and self-completion web survey experiments were already earlier on the table of experiments, a new mode - web- interviews for quantitative research - emerged as a response to COVID. It was expected, that non-contact web interview might be interesting and comfortable option for respondents who would like to lower risk of infection. The web mode with its interviewer effect is also suitable mode for combination with traditional face to face data collection.
Estonia was one those countries who used mixed mode interview methods for 2020-2021 year data collection of European Social Survey. Video interviews were combined with face to face interviews. 15.6% of total interviews were collected in video mode. This was comparatively high share, although the ESS country team had even higher expectations.
The presentation describes briefly the key elements of set up of video interviews in the country, challengers and solution. We analyse also the main differences of video interviews in terms of sample, response behaviour and data quality.
Mr Nikola Jovanovski (Sample Solutions) - Presenting Author
Mr Carsten Broich (Sample Solutions)
Mr Alexander Nisetich (RTI International)
Mr Luis Sevilla Kreysa (RTI International)
Mr Charles Lau (RTI International)
Throughout the Covid-19 pandemic, surveying developing nations has witnessed a primary shift from Face-to-Face (FTF) interviewing to the utilization of telephone, especially with cellular phones - by either CATI, SMS or IVR, due to social distancing and lockdown mandates.While telephone surveys are not an exact replacement of FTF, they can be a viable option for data collection during events that impact face-to-face interviewing.
Considering that developing countries have seen growth in mobile phone subscribers at the same time as landline subscriptions are in decline (ITU), interviewing respondents on cellular devices is becoming a more practical mode for assessing public opinion, attitudes, and public health outcomes, especially in scenarios where Face-To-Face does not allow it. We compare the two approaches and the trade-offs involved when selecting between telephone and FTF.
This paper showcases the strengths and weaknesses of screened random digit dial (RDD) sampling for surveying of populations during force majeure scenarios. It also compares benefits and limitations when using RDD sampling for CATI, IVR and SMS surveying in developing countries.
We describe the process by which an RDD sample is obtained using the official numbering plans of each country, covering all generatable telephone combinations, and following the general distribution of telephone respondents.
We then showcase the sample’s performance, which includes: cooperation rates and response rates from recent RDD phone surveys.. Finally, we discuss sampling methods and considerations to mitigate the response bias and potential causes of the over/underrepresentation of certain demographics as a result of the sampling error. We describe different design and sampling approaches to enhance and improve RDD sampling: implementation of quotas during data collection, use of mixed modes, and potential benefits of routine RDD surveys.
Mr Michael Blohm (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Mrs Alexandra Asimov (GESIS - Leibniz Institute for the Social Sciences)
Over the past decade, rising survey costs and decreasing response rates have forced large F2F surveys to consider more cost-effective data collection techniques, such as self-administered modes. This process has been accelerated by the COVID pandemic and the inability to conduct face-to-face interviews.
The consequences of the exclusive use of self-administered modes for large-scale surveys with a relatively long survey duration of 50-60 minutes remain understudied. Findings from the German part of the EVS 2017 and the ALLBUS methodological study (2020) indicate using a concurrent mixed-mode design (mail and web) in such surveys is possible. However, the concurrent mode-choice sequence leads to a high proportion of mail surveys (70-80%). Thus, implementing this sequence may imply losing some of the advantages of a web mode like automatic routing for example. Assuming that the internet and computer skills of the general population will be so widespread in the near future that a strictly online mode survey can be implemented (without or with very little use of the paper version as an alternative), it is still not clear what consequences an increased use of online interviews would have on net samples of the general population in the sense of data quality.
For the German General Social Survey 2021 (population 18+ / in private household / duration 55 min. / mixed-mode design mail and web), it was decided to test a push-to-web approach to increase the online interviews. In an experimental design, a concurrent as well as a push-to-web design were implemented.
In this presentation, we examine the consequences of the mode choice sequence with respect to three different aspects of data quality: response rates, distribution of substantive and demographic variables, and item-nonresponse/completeness of data.
Ms Blanka Szeitl (Center for Social Sciences)
Professor Vera Messing (Center for Social Sciences) - Presenting Author
Dr Bence Ságvári (Center for Social Sciences)
The changing technological and social environment of survey data collection has led researchers to seek for new avenues of data collection techniques and a rethinking of traditional survey methods. Most recently the COVID-19 pandemic and related lockdowns impeded the realization of personal face-to-face survey data collection and forced social surveys to look for self-completion solutions. However, the method of data collection may have an important influence on who participates in a survey and how people answer questions. These two seemingly different phenomena are summarized under the term "mode effect". This paper compares data collected through a self-completion (ESS Push-to-Web) and face-to-face (classical ESS) survey modes on attitudinal questions in Hungary. Both surveys used probability-based sampling and identical questionnaires. We discuss the mode effect by comparing four data sets: the PtW data with the data of the classical face-to-face survey of two ESS survey rounds (round 9 and round 10) between 2019 and 2022. Attitudes towards immigrants and gay and lesbian as measured by these surveys are evaluated in one-, two- and multi-dimensional (GLM models) analyses. The comparison of classical face-to-face and experimental PtW design finds that although the response rates of the two survey modes are quite similar, the socio-demographic composition of respondents are significantly different. Another important finding of the study is that survey mode has an independent influence on measurements: in GLM models where all other demographic factors are fixed, survey mode has a significant effect on how respondents report their attitudes, and this effect overrides the original demographic correlations.