All time references are in CEST
Live video interviewing in survey practice 2 |
|
| Session Organisers | Dr Dina Neiger (The Social Research Centre) Mrs Emma Farrell (Australian Bureau of Statistics) Mr Tim Hanson (European Social Survey (City, University of London)) Mr Brad Edwards (Westat) Dr Andrew Hupp (University of Michigan, Survey Research Center) Dr Frederick Conrad (University of Michigan, Institute for Social Research) |
| Time | Wednesday 19 July, 16:00 - 17:30 |
| Room | U6-11 |
Whether it is called ‘video assisted live interviewing’ (VALI), ‘video interviewing’ (VI) or a label that includes both ‘computer-assisted’ and ‘video’ terms, many research organisations around the world have added video communication technology to their suite of data collection methods. While the feasibility of live video interviewing was investigated prior to 2020, the approach was thrust into the mainstream as a result of COVID-19 pandemic that made in-person, face-to-face interviewing impractical or impossible at short to no notice for most data collection contexts. The method is now being extended into a number of longer-term or ‘business as new normal’ situations.
We are looking for survey methodology submissions about surveys conducting interviews via live video. Our focus is on video interviews for quantitative surveys. Household and establishment surveys are of equal interest. How has video interviewing faired across different types of surveys (for example subject matter, interview length, population), recruitment approaches and technology set ups? What kind of adjustments to survey content, data collection procedures, materials, has the medium made necessary? What other benefits or promises have resulted from using the method.
Experimental studies are welcome but not necessary. Qualitative evaluations, exploratory research and theoretical musings about intriguing respondent or interviewer behaviour which may be helpful for other organisations embarking on using the method are also within scope.
The intent of the session is to explore advancements in and barriers to video interviewing around the world and to encourage survey practitioners involved in video interviewing to present papers exploring and reporting on the application of this method in practice and implications for total survey error and survey operations.
It is hoped that this session would be of interest to survey practitioners experienced in the use of video interviewing as well as those who are interested
Keywords: live video interiviewing
Dr Diana Zavala-Rojas (European Social Survey ERIC, RECSM - Universitat Pompeu Fabra)
Mr Oguz Korkut Keles (RECSM - Universitat Pompeu Fabra) - Presenting Author
Ms Hannah Schwarz (RECSM - Universitat Pompeu Fabra)
Ms Elena Romanin (RECSM - Universitat Pompeu Fabra)
The Covid-19 pandemic has posed new challenges to the traditional mode of administration of household survey programmes. The unfeasibility of performing in-person interviews due to the Covid-19 restrictions accelerated the trend towards alternative modes of survey data collection, such as live video interviewing. This paper investigates whether survey data collected with two different interviewing modes, face-to-face and video interview, are statistically comparable across groups. Indeed, measurement invariance is a necessary prerequisite to make meaningful comparisons across groups. A difficulty in testing this is that the number of cases per group varies substantially. Using data from the European Social Survey Round 10 we check if the measurement invariance condition is met for two concepts, social trust and attitudes towards immigration, in six different countries. We assess this using the bias-corrected (BC) bootstrap confidence interval methodology.
Mr Ævar Þórólfsson (Social Science Research Institute of the University of Iceland) - Presenting Author
Dr Guðbjörg Andrea Jónsdóttir (Social Science Research Institute of the University of Iceland)
Mr Árni Bragi Hjaltason (Social Science Research Institute of the University of Iceland)
Mr Helgi Guðmundsson (Social Science Research Institute of the University of Iceland)
Although COVID-19 restrictions are much less than they were in 2020-2022, some of them might return and some population segments are still minimizing contact with strangers. This can cause problems for researchers using face-to-face interviews. Using other means of data collection as a proxy, such as video interviews, can minimize the damaging effects on response rates. This paper discusses the data collection in round 10 of ESS conducted in Iceland from July 2021 to February 2022. A random sample of 2758 individuals was drawn from the National Population Register. An advance letter introducing the choice between participating through a face-to-face interview or a video call via a computer was sent by mail. The letter was followed up by a telephone call to schedule an interview of respondents’ choice. After accounting for 70 ineligible individuals in the sample, a net response rate (AAPOR RR1) was 33.6% with 903 completed interviews. Just under 37% of respondents (n = 333) chose a video call. The Whereby platform was selected for the video calls due to its user friendliness. Contrary to alternative solutions, Whereby does not require the respondent to login or install any software. Those who chose to be interviewed via a video call were overall younger (m = 42.2) than those who answered by other modes of data-collection (m = 54.8). Furthermore, a video interview was more popular among respondents with a tertiary degree (44.9%) than primary or secondary education (32.3%). The paper explores what distinguishes respondents choosing different interviewing modes and how the option of a video call changes the representativeness of the survey. Furthermore, we explore how video interviews change the interview duration as well as the attitudes of the respondents towards the interview process.
Dr Dina Neiger (The Social Research Centre) - Presenting Author
Dr Benjamin Phillips (The Social Research Centre)
Mr Grant Lester (The Social Research Centre)
Mr Sam Slamowicz (The Social Research Centre)
Ms Emma Farrell (Australian Bureau of Statistics)
Ms Kirsten Gerlach (Australian Bureau of Statistics)
Mr Philip Carmo (Australian Bureau of Statistics)
The Social Research Centre in collaboration with the Australia Bureau of Statistics (ABS), systematically trialled video-assisted live interviewing (VALI; n = 600) in parallel with other survey modes and sampling frames in a study fielded December 2022. This included web mode using a probability-based online panel, Life in Australia™ (n = 600), two mobile RDD CATI streams (high effort, n = 500; low effort, n = 500), RDD SMS push-to-web (n = 600), and four nonprobability access panels (n = 850 per panel). The questionnaire used items with available high-quality benchmarks across a range of domains including health, substance use, disability, caring, psychological distress, and employment status. We deliberately included items likely subject to mode effects.
The VALI trial built on ABS learnings from implementing VALI during the pandemic for national household surveys that could no longer use face-to-face mode. VALI respondents were recruited from Life in Australia™. As the sample was drawn from a panel, we provide detailed information on drivers of non-response at various stages using panel profile and participation history: the expression of interest in VALI interviewing, scheduling, and completing an interview.
We compare error in VALI to the other survey methods trialled with respect to the benchmarks. The broad range of modes included in the study help to situate VALI performance methods with respect to both self- and interviewer-administered modes and probability and nonprobability frames. We also discuss lessons learned with respect to implementation.
This paper contributes to the emerging evidence base on VALI methods from Europe and the U.S. It provides detailed information on drivers of nonresponse to VALI interviews and errors in VALI in comparison to other methods with respect to external benchmarks.
Dr Jennifer Kelley (Westat) - Presenting Author
Mr Jesus Arrue (Westat)
Mr Brad Edwards (Westat)
Mr Rick Dulaney (Westat)
The use of online video technologies to support survey interviews was still in early development at the start of the pandemic. These technologies marry traditional computer-assisted interviewing (CAI) systems with remote video capability. We refer to this new mode as “computer-assisted video interviewing” (CAVI). The scant literature available has focused on experimental designs to assess CAVI quality (Schober et al., 2020; Endres et al., 2022). The results show CAVI’s potential: there is little evidence of interviewer effects (Schober et al., 2020), and it seems to produce data that is more comparable to CAPI than to online interviewing (Endres et al., 2022). Whether these findings will hold up in a full-fledged production environment is still being determined. Further, there is still much to learn about CAVI, including potential mode effects for CAVI compared to in-person (i.e., CAPI) and telephone (CATI).
To our knowledge, no organizations have implemented CAVI in full production. In spring 2022, the Medical Expenditure Panel Survey (MEPS) began offering CAVI on a limited basis to returning households that wished to avoid face-to-face interaction due to Covid concerns. With this successful inaugural implementation of CAVI, MEPS began offering CAVI as a primary and secondary mode of data collection in the fall of 2022.
After completing a full data collection cycle in which over 20 percent of interviews were completed using CAVI, this paper examines potential mode effects in a production setting. We compare CAVI to CAPI and CATI on several production metrics and quality measures, including the distributions of key statistics in the MEPS. The results will provide insight into whether CAVI can leverage the benefits of CAPI (e.g., building respondent rapport) and CATI (e.g., reducing cost) without sacrificing quality.
Miss Theresia Ell (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Dr Lydia Repke (GESIS - Leibniz Institute for the Social Sciences)
Dr Henning Silber (GESIS - Leibniz Institute for the Social Sciences)
Healthy social relationships are essential for human well-being and the functioning of society. They influence individual actions, attitudes, and behaviors. Accordingly, the outbreak of the Covid-19 pandemic and the policies to contain it are associated with high social costs. While the far-reaching effects at the macro level can be understood using administrative data, the same is not true for the micro level (i.e., the individual), the meso level (e.g., social inclusion), and the interaction between these two levels.
The LoneCovid project addresses this issue by aiming to collect quantitative egocentric network data of 160 members in Germany's general adult population in May 2023. Network data can be used to study issues related to social interactions, such as how actors influence, support, or even harm each other. We will select our participants by drawing a stratified random sample from the probability-based GESIS Panel. In live video-based interviews, participants will have to nominate their habitual contacts and provide information on them (e.g., sociodemographics, relationship type) and the relationships between them (who knows whom). To facilitate the elicitation of such complex data, we will use the open-source software Network Canvas. This interviewing tool is highly visual and based on consistent interaction (e.g., dragging and tapping), thereby reducing respondent burden.
In our presentation, we will first briefly introduce the research project. Second, we will describe the sampling design and illustrate the implementation of an egocentric network study whose data were collected with video-based interviews using visual and interactive software. Finally, we will summarize the lessons learned during the data collection and give best practice advice on how researchers can gather complex egocentric network data within live video interviews.