All time references are in CEST
Live video interviewing in survey practice 3 |
|
| Session Organisers | Dr Dina Neiger (The Social Research Centre) Mrs Emma Farrell (Australian Bureau of Statistics) Mr Tim Hanson (European Social Survey (City, University of London)) Mr Brad Edwards (Westat) Dr Andrew Hupp (University of Michigan, Survey Research Center) Dr Frederick Conrad (University of Michigan, Institute for Social Research) |
| Time | Thursday 20 July, 09:00 - 10:30 |
| Room | U6-01e |
Whether it is called ‘video assisted live interviewing’ (VALI), ‘video interviewing’ (VI) or a label that includes both ‘computer-assisted’ and ‘video’ terms, many research organisations around the world have added video communication technology to their suite of data collection methods. While the feasibility of live video interviewing was investigated prior to 2020, the approach was thrust into the mainstream as a result of COVID-19 pandemic that made in-person, face-to-face interviewing impractical or impossible at short to no notice for most data collection contexts. The method is now being extended into a number of longer-term or ‘business as new normal’ situations.
We are looking for survey methodology submissions about surveys conducting interviews via live video. Our focus is on video interviews for quantitative surveys. Household and establishment surveys are of equal interest. How has video interviewing faired across different types of surveys (for example subject matter, interview length, population), recruitment approaches and technology set ups? What kind of adjustments to survey content, data collection procedures, materials, has the medium made necessary? What other benefits or promises have resulted from using the method.
Experimental studies are welcome but not necessary. Qualitative evaluations, exploratory research and theoretical musings about intriguing respondent or interviewer behaviour which may be helpful for other organisations embarking on using the method are also within scope.
The intent of the session is to explore advancements in and barriers to video interviewing around the world and to encourage survey practitioners involved in video interviewing to present papers exploring and reporting on the application of this method in practice and implications for total survey error and survey operations.
It is hoped that this session would be of interest to survey practitioners experienced in the use of video interviewing as well as those who are interested
Keywords: live video interiviewing
Dr Rick Dulaney (WESTAT) - Presenting Author
Dr Jennifer Kelley (WESTAT)
Dr Jesus Arrue (WESTAT)
Dr Brad Edwards (WESTAT)
Video technology has been used for years for qualitative research (Forrestal et al., 2015; Janghorban et al., 2014), but until the COVID-19 pandemic shut down face-to-face interviewing, it had rarely been utilized as data collection mode. At that time, the use of video meeting technologies exploded, and survey organizations quickly began exploring this medium as a new mode of data collection.
Beginning in the spring of 2022, the Medical Expenditure Panel Survey (MEPS) offered video interviews on a limited basis to returning households that wished to avoid CAPI face-to-face interaction due to Covid concerns. With the successful implementation of computer-assisted video-interviewing (CAVI), MEPS began offering video interviews as a primary and secondary mode option for the fall 2022 data collection period. This second field period had a significant increase in those adopting CAVI. Field staff completed 20.8% of the 12,280 interviews using CAVI, 18.3% by telephone (using CATI), and 60.9% in person (CAPI).
Given that CAVI is a viable alternative to CAPI and CATI, it is essential to know what type of respondents are adopting CAVI not only to assess potential bias but also to aid and maximize field operation efforts. This presentation examines CAVI respondents and household characteristics compared to CAPI and CATI respondents and households. We analyze the demographic characteristics of those respondents and households adopting CAVI in both field periods compared to those using other modes of data collection. We also analyze what type of respondents who adopted CAVI during the first field period are still using it for the second field period and what type of respondents are switching to a different mode of data collection. The findings will provide a better insight into the type of respondents adopting CAVI overall and in consecutive periods.
Ms Lena Centeno (Westat) - Presenting Author
Ms Jennifer Kelley (Westat)
Mr Jesus Arrue (Westat)
Mr Brad Edwards (Westat)
Mr Ryan Hubbard (Westat)
Mr Rick Dulaney (Westat)
Video interviewing using CAPI or CATI instruments has emerged as a new mode of data collection (Schober, et al. 2020). We distinguish this mode from other video interviewing (in cognitive interviews for quantitative or qualitative surveys, or in other fields such as telehealth) with the term Computer-Assisted Video Interviewing, CAVI for short). Pre-pandemic, the Medical Expenditure Panel Survey (MEPS) was a continuous in-person CAPI study, with heavy reliance on show cards and medical records. When COVID hit in early 2020 MEPS quickly shifted to telephone (i.e., distributed CATI), but by the start of 2022 had swung back to mostly in-person. MEPS introduced CAVI in the spring 2022 cycle as a primary option for households that preferred to avoid face-to-face interaction. All 300 interviewers were trained on CAVI by April.
This paper presents results of our first full data collection cycle of CAVI implementation on MEPS, with a focus on the survey operations experience in the second half of 2022. Between July and December Westat field staff completed 12,280 interviews: 2,553 (20.8%) by CAVI, 2,249 (18.3%) by telephone, and 7,478 (60.9%) in person. Four panels were in play. The proportion of CAVI interviews varied by panel in predictable ways, based on years in survey, and was related to previous interview mode. CAVI production varied greatly by interviewer. We profile the “CAVI savvy” and reluctant adopter groups. Why some respondents did not accept the CAVI offer, and the role of CAVI in persuading respondents to participate in the survey are among the topics explored in this presentation. We conclude with next steps for operations in 2023.
Mr Yves Fradier (Kantar Public)
Mr Laurent Martin (GIDE) - Presenting Author
The ESS is an academically driven cross-national survey that has been conducted across Europe since its establishment in 2001. Every two years, face-to-face interviews are conducted with newly selected, cross-sectional samples.
In a context of declining participation rates, the COVID crisis aggravated the situation, and we had to add a new item to the long list of reasons for refusing to answer : fear of contagion.
The people who took this position normally dropped out, but for the first time we had the technical possibility of interviewing them anyway, by offering them a video interview. Out of 1977 completed questionnaires, 43 were conducted in CAVIsio, which is not a usual video-conferencing solution à la Teams or Zoom: it is an innovative tool dedicated to Social and Market Research as it let the interviewer view the video flow and the questionnaire in a same single window, while the respondent, on his side, is able to see the showcards shared by the interviewer beside the video flow
Their profile was very specific. Namely, more than the majority are men (56%). While the sample concentrates a population of active age (with 60% in permanent employment), the average age of CAVIsio respondents is slightly lower (39 years old compared to 43 years old for face-to-face respondents). The most highly educated are more represented: 12% have a university diploma of technology (DUT) or a higher technician's certificate (BTS) and 7% have a doctorate. Moreover, 60.5% consider that they live comfortably. These results give us a clearer picture of the profile that could be reached by CAVIsio.
Finally, the CAVIsio format would lead to an increase in the response rate by converting refusals into face-to-face interviews and to a better coverage of the general population.
Ms Christine Carr (RTI) - Presenting Author
Ms Katelan McDaniel (RTI)
Ms Wendy Reed (RTI)
Mr Tim Smith (RTI)
Dr Paul Geiger (RTI)
Dr Heidi Guyer (RTI)
Mr R Suresh (RTI)
The Mental and Substance Use Disorders Prevalence Study (MDPS) began data collection in October 2020 and concluded data collection in October 2022. The study was designed to estimate the prevalence of serious mental and substance use disorders among adults in the United States, ages 18-65, residing in household and non-household settings such as prisons, homeless shelters, and state psychiatric hospitals. The data were collected via clinical interviews, which were completed by live video, phone, or in-person.
Over 3,500 live video clinical interviews, 1,600 phone clinical interviews, and 600 in-person clinical interviews were conducted for the MDPS project in the household, prison, homeless shelter, and hospital settings. All clinical interviews with household respondents were conducted via Zoom (live video or phone) whereas facility respondents completed video- or phone-based interviews when in-person data collection was not possible due to COVID-19 safety precautions. Overall, nearly 67% (n =3179 ) of household interviews were conducted by video and about 31% (n = 280) of the interviews in non-household settings (hospitals, homeless shelters and prisons) were conducted by video. Clinical Interviewers used two devices when conducting the video-based clinical interviews—a laptop to administer the clinical interview and record the responses and a tablet for the Zoom video connection. This presentation will provide an overview of MDPS live video clinical interviewing conducted in household and non-household settings, focusing on scheduling, video interviewing logistics and adjustments made to data collection procedures, data quality (10% of completed interviews were reviewed; approximately 4% resulted in disorder-level scoring changes), interview administration time by mode and sample, and prevalence rates of mental health and substance use disorders by mode. Video interviewing benefits, challenges, and lessons learned will also be discussed.