The Switch from Face-to-Face Interviewing to Online Data Collection |
|
| Session Organiser |
Ms Gerry Nicolaas (NatCen Social Research) |
| Time | Tuesday 16th July, 16:00 - 17:00 |
| Room | D22 |
Over the last couple of decades, the survey research industry has witnessed the shift from single mode designs to mixed mode data collection. This session focuses specifically on the shift from face-to-face interviewing to mixed mode designs that include online data collection. The presentations explore the impact that this shift can have on non-response and measurement.
Keywords: face-to-face interviewing, online data collection, mixed mode
Dr Alexandru Cernat (University of Manchester) - Presenting Author
The shift from single mode designs to mixed mode, where a combination of interview modes are used to collect data, is one of the key challenges of contemporary longitudinal studies. The decision about what modes to use and how to implement them can have a big impact, influencing costs, field-work procedures, non-response and measurement error.
The research proposed here aims to better understand one of the key characteristics of a mixed mode design: how people transition from one mode to another in time. This is essential for a number of reasons. Firstly, it can inform targeting strategies. For example, it can be used to target those people that are more likely to shift from a cheap mode to a more expensive one. It can also be used in models for dealing with non-response after data collection, such as weighting models. Thirdly, it can be used to explain measurement error that appears due to the mode design.
This paper will investigate the process of changing modes by looking at waves 5-10 of the UKHLS Innovation Panel where a sequential Web-Face to face design was used. Latent class analysis will be used to find the underlying patterns of change in time of modes. The clusters found will be used both as dependent variables, to understand who are the types of respondents in each, and as independent variables, to predict future wave non-response and mode selection. Findings will inform the design and use of the main UKHLS study.
Mr Eliud Kibuchi (University of Southampton) - Presenting Author
Professor Patrick Sturgis (University of Southampton)
Professor Gabriele Durrant (University of Southampton)
Dr Olga Maslovskaya (University of Southampton)
Face-to-face interviews are held as “gold standard” method in terms of data quality which all other modes of data collection are compared to. However, declining response rates, increased survey requests and costs have promoted many surveys to switch to online administration. The available evidence on data quality between face-to-face and online surveys is mixed. Effective evaluation of data quality between the two modes is further complicated by selection effects that are confounded with measurement effects. This study applies propensity score matching to online probability and face-to-face surveys as a means of separating the measurement and sample selection components of the mode difference. In addition, we evaluate different methods of handling survey weights in propensity score matching analyses under a complex survey design. The results show measurement effects contribute the bulk of mode differences between the online and face-to-face samples before and after matching. However, propensity score matching had only a minimal effect on the magnitude of mode effects for the surveys considered. This was true even when both surveys were conducted online, which suggests that propensity score matching cannot be assumed to remove selection differences between surveys
Mrs Karen Blanke (Federal Statistical Office, Germany)
Ms Christine Kronz (Federal Statistical Office of Germany) - Presenting Author
There is an ongoing discussion on the future of data collection in social surveys in Official Statistics. The debate is rather complex, as there are several perspectives to be taken into account: Firstly, the general question, if we still need any data collection, as data are possibly available by other sources and technics (e.g. administrative data, big data, sensors). Secondly, if surveys are still considered to be as relevant, by which mode(s). Thirdly, If Online-First is to be taken seriously, which devices are of main impact. Finally, If Smartphones and possibly Apps are of main relevance, can we still apply our survey designs as adopted in the past.
Partly, these different topics are investigated in the ESSnet project Mixed Mode Designs in Social Surveys (MIMOD), conducted in 2018/2019, coordinated by ISTAT in partnership with CBS (Netherlands), SSB (Norway), STAT (Austria) and Destatis (Germany). MIMOD aims at supporting NSIs in facing challenges when implementing multi-mode and multi-devices data collection within social surveys at the European Statistical System (ESS). While five work packages cover the entire topics, one work package is addressing the use of smartphones for social surveys. The latest will be captured by this presentation. There have been several tasks undertaken to address the issue: a set of fitness criteria have been developed to assess the social surveys to be fit for smartphones, stylesheets are provided to show best practices on the design on smartphones. Conclusions and further reflections on the future of social surveys within the scope of Social Surveys in the ESS shall open the floor for discussion.