Online Probing in Comparison to Face-to-Face Cognitive Interviewing |
|
| Coordinator 1 | Miss Catherine Fenton (National Centre for Social Research ) |
| Coordinator 2 | Dr Ruxandra Comanaru (National Centre for Social Research ) |
Proposal for a session at the 8th Conference of the European Survey Research Association (ESRA)
University of Zagreb, Croatia, 15th to 19th July 2019
Session organisers: Ruxandra Comanaru and Catherine Fenton (NatCen, UK)
Email: Ruxandra.Comanaru@natcen.ac.uk & Catherine.Fenton@natcen.ac.uk
Online probing in comparison to face-to-face cognitive interviewing
Online probing has become a popular method for evaluating survey questions due to the advantages of recruiting respondents quickly and in a more cost-effective way. Cognitive testing methods allow us to explore the processes by which respondents answer survey questions, exploring whether respondents understand the concepts used; are able to provide an answer in a consistent way; and in a way that the researcher intended (Collins, 2014). When adopting an online probing method, the cognitive interviewer is absent and the respondent answers the probing questions in a self-administered form.
Online probing has been regarded as an effective pretesting method due to its advantages (Edgar, 2012). Researchers highlight the benefits of using online probing methods, including, being able to quantify the findings (Behr et. al, 2012) and recruitment of respondents can be quicker, more cost effective and more accessible. Additionally, as the online survey is completed without a cognitive interviewer, this eliminates the interviewer effects and may increase the reliability of the findings (Conrad and Blair, 2009). Nevertheless, there are limitations to online probing in comparison to face-to-face cognitive testing. During face-to-face cognitive testing, interviewers can probe for more information where more detail is required to explore the respondents understanding of the question, and this method may therefore provide more in-depth information (Meitinger and Behr, 2016).
Research literature and studies have explored online probing methods. Despite this, there are still gaps that remain about the primary differences between online probing and more traditional cognitive testing methods, and when to use which method. Furthermore, there is much debate about which kind of items can be evaluated by web probing and around the sample size most appropriate for using this method.
For this session, we invite papers comparing findings from web probing with cognitive interviewing methods, discussing the factors that might guide decisions about when to use web probing; and any practical considerations in using web probing methods either on their own or in conjunction with other testing methods.