Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Wednesday 15th July, 11:00 - 12:30 Room: O-202

Investigating Survey Fieldwork Processes: Interviewers and Their Strategies
Convenor	Dr Wojciech Jablonski (University of Lodz )

Session Details

This session invites presentations dealing with different aspects of fieldwork in interview surveys – both in person (PAPI/CAPI) and over the telephone (CATI). In particular, we are interested in two issues. On the one hand, we will focus of the fieldwork procedures, guidelines, sets of rules, etc. implemented in order to keep the research process standardized and achieve high quality of survey data. On the other, we will investigate the problem of complying with these principles during fieldwork.

Topics that might come under this theme include (but are not limited to):
- innovative practices in interviewers' qualification and training (general, project-specific, and refresher);
- procedures of monitoring and evaluating interviewers’ job performance (in particular, detecting interviewers' deviations);
- analysis of interviewers’ behaviour during survey introduction and while asking questions/recording answers;
- interviewers’ attitude toward their job (specifically the difficulties they encounter while administering the survey, and the solutions they implement in order to overcome these problems).

Paper Details

1. A new method for the analysis of interviewer variance, with an empirical application
Professor Patrick Sturgis (University of Southampton)
Dr Ian Brunton-smith (University of Surrey)
Dr George Leckie (University of Bristol)

In this paper we set out a new method for the analysis of interviewer variance; the mixed-effects location scale model extends the random part of a standard 2-level mixed effects model by introducing random effects on the level 1 variance (Leckie, 2014). This enables response variance to be re-parameterized in a way that affords a more flexible and causally focused assessment of the factors associated with interviewer-induced response variability. We apply this approach to data from wave 3 of the UK Household Longitudinal Survey (UKHLS), which we link to a diverse range of interviewer characteristics.

2. The Impact of ‘Third-Party Help’ on Data Quality in Face-to-Face Interviews
Dr Melissa Quetulio-navarra (Wageningen University )
Dr Wander Van Der Vaart (University of Humanistic Studies, Utrecht)
Professor Anke Niehof (Wageningen University)

In some survey research settings it may be not attainable to interview individual respondents without involving bystanders or ‘third-parties’ in the interview. Due to complex living circumstances or group culture, respondents may be ‘helped’ by others in answering questions. Involvement of ‘third parties’ raises questions on data quality and poses challenges to the data collection process. Recognizing this, a natural field experiment was performed in the Philippines that allowed for spontaneous “third party help” during parts of the interview. Results showed that “third party help” did not negatively affect data quality, but rather improved it for most issues.

3. Standardised Interviewer Training, Accreditation and Quality Procedures on the Millennium Cohort Study Age 14 survey
Ms Nickie Rose (Ipsos MORI, London, UK)
Ms Anne Conolly (Ipsos MORI, London, UK)
Dr Rachel Williams (Ipsos MORI, London, UK)

Ensuring that interviewers work in a standardised way is key to achieving high quality survey data. Survey researchers use various techniques both during interviewer training and fieldwork to keep the research process standardised, but these techniques are constantly evolving and will depend on the particular requirements of the study. This paper reports on innovations in the standardised interviewer training that was designed for the sixth wave of the UK Millennium Cohort Study (MCS), the interviewer accreditation process, and the fieldwork quality and monitoring procedures that have been put into place.

4. Modifying Interviewer Strategies to Reduce Cost of Data Collection
Dr Rachael Walsh (U.S. Census Bureau)
Mr Scott Boggess (U.S. Census Bureau)

In an attempt to reduce the cost of data collection while maintaining data quality, the National Ambulatory Medical Care Survey (NAMCS) staff reviewed the fieldwork paradata and designed an experiment to test new data collection procedures in 2015. The testing protocol requires interviewers to first attempt to contact the physician’s office over the telephone, thereby reducing the number of personal visits required to complete the CAPI interview. This paper discusses the analytics supporting the protocol design, and the procedures used to monitor and evaluate interviewers’ compliance. Monitoring interviewers’ strategies enables the detection of deviation from the testing protocol.

5. Using panel data to identify interviewer effects? A comparison with common interviewer effect identification strategies
Mr Simon Kühne (Socio-Economic Panel (DIW Berlin))
Professor Martin Kroh (Socio-Economic Panel (DIW Berlin))

Past studies generally applied three main strategies to identify interviewer effects: analyzing the influence of observable interviewer characteristics on responses to related questions, estimating intra-interviewer correlation, and using data quality indicators. We propose a novel strategy based upon longitudinal data on respondents and interviewers from the Socio-Economic Panel Study (SOEP). We argue that changes in responses accompanying a change in interviewer serve as an indicator of interviewer effects. To evaluate the new approach, we address differences between identification strategies and discuss their respective (dis-)advantages and peculiarities.