ESRA 2025 Preliminary Program
All time references are in CEST
Interviewers across the Survey Life Cycle 2 |
| Session Organisers |
Dr Mariel Leonard (DIW-Berlin)
Dr Zachary Smith (National Center for Health Statistics (NCHS))
|
| Time | Tuesday 15 July, 15:30 - 17:00 |
| Room |
Ruppert 011 |
Interviewers are central to survey operations. From qualitative question design and evaluation – cognitive interviewers, focus group moderators, and even expert reviewers – to quantitative survey administration in the field. A large body of literature has identified various ways the identity, behavior, and disposition of interviewers influence the quality of data collected. And, a growing consensus is developing that in both qualitative and quantitative aspects, interviewers should be understood not merely as mindless, faceless data collection machines, but as researchers that contribute to the research process. Indeed, the consequences of ignoring interviewers’ humanity and research capabilities may be particularly important for data quality, as research on interviewer effects has already shown.
This panel invites contributions addressing either qualitative pre-fielding or quantitative survey administration that consider:
1. Whether and how best interviewers can be incorporated into the research team;
2. How interviewers affect the quality of data collected (positively or negatively);
3. How interviewers navigate complex issues, for example, sensitive survey topics or respondents unaccustomed to the survey process;
4. Considerations of the “interviewer” in a self-administered context.
Keywords: interviewers, interviewer effects, cognitive interviewing, data quality, sensitive topics
Papers
Interviewer Background and Its Influence in Interviews with Muslims
Ms Katrin Pfündel (Federal Office for Migration and Refugees) - Presenting Author
Dr Anja Stichs (Federal Office for Migration and Refugees)
Dr Amrei Maddox (Federal Office for Migration and Refugees)
Muslim immigrants are a particular hard-to-reach group in survey research, often facing challenges related to trust, language barriers and cultural differences, exacerbated by experiences of social exclusion and discrimination within this group. Co-ethnic interviewers might be able to surmount these hurdles. This study, thus, investigates the effect of interviewer characteristics, particularly the use of foreign interviewers, on survey responses among Muslim immigrants.
In our article, we present the design of the Muslim Life in Germany 2020 study as an example of best practice for generating reliable data on Muslims. The study involved more than 4,500 face-to-face interviews with immigrants and their descendants from 23 Muslim-majority countries, including 3,472 Muslims. In view of the aforementioned obstacles to interviewing Muslims in Germany, alongside native interviewers also Syrian interviewers were sent into the field. As a result, 43% of interviews with Muslims were conducted by these Arabic-speaking interviewers.
In terms of investigating interviewer effects, we apply bivariate and multivariate analyses to the resulting data to examine how the nationality, ethnic background and language of interviewers may influence the willingness of immigrants and their descendants from Muslim-majority countries to take part in surveys and their response behaviour to culturally sensitive questions about religion and social cohesion. We thus investigate the potential of interviewers who share similar cultural or religious backgrounds with respondents to increase participation rates and reduce the share of missing values.
Mixed Method Research of Field Representative Feedback on the National Health Interview Survey Follow-up Health Study
Dr Rodney Terry (U.S. Census Bureau) - Presenting Author
For the U.S. National Center for Health Statistics, the U.S. Census Bureau conducted focus group and survey research of Field Representatives’ (FR) feedback on their experience asking respondents about the U.S. National Health Interview Survey (NHIS) Follow-up Health Study (FHS). As part of the NHIS interview, FRs described the study and asked a sample of NHIS adult respondents if they were willing to be contacted to schedule an appointment to participate in the FHS. The study protocol consisted of a home health exam where a trained health representative collected physical measurements (e.g., height) and blood and urine for testing. The purpose of the focus group and survey research was to collect feedback from Field Representatives who asked NHIS respondents about the NHIS FHS pilot project to identify strategies that may help increase FHS participation. We conducted five focus groups with 35 FRs, followed by an online survey of 71 FRs. Results showed that the FHS introduction was easy to learn and administer, and that the training and FHS brochure were helpful learning tools for both the FRs and respondents, respectively. However, FRs reported that NHIS respondents said the study was very invasive on the body and time consuming. Further, while the incentive was helpful for gaining the cooperation of many respondents, it was not effective in motivating reluctant respondents and respondents from high income households. To improve the study, we recommended that NCHS investigate several strategies, including increasing the incentive, notifying and educating the respondent about the FHS study earlier in the interview or before the FR’s visit, and offering alternate forms of participation. The results of this research will help inform future iterations of the FHS and provide further insights into how interviewers navigate a complex and sensitive issue.
The Importance of Interviewer Feedback on Survey Design: Results and Interviewer Perspectives
Mr Douglas Williams (US Bureau of Labor Statistics) - Presenting Author
Dr Robin Kaplan (US Bureau of Labor Statistics)
Dr Erica Yu (US Bureau of Labor Statistics)
Survey data collectors, or interviewers, are a key component of the survey lifecycle. While surveys conducted via self-administered approaches, such as paper or web, continue to proliferate to mitigate increasing costs, the most critical and complex data collections continue to have a strong reliance on interviewers. This is generally to assist with complex topics, or to negotiate a common understanding of ambiguous responses that interviewers code on the spot to discrete response options. However, interviewers can be a rich and often overlooked resource for understanding the survey process, such as identifying respondent difficulty, poor questions, burden, or frequently misunderstood concepts within the questionnaire. Although interviewers may be capable of navigating such difficulties on their own, such non-standardized procedures can lead to interviewer and mode effects. In the growing era of mixed-mode surveys, relying on interviewers to remedy these issues can be short-sighted. In this paper we discuss our process for collecting qualitative feedback from interviewers collecting data for the Current Population Survey (CPS), the national labor force survey in the United States. The CPS is in the midst of a modernization effort, which will add a web mode in addition to personal visit and telephone collection. This effort began with extensive pretesting that identified potential challenges and break-downs of the cognitive response process for some CPS concepts. We then leveraged these insights to collect additional feedback and a new perspective from a large-scale self-administered survey of CPS interviewers. Information was collected on which questions posed the most respondent difficulty, concepts that required clarification, methods used to overcome difficulty, and approaches to hypothetical vignettes depicting complex interactions. We share some results, focusing on feedback collected from interviewers on how they address interview issues and how such feedback is important to the survey modernization process.
