Third-Party Presence in Face-to-Face Surveys |
|
| Chair | Ms Magali Rheault (Gallup ) |
It has long been acknowledged that face-to-face interviews should ideally involve only interviewer and respondent, without the presence of others. Third party presence may compromise data quality as respondents may refuse answers to sensitive questions or questions relating to the present person or tailor their answer to the present person. However, research on third party effects is scarce and results are mixed. A weakness of most of these analyses is that they are based on cross-sectional data and thus results may be biased by unobserved heterogeneity.
In our presentation, we focus on the effect of partner presence on responses to questions on the division of household labor. Existing literature shows such effects. Results might, however, be biased due to unobserved heterogeneity. If third party presence depends on unobserved factors also affecting survey responses, over- or underestimation of third party effects may be the result. The spouse might for example be more likely present during the interview in more conservative couples so that responses on items measuring the division of labor may not (only) be affected by spousal presence but also depend on their (unobserved) conservative values associated with a more conservative division of labor. A positive estimation bias of the effect of partner presence on own share of household labor reported by women would result as well as a negative bias of the effect of partner presence on own share of household labor reported by men.
We use the first seven waves of the German Family Panel (pairfam), a large, nationwide randomly sampled panel study with annual interviews, to investigate whether respondents’ answers to five items regarding the division of household chores are affected by the presence of the partner. To rule out that estimates of the effects of partner presence are biased by time time-constant unobserved heterogeneity we apply fixed-effects models which take advantage of intra-individual changes of partner-presence over the waves. The analysis includes more than 4,500 respondents who had the same partner in at least two panel waves. In approximately 30 % of the partnerships the partner was present in at least one of the seven panel waves (and not present in another wave).
The Centro de Investigaciones Sociológicas is a public institution that carries out around 40-50 political and social attitudes face to face surveys every year, including a monthly barometer. Most of its surveys are face-to face. Since 2008, all of its surveys include a question addressed to interviewers to check whether there are third-persons persons present and whether they actively participate in any way during the survey.
In the paper/presentation the effect of third parties being present during survey interviews will be analyzed at three different levels: survey, question and individual. The first exercise will consist on an exploratory analysis on the incidence of third party presence/interference, to see its evolution in time and to check on whether some pattern can be discerned. The main topic of each of the surveys will be coded in order to measure the extent to which third party presence is associated with the type of survey topic, and the potential influence of other survey level characteristics, such as interview duration or place where the interview takes place (house door, indoors, etc.) will also be analyzed.
The second part of the paper/presentation will look at the impact of third party presence/intervention depending on type of question and to the individual characteristics that might make third party presence more likely (socioeconomic status, household composition, among others). The data from the face-to-face Spanish General Social Survey (Encuesta Social General de España, ESGE 2013 edition), with a full probability sample of 5,094 respondents will be used for this purpose. It is a good survey to test the effect of third parties presence since its questionnaire contains both a large set of “factual” questions about the interviewee, his/her partner and family composition and also some sensitive questions about ideology and vote recall, whose results are more prone to be influenced by third party presence. This combination of questions is thus a good opportunity to check whether findings on previous analysis of third party presence/interference are applicable in Spain both regarding “factual” and sensitive questions (Aquilino 1993, Aquilino&Loscuito 1990, Smith 1995, Evans et al 2011). This survey´s questionnaire also contains a couple of more detailed questions about third parties that find out about presence, actual interference, and an additional question that inquires about who that person was in particular.
References:
Aquilino, W. S. (1993): “Effects of spouse presence during the interview on survey responses concerning marriage”, Public Opinion Quarterly, 57, 358 –376.
Aquilino, W. S., & LoScuito, L. A. (1990): “Effect of interview mode on self-reported drug use”, Public Opinion Quarterly, 54, 362–395.
Evans, Ann, Gray, E. & Reimondos, A. (2011), “Relationship reports: The effect of presence of others during the interview”, paper presented at the HILDA conference 14th July 2011 Ann Evans, Edith Gray and Anna Reimondos
Smith, Tom W. (1995): The impact of the presence of others on a respondent´s answers to questions, NORC GSS Methodological Report 86.
The presence of a third party during the survey interview generates an important contextual factor that could affect the reporting of sensitive information (Aquilino, 1997; Casterline and Chidambaram, 1984; Pollner and Adams, 1994). In spite of being instructed to conduct interviews in a private setting, interviewers can find it difficult to establish complete privacy. Previous research has found significant variation in the privacy achieved across interviewers (Mneimneh et al., 2013, 2015). While some interviewers have greater success in limiting the presence of others during the interview, others do not. Less understood are predictors of such interviewer variation and how they affect the association between third person presence and the reporting of sensitive information. The effects of third person presence on reporting sensitive information is also dependent upon the accuracy of recording of such presence by the interviewer. Generally, the presence of a third party is noted by the interviewer at the end of the survey interview. However, these observations may be subject to measurement error. Previous literature is silent about the quality of such observation measures.
In this presentation, we address these gaps in the literature and extend the research on third party presence during face-to-face interviews by using data from a recent face-to-face survey of a nationally representative sample in Jordan. The questionnaire collects information on a number of sensitive items related to political and religious attitudes. Information about third person presence came from observations recorded multiple times during the interview, at the end of a number of questionnaire sections, as well as at the end of the questionnaire as an overall measure. The end of the interview measure is the standard method used across surveys for collecting observation on third party presence during the interview. In addition to the commonly used measures of interview privacy (who was present and duration of presence), detailed data were collected on the specific situation that lead to the presence of others during the interview. Moreover, interviewers’ socio-demographic characteristics, including gender, age, education, religious attire (the presence of a veil for females), and previous interviewing experience were recorded. Additional measures related to interviewer’s attitudes toward privacy in general and towards obtaining privacy in the survey interview were collected using a self-administered questionnaire completed by all interviewers prior to data collection.
Using these data, this investigation extends the literature by exploring the quality of third party presence observations. Section-specific measures will be compared to the standard end of the interview measures. Predictors of each of the two types of measures (including interviewer-level variables) will be explored. Finally, the predictive power of each of the types of measures will be tested by investigating their effect on reporting sensitive information.