Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Social Desirability Bias in Sensitive Surveys: Theoretical Explanations and Data Collection Methods 1 |
|
| Convenor | Dr Ivar Krumpal (University of Leipzig) |
| Coordinator 1 | Professor Ben Jann (University of Bern) |
| Coordinator 2 | Professor Mark Trappmann (Institute for Employment Research Nürnberg) |
Survey measures of sensitive characteristics (e.g. sexual behaviour, health indicators, illicit work, voting preferences, income, or unsocial opinions) based on respondents' self-reports are often distorted by social desirability bias. More specifically, surveys tend to overestimate socially desirable behaviours or opinions and underestimate socially undesirable ones, because respondents adjust their answers in accordance with perceived public norms. Furthermore, nonresponse has a negative impact on data quality, especially when the missing data is systematically related to key variables of the survey. Besides psychological aspects (such as a respondent's inclination to engage in impression management or self-deception), cumulative empirical evidence indicates that the use of specific data collection strategies influences the extent of social desirability bias in sensitive surveys. A better data quality can be achieved by choosing appropriate data collection methodologies.
This session has three main goals: (1) discuss the theoretical foundation of the research on social desirability bias in the context of a general theory of human psychology and social behaviour. For example, a clearer understanding of the social interactions between the actors that are involved in the data collection process (respondents, interviewers, and data collection institutions) could provide empirical researchers with a substantiated basis for optimizing the survey design to achieve high quality data; (2) present experimental results evaluating conventional methods of data collection for sensitive surveys (e.g. randomized response techniques and its variants) as well as innovative and new survey designs (e.g. mixed-mode surveys, item sum techniques). This also includes advancements in the methods for statistical analysis of data generated by these techniques; (3) discuss future perspectives for tackling the problem of social desirability and present possible alternative approaches for collecting sensitive data. This may include, for example, record linkage approaches, surveys without questions (e.g. biomarkers), and non-reactive measurement.
One class of data collection strategies for eliciting sensitive information are so-called "dejeopardizing techniques", out of which the randomized response technique (RRT) is the best investigated one. Focusing on the RRT, this paper explores this method to improve the quality of data about sensitive labor market topics, such as receipt of basic income support. In a 2010 telephone survey (n=3,211), we experimentally tested two techniques for asking such sensitive questions: direct questioning and the randomized response technique.
First, we compare the percent of socially undesirable responses (indication of transfer payments, i.e. receipt of basic income support) across the two techniques. In addition, because the sampled persons were selected from German administrative records, we know the percent of respondents who have received transfer payments and thus the percent who should have reported receipt. Thus we can also validate the reported percent from each method against the known true rate for the responding cases hence assessing the bias of our estimates. Such administrative record data is quite rare in the literature on sensitive questions, and allows us a unique opportunity to evaluate the "more is better" assumption which is so often used in the literature. Being able to assess the amount of 'non-compliance' to the RRT instructions, insights into the functioning of the RRT also in specific sub-populations can be assessed using multivariate analyses. Thus this paper provides insights into a variety of practical and theoretical factors contributing to a successful implementation of the RRT.
Privacy protecting (PP) models for surveys have been developed since Warner (1965). Such methods (e.g. the randomized response technique) aim to protect the respondents' privacy and, thus, to improve the quality of the results one obtains, particularly in the case of gathering information on so-called sensitive characteristics (like, e.g., tax evasion). Common assumptions for PP techniques to work concern the respondents' cognitive skills and honesty. Under these conditions the techniques increase the rate of truthful answers and, therefore, the quality of the estimates. Evaluating PP methods by taking into account the respondents' behavior shows that these assumptions do not always hold. Nathan and Sirken (1998) approach this problem extending the classical randomized response technique by describing the respondents' answering process during a survey. However, being a purely qualitative approach, only few results (e.g. the effect of violating the PP methods' assumptions) can be deduced from the Nathan and Sirken model.
Therefore, in this talk we propose to fill this gap by adapting a discrete choice model (McFadden 1974) to describe the answering process. This model is based on the utility theory where individuals choose the set of alternatives maximizing their outcome. Our approach allows modeling the respondents' decision making process and analyzing the effect different survey methods have on their answers. Furthermore, we discuss possible consequences for the estimation of sensitive characteristics due to the violation of the PP methods' assumptions.
When using business surveys to study bribery, one must overcome the problem of acquiring valid data on such acts. Firms must be provided with an incentive to reveal their true behaviour if the data is to be trusted. This could be achieved by increasing the benefit accrued from truth-telling; or by reducing the perceived cost of truth-telling. Different mechanisms have been used to do this; these include the Randomised Response technique; and indirect questioning methods. These methods shield interviewees from being identified as guilty whilst providing some information about the level of guilt within the sample. This paper assesses the usefulness of these techniques in obtaining information about corruption and tax evasion amongst companies in Nigeria. Results from the randomised response procedure are compared with indirect questioning methods that are used to get truthful responses to sensitive questions. Through a model that allows for false-reporting this study calculates a lower bound estimate of the rate of truth-telling/lying. Results from previous studies [Azfar & Murrell , 2009, Jensen & Rahman , 2011] using data from other countries are compared with the results from the Nigerian sample. This paper finds that whilst the Randomised Response technique provides an incentive to tell the truth; many firms still choose not to do so. Furthermore, asking indirect questions about tax evasion and corruption seems to generate more honest answers than using the randomised response technique. Companies in Romania seem more willing to admit to guilt compared with companies in Nigeria.