ESRA 2025 Preliminary Program

Monday 14 July    Tuesday 15 July    Wednesday 16 July    Thursday 17 July    Friday 18 July   

All time references are in CEST

Cheating in Indirect Questioning Sensitive Surveys
Session Organiser	Professor Pier Francesco Perri (University of Calabria, Italy)
Time	Tuesday 15 July, 09:00 - 10:30
Room	Ruppert 111

In surveys on highly personal, sensitive, or potentially incriminating topics, conventional direct questioning methods are known to elicit high rates of nonresponse and dishonest answers, thus impairing data quality and biasing prevalence estimates of the population characteristics of interest. While this issue cannot be entirely eliminated, it can potentially be mitigated by increasing respondents' willingness to cooperate.
Following this rationale, indirect questioning techniques (IQTs) have recently gained popularity as an effective means for obtaining truthful responses to sensitive questions by guaranteeing participants the confidentiality of their answers. Importantly, the validity of most IQTs depends on two key assumptions about participants' behavior, namely that they (1) are completely honest in their responses, and (2) fully understand and accurately follow the instructions of the chosen method. However, empirical evidence suggests that even when granted full confidentiality by an IQT, some participants "cheat" the procedure either by providing a self-protective response due to insufficient trust towards the method, or by noncompliance with the instructions due to insufficient understanding. Such cheating behaviors introduce additional sources of bias that must be controlled for if accurate estimates are to be obtained.
The current session aims to improve both the theoretical and practical understanding of cheating behavior in sensitive surveys. We welcome contributions that: (1) improve established techniques to reduce cheating behavior; (2) propose new methods for detecting cheaters; (3) adjust prevalence estimates for sensitive characteristics by accounting for cheating behavior; and (4) distinguish between intentional cheating and unintentional noncompliance with instructions.

Papers

Modeling Evasive Response Bias In Randomized Response: Cheater Detection Versus Self-Protective No-Saying

Dr Khadiga Sayed (Utrecht University ) - Presenting Author
Dr Maarten Cruyff (Utrecht University )
Professor Peter van der Heijden (Utrecht University )

Randomized response is an interview technique for sensitive questions designed to eliminate evasive
response bias. Since this elimination is only partially successful, two models have been proposed for
modeling evasive response bias: the cheater detection model for a design with two sub-samples with
different randomization probabilities and the self-protective no sayers model for a design with multiple
sensitive questions. This paper shows the correspondence between these models, and introduces models
for the new, hybrid “ever/last year” design that account for self-protective no saying and cheating. The
model for one set of ever/last year questions has a degree of freedom that can be used for the inclusion of
a response bias parameter. Models with multiple degrees of freedom are introduced for extensions of the
design with a third randomized response question and a second set of ever/last year questions. The models
are illustrated with two surveys on doping use. We conclude with a discussion of the pros and cons of the
ever/last year design and its potential for future research.

Substance or Noise? An Examination of the Retest Stability of Responses in the Extended Crosswise Model

Mrs Julia Meisters (Heinrich Heine University Duesseldorf) - Presenting Author
Mr Adrian Hoffmann (Heinrich Heine University Duesseldorf)
Mr Jochen Musch (Heinrich Heine University Duesseldorf)

Indirect questioning techniques are designed to provide more accurate prevalence estimates for sensitive attributes than conventional direct questions (DQ). However, sometimes respondents do not adhere to the instructions of indirect questioning techniques, for example due to a lack of understanding or trust in the procedure, thereby compromising the validity of the estimates. One type of instruction non-adherence that has been the focus of several studies is non-systematic random responding. In particular, the Extended Crosswise Model (ECWM; Heck et al., 2018), an advanced indirect questioning technique, has been criticized for potentially encouraging such non-systematic random responding behavior. If in fact such behavior occurs to a considerable extent, it would lead to temporally inconsistent individual responses and thus to reduced retest stability. This study is the first to assess individual response consistency in the ECWM and compare it with that of conventional DQ. With a retest interval of approximately 10 days, we asked N = 2,317 mothers whether they had smoked during a previous pregnancy, using the same randomization device in the ECWM condition at both time points. In both conditions, the majority of respondents provided consistent individual responses (ECWM: 89%, DQ: 95%), and consequently the prevalence estimates remained almost identical over time. These results suggest that response behavior remains stable in the ECWM, which is inconsistent with the assumption that respondents predominantly answer ECWM questions randomly.

“More or less is better” – an advanced validation strategy for disentangling a successful control of social desirability bias and random responding in indirect questioning surveys

Dr Adrian Hoffmann (Heinrich Heine University Duesseldorf) - Presenting Author
Dr Julia Meisters (Heinrich Heine University Duesseldorf)
Professor Jochen Musch (Heinrich Heine University Duesseldorf)

In comparative validation studies, the validity of prevalence estimates obtained using indirect questioning techniques (IQTs) is typically determined by their relative magnitude compared with estimates from a direct questioning (DQ) control condition. Estimates based on IQTs are considered more valid if they are higher than DQ estimates for socially undesirable attributes (“more is better” assumption), or lower for socially desirable attributes (“less is better” assumption), as they are potentially less distorted by the influence of social desirability bias. More recently, several critical publications have raised the question of whether the promising results observed in comparative validations of IQTs may have been primarily due to a form of participant noncompliance, i.e., random responses due to lack of trust or understanding. However, this question cannot be answered on the basis of the available data, as comparative validations usually do not allow to distinguish between successful control of social desirability bias and the influence of random responses. To address this issue, we present an advanced validation strategy that relies on an integrated “more or less is better” assumption by orthogonally manipulating the direction of social desirability (undesirable vs. desirable) and the prevalence (high vs. low) of several sensitive attributes. We apply this strategy to an empirical validation of the Extended Crosswise Model (ECWM; Heck et al., 2018), a current IQT with an additional mechanism for detecting instructional noncompliance. Our results generally indicate that the ECWM successfully controls for social desirability bias. While our results do not rule out a small proportion of random responses, particularly when examining socially undesirable attributes with high prevalence, they do rule out random responses as a major factor that can provide a sufficient alternative explanation for previous findings attesting to the improved validity of the ECWM compared to DQ estimates.

“More or less is better” – an advanced validation strategy for disentangling a successful control of social desirability bias and random responding in indirect questioning surveys

Dr Adrian Hoffmann (Heinrich Heine University Duesseldorf) - Presenting Author
Dr Julia Meisters (Heinrich Heine University Duesseldorf)
Professor Jochen Musch (Heinrich Heine University Duesseldorf)

In comparative validation studies, the validity of prevalence estimates obtained using indirect questioning techniques (IQTs) is typically determined by their relative magnitude compared with estimates from a direct questioning (DQ) control condition. Estimates based on IQTs are considered more valid if they are higher than DQ estimates for socially undesirable attributes (“more is better” assumption), or lower for socially desirable attributes (“less is better” assumption), as they are potentially less distorted by the influence of social desirability bias. More recently, several critical publications have raised the question of whether the promising results observed in comparative validations of IQTs may have been primarily due to a form of participant noncompliance, i.e., random responses due to lack of trust or understanding. However, this question cannot be answered on the basis of the available data, as comparative validations usually do not allow to distinguish between successful control of social desirability bias and the influence of random responses. To address this issue, we present an advanced validation strategy that relies on an integrated “more or less is better” assumption by orthogonally manipulating the direction of social desirability (undesirable vs. desirable) and the prevalence (high vs. low) of several sensitive attributes. We apply this strategy to an empirical validation of the Extended Crosswise Model (ECWM; Heck et al., 2018), a current IQT with an additional mechanism for detecting instructional noncompliance. Our results generally indicate that the ECWM successfully controls for social desirability bias. While our results do not rule out a small proportion of random responses, particularly when examining socially undesirable attributes with high prevalence, they do rule out random responses as a major factor that can provide a sufficient alternative explanation for previous findings attesting to the improved validity of the ECWM compared to DQ estimates.

Examining Public Support for the Death Penalty in Taiwan: Insights from an Indirect Questioning Technique Survey

Dr Shu-Hui Hsieh (Research Center for Humanities and Social Sciences, Academia Sinica) - Presenting Author
Dr Adrian Hoffmann (Department of Experimental Psychology, University of Duesseldorf )

Public opinion on the death penalty has been widely studied in the USA and parts of Asia, including China, but there is little empirical research on this issue in Taiwan. This paper examines public attitudes toward the death penalty in Taiwan, where it remains a legal punishment for offenses such as murder, treason, drug trafficking, and certain military crimes. However, since the early 2000s, executions have been carried out exclusively for murder. Although legal controversies in the 1990s and shifting official attitudes toward abolition led to a decline in executions, our 2024 web survey reveals that public support for the death penalty remains overwhelmingly high when measured using conventional direct questioning (DQ). This study highlights the complexity of these attitudes, shaped by value conflicts and ambivalence, with traditional collectivist perspectives that emphasize the responsibility of the state to maintain law and order, in contradiction to increasingly influential individualist perspectives that emphasize the protection of each individual life.


To address the influence of social desirability bias (SDB) when surveying sensitive topics like support for abolishing the death penalty, this study employs the Cheating Detection Triangular Model (CDTRM), an indirect questioning technique. The CDTRM is compared to conventional DQ methods with respect to estimation validity, and additionally provides an estimate for the proportion of respondents who provide self-protective responses by disregarding instructions, using a dedicated model parameter. Our results indicate that the CDTRM effectively controls the validity-threatening influence of SDB in a survey on death penalty topic in Taiwan. Furthermore, its superiority in validity over conventional DQ and the utility of the CDTRM cheating detection mechanism, decrease when the instructions are easy to understand.