Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Thursday 16th July, 11:00 - 12:30 Room: L-102

Multifactorial Survey Experiments (Factorial Surveys, Choice Experiments and Conjoint Analysis) 2
Convenor	Professor Katrin Auspurg (Goethe-University Frankfurt )
Coordinator 1	Dr Carsten Sauer (Bielefeld University)
Coordinator 2	Professor Peter M. Steiner (University of Wisconsin-Madison)

Session Details

There is a fast growing trend in the social sciences to combine the advantages of multifactorial experimental designs with surveys. Factorial surveys – often labelled as “vignettes studies” – have been used for more than 30 years to gather data on descriptions of hypothetical situations or objects to explore principles of judgment and decision making. Choice experiments help to explore respondents’ preferences and willingness to pay. In addition there is increasing use of conjoint analyses in sociology and political sciences. The experimental design provides a high internal validity, while the survey design improves external validity. Computer assisted interviewing that allows implementing many different treatments have made these methods even more popular.

Despite frequent use there are still many open questions concerning design features of multifactorial survey experiments that offer most reliable and valid results. We are interested in methodological research on the design of factorial surveys, choice experiments or conjoint analyses (e.g., validity of tabular vs. text presentations, video-vignettes), sampling techniques to select the experimental treatments (random sampling vs. fractional sampling), analysis strategies (e.g. accounting for the multi-level structure of response data; testing validity in regard to respondents’ attitudes, beliefs, and behavior).


Questions could be:
Design of questionnaire: How to present the vignettes to the respondents? What kind of answering scale provides most valid results? How to prevent order effects? How do respondents cope with the information provided on vignettes or choice sets?
Sampling techniques: What are the benefits and drawbacks of fractional versus random sampling? How do sampling techniques like blocking by respondent strata and interviewers influence efficiency of estimates?
Analysis strategies and validity: Which models provide unbiased estimates? How to address possible censoring of responses? Respondents’ idiosyncracies? When to use multilevel analyses, and how to validate results?

Paper Details

1. Interval level measurement of preferences using factorial surveys with attribute difference scaling
Mr Volker Lang (Bielefeld University)
Professor Martin Groß (University of Tübingen)

We introduce a new procedure for the interval level measurement of preferences in factorial surveys: The attribute difference scaling method which enables the implementation of factorial surveys without assuming inter-individual homogeneous rating behavior. This study compares an attribute scaled and a standard factorial survey with respect to their interval scaling quality and their out-of-sample predictive validity regarding a set of discrete choices. Furthermore, this comparison is carried out with four different rating measurement instruments, enabling a comprehensive assessment of instrument effects. As test case a five dimensional, full factorial design measuring students’ preferences regarding internships is used.

2. Hiding Sensitive Topics by Design? An Experiment on the Reduction of Social Desirability Bias in Factorial Surveys
Professor Thomas Hinz (University of Konstanz)
Mrs Sandra Walzenbach (University of Konstanz)

It has been argued that factorial survey designs can help to overcome social desirability bias when potentially sensitive dimensions are ‘hidden’ in vignette texts. In order to evaluate which implementation works best for sensitive topics, we implemented a split half experiment in an online factorial survey module, which was answered by a general population sample. Our vignette module assesses respondents’ judgments on just fees for early childcare and includes the child’s religious denomination (Christian, Muslim, none) as a sensitive dimension. This set up enables us to compare the widely used within-respondent variation with an alternative between-respondent implementation.

3. Using Factorial Surveys and Stated Choice Experiments to Investigate Discriminatory Attitudes and Preferences
Dr Heiko Beyer (University of Wuppertal)
Professor Ulf Liebe (University of Bern)

Empirical research on discrimination grapples with the social undesirability of its object. In many studies using regular survey methods, estimates are biased, and the social context of discrimination is not taken into account. Several methods have been developed, especially to deal with the first problem. In this regard, the estimation of the ‘true value’ of discriminatory attitudes is at the centre of interest. However, methodological contributions focusing on the social context of attitude communication and discriminatory behaviour, as well as the correlation between both, are rare. We present two experimental methods which address those issues: factorial surveys and choice experiments.

4. Elimination and Selection by aspects decision rules in discrete choice experiments
Dr Seda Erdem (University of Stirling)
Dr Danny Campbell (University of Stirling)
Professor Carl Thompson (University of York)

This paper proposes a flexible method for exploring and accommodating situations where respondents exhibit elimination and selection by aspect decision rules in discrete choice experiments, whilst addressing preference heterogeneity. We present an empirical case study on the public's preferences for health service innovations. We show that allowing for elimination-by-aspects and/or selection-by-aspects behavioural rules leads to substantial improvements in model fit and, importantly, has implications for willingness to pay estimates and scenario analysis.

5. Aspects of Validity: Scenario-Technique, Self-Report & Social Desirability
Mrs Lena Verneuer (University of Bielefeld)

The panel study 'Crime in the modern City' focusses on the emergence and development of deviant and delinquent behaviour of adolescents. A verbal scenario is implemented in the questionnaire since 2013. It serves as a measurement for (hypothetical) reactions to deviant behaviour in a specific conflictual situation. As this type of measurement should trigger certain scripts for reactions, the evocation of external social norms, carrying aspects of Social Desirability is possible, too. With the application of Latent Class Analysis this potential response bias can be anticipated: It gives the opportunity to separate honest from desirable answering patterns for further analyses.