Factorial Surveys - Methods and Applications 4 |
|
| Session Organisers | Dr Lena M. Verneuer (University of Bielefeld) Professor Stefanie Eifler (Catholic University of Eichstätt-Ingolstadt) Dr Hermann Dülmer (University of Cologne) |
| Time | Friday 19th July, 09:00 - 10:30 |
| Room | D23 |
Since vignette-designs (e.g. factorial surveys; scenario techniques) as indirect measurement
techniques are very common in the social sciences by now, many different applications
can be found. Depending on theoretical and methodological objectives, the
applied techniques vary in a broad range and lead to different and sometimes inconsistent
results. Due to this diversity, findings on methodological and substantial issues
can have different meanings and impacts for further research.
This session chooses one way of anticipating the divers field of factorial surveys and
vignette-designs in general and aims at shedding light on the stage of affairs by discussing
recent developments and pooling new findings of projects that try to enrich the
discussion. The focus of the session is explicitly broad and all contributions dealing
with different analytical strategies, empirical designs or substantial research that make
use of factorial surveys or other vignette-designs are welcome. Papers matching one of
the following aspects are cordially invited to be part of this session:
• comparison and discussion of design-related questions regarding methodological
or substantive aspects,
• new developments in measuring intentions with vignettes,
• theoretical ideas for modelling the relationship between intentions and behavior
for further empirical analyses,
• cross validation strategies (new approaches, replications),
• discussion of (dis-)advantages of vignette-designs, validations strategies and/or
measures,
• issues of data-collection,
• substantive applications of factorial surveys
Keywords: Factorial Surveys; Vignette-Designs;
Mr Philip Adebahr (University of Technology Chemnitz) - Presenting Author
Professor Peter Kriwy (University of Technology Chemnitz)
Factorial Surveys commonly use textual vignettes. These types of vignettes can face some problems for example, respondents do not understand words equally or the wording influences the answers. Vignettes that intent to vary bodyweight and use words like “fat”, “obese” or “high weight” do not take into account, that respondents perceive different body sizes as “high weight”. On the other hand BMI, as a more objective category, is difficult to assess by respondents. Instead, body images provide more information and thus are more objective and less misleading. Because studies using images are rare, there is a lot of basic work to do. In our presentation we approach the question, how valid images are regarding the measurement of body weight in vignettes? To answer this question, we used a study from 2017 with 40 respondents of the general population and 60 who participated in a weight loss program, each of which answered 10 vignettes. In the personal questionnaire, the respondents were asked about their own body weight, height and they also assigned themselves to five body images. Three of this images (slight overweight, obese, heavily obese) were also used in the vignettes. The comparison of the (self-reported) BMI with the body images shows significantly high criterion validity. We therefore assume that respondents discriminate significantly between the images in the vignettes. The use of body images has proven successful in this study and can be recommended for factorial surveys that need operationalization of body weight.
Miss Livia Ridolfi (Università Milano Bicocca) - Presenting Author
Extensive literature shows that women prioritize social status and resources while men prioritize attractiveness and youth in selecting a long-term partner. In this regard, a biosocial debate discusses to what extent heterosexual mating preferences are associated with biological and social constraints, but empirical research lacks experimental research technique privileging self-reporting survey. Moreover, the association between gender egalitarianism values and mating preferences is still not clear even among highly educated people.
This study aims to investigate whether mating preferences vary between sexes among highly educated students of the University of Milano—Bicocca. A full factorial survey design was set up by defining fictive profiles of potential partners with a set of biological (age, height, facial beauty) and socio-economic traits (educational level, wealth, career prospects, and religious values). Face pictures were previously selected on the basis of their attractiveness level; all the other traits were listed schematically. Trough an online survey, 243 female and 243 male students – all Italian, born between 1995 and 1997, cisgender and heterosexual -- were tasked to rate six random profiles each. Data on demographics and gender egalitarianism values were also collected.
Multilevel analysis results show that, on average, women are stricter in ratings and show stronger preferences for a partner's wealth and career prospects, compared to men. Educational level and religious values are, on average, valued the same by women and men. Facial beauty is essential for both sexes, but their preference for height goes in the opposite direction. Further analyses do not show significant interactions between respondents' preferences and gender egalitarianism values.
Dr Steffen Lepa (Hanover University of Music, Drama and Media) - Presenting Author
Dr Hauke Egermann (University of Yorck)
Mr Martin Herzog (Technische Universität Berlin)
Dr Jochen Steffens (Technische Universität Berlin)
Mr Andreas Schönrock (Humboldt Universität Berlin)
Within a cross-national research and development project, funded by the Horizon 2020 program of the European Community, our research group's task was to develop a fully-automatic music recommender for music branding scenarios (e.g. music played at the point of sale or on corporate websites), providing capacities that significantly differ from traditional music recommenders: In such a scenario, the perceived semantic expression of music titles is of main interest since it has to meet marketing intentions, whereas consumers’ personal preferences, as hitherto researched in music sociology and psychology, are of minor importance. In a first step, we thereto drew on a factorial survey design with n=3485 participants from three European countries that were listening to 183 different popular music titles and rated them on 51 expression items that had been developed by an expert focus group and survey. This procedure was done in order to develop a multi-lingual factor measurement instrument for branding-relevant musical expression (drawing on EFA/CFA). The resulting 4-factor solution could be successfully validated in a second wave with n=6659 subjects and 366 other music titles, this time employing exploratory structural equation modeling in order to arrive at orthogonal factor scores. In a second step we then used music information retrieval and machine learning techniques to render algorithmic audio descriptors for all the music titles presented in both survey waves. We then tested different variants of regression models for their ability to translate popular music's structural attributes and perceptual cues into mean scores on each of the four music expression factors. In result, the recommender drawing on four hierarchical regression models is able to predict collectively attributed meanings towards popular music titles with an accuracy from R²=40-71% (hold out sample) to R²=76-83% (full sample).
Professor Gunnar Otte (Johannes Gutenberg University Mainz) - Presenting Author
Mr Dave Balzer (Johannes Gutenberg University Mainz)
Mrs Luisa Wingerter (Johannes Gutenberg University Mainz)
The arts are only rarely the topic of survey research. If they are studied, rather simple meas-urements prevail. In the so far largest survey on cultural participation in Germany, which was fielded in 2018 and draws on a random sample of the general population aged 15 years and older, several innovative measurement instruments have been implemented, among them a factorial survey module. To our knowledge, it is the first factorial survey application in this field of research. The module is administered to the subsample of respondents who visited a play at least once in the last 12 months. The vignettes are short descriptions of per-formances and are intended to model facets that are contentious among the drama audience. They contain five dimensions: (a) large, festive vs. small, sober type of stage; (b) cheerful vs. thought-provoking play; (c) classic vs. new material in an original vs. modern produc-tion; (d) being critical of society; and (e) displaying nudity and violence. For each portrayed performance, respondents are asked to indicate their attitude towards public funding on an 11-point Likert scale. A special feature of our factorial survey design is that two more di-mensions are added to each vignette in a second step: (f) admission price; (g) positive vs. controversial critical reviews. After presenting these additional dimensions, respondents are asked to indicate, on the same rating scale, whether they are willing to visit the performance. In our D-efficient experimental design, each respondent evaluates one out of twelve decks with six vignettes. We employ multilevel modelling to analyze the effects of vignette di-mensions and respondent characteristics on our two target variables and to investigate the main social cleavages in the drama audience.