All time references are in CEST
Vignette Experiments in Survey Research: Exploring Discrimination Mechanisms through Controlled Scenarios |
|
| Session Organisers | Professor Giovanni Busetta (University of Messina) Professor Maria Gabriella Campolo (University of Messina) Dr Louis Lippens (Ghent University) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
Discrimination remains one of the most pervasive and complex social challenges today. Vignette experiments offer a powerful methodological tool to investigate these phenomena in controlled settings, allowing researchers to manipulate relevant variables and observe participants' responses. This type of experiment provides significant advantages in measuring discriminatory attitudes and intentions that might not surface through traditional methods, such as questionnaires or correspondence experiments.
This session aims to explore the use of vignette experiments in social research to study the mechanisms of discrimination, focusing on grounds including race, gender, age, and sexual orientation. We will examine the methodological benefits of vignettes, such as the ability to systematically control contextual and personal factors, as well as the challenges related to their implementation, including the interpretation and generalizability of the findings.
We invite theoretical, methodological, and empirical contributions that use vignette experiments to investigate discrimination dynamics and their effects at individual and societal levels. The session aims to foster discussions on enhancing the quality of data collected through experiments, reducing Total Survey Error, and reflecting on how we can apply these techniques to future social research.
Keywords: Vignette experiments, Discrimination mechanisms, Implicit bias, Ethnicity, Gender
Professor Knut Petzold (Zittau/Görlitz University of Applied Sciences) - Presenting Author
Vignette experiments are increasingly used to investigate ethnic, religious or gender-based discrimination. Since hypothetical descriptions of people are presented, the method is considered suitable for collecting sensitive information in an ethically justifiable and unobtrusive manner. Discriminatory response behaviour can be causally attributed to the characteristics presented. However, the application is based on the assumption that the results obtained through vignette experiments allow conclusions to be drawn about actual discriminatory behavior in real situations, which has so far been inadequately tested.
We challenge this assumption as the results of a vignette experiment are being validated using a respective field experiment. By the example of international university graduates in the German labor market, who apply for job offers at a online-platform, a correspondence test among employers in different industries and labor market segments (n = 864) provides the behavioural data. We investigate to what extent possible discrimination rates in employers' responses observed in the field experiment can be replicated in a vignette experiment among the same employers, while ensuring the greatest possible comparability. We conclude with some implications on the predictive validity of vignette experiments when measuring ethnic discrimination.
Miss Louise Devos (Ghent University) - Presenting Author
Dr Louis Lippens (Ghent University)
Miss Dagmar Claus (Ghent University)
Professor Stijn Baert (Ghent University)
Regional mobility is crucial for addressing labour shortages. This study examines the influence of regional origin on hiring decisions in the Flemish labour market, focusing on perceptions of Flemish recruiters towards Walloon candidates. Through a vignette experiment, 81 Flemish recruiters—selected for their lack of strong social desirability bias—evaluate fictitious resumes of school leavers signalling regional origin through their name, places of birth and residence, secondary education, and language proficiency. Walloon candidates consistently score lower on key hiring metrics, with their perceived origin reducing interview likelihood by 12.74% on average. Structural equation modelling reveals that Flemish employers hold negative perceptions of Walloon candidates, particularly regarding availability, interpersonal competencies, attitude, and willingness of employers, employees, and clients to cooperate with them. These findings highlight the persistent role of regional stereotypes in reinforcing labour market inequalities.
Miss Louise Devos (Ghent University) - Presenting Author
Dr Louis Lippens (Ghent University)
Professor Stijn Baert (Ghent University)
Ethnic identity signals significantly influence employment decisions. This study investigates how multiple indicators of ethnic identity—including name-based cues (e.g., combinations of majority-sounding first names and foreign-sounding surnames, fully ethnic names, or majority-sounding names), migration background, and culture-specific extracurricular activities—affect recruiters’ assessments of job candidates in the Belgian context. We consider various ethnic backgrounds.
The experiment involves genuine recruiters who evaluate the likelihood of hiring fictitious candidates with diverse profiles for positions within their own companies. Furthermore, recruiters are also asked to predict their colleagues' responses, offering a more nuanced view of potential biases. Additionally, the study provides insights into the mechanisms underlying hiring discrimination. Recruiters assess candidates based on perceptions related to both taste-based and statistical discrimination, evaluating attributes like collaboration potential and productivity. We analyse how these signals interact with job characteristics such as customer contact, internal interaction, and whether the role is in a shortage occupation.
We control for a range of recruiter characteristics, such as gender, level of education, political preferences, and experience. Discrimination in hiring decisions is also analysed in relation to company characteristics, including firm size, region, multinational status, and profit orientation. Regional differences are mapped, with attention to how political preferences may shape discriminatory practices.
Our findings shed light on the mechanisms of unequal treatment in recruitment for individuals with migration backgrounds and offer policy and practical recommendations for employers and employees on mitigating biases in the labour market.
Miss Juliane Kuehn (Friedrich-Alexander Universität Erlangen-Nürnberg) - Presenting Author
Vignette experiments, alongside correspondence studies, are widely used to examine discrimination in hiring based on personal characteristics such as gender, ethnicity, or physical attractiveness. However, these methods predominantly capture discrimination in the initial recruitment phase, focusing solely on written materials. They overlook subsequent stages of the hiring process, particularly personal interviews, where factors such as voice, language, posture, or perceived likeability significantly influence hiring decisions.
This study extends existing vignette methodologies by integrating application videos alongside written documents. Application videos create a more realistic hiring context by introducing additional evaluation criteria and providing richer information about applicants. This approach can reduce stereotypical perceptions and mitigate statistical discrimination, especially for marginalized groups.
To isolate the causal effects of individual characteristics, the study employs deepfake technology to manipulate attributes systematically. For example, face-swapping technology alters an applicant’s external appearance while keeping the audio track constant, isolating the impact of appearance-based discrimination. Alternatively, the external appearance can remain fixed while varying the audio track to assess the influence of vocal characteristics.
Preliminary findings suggest that ethnic minorities, particularly applicants with African or Asian appearances, benefit from the inclusion of application videos, while applicants with German appearances experience no disadvantages. A subsequent vignette experiment will assess the generalizability of these findings, exploring how effects vary with applicants’ performance levels and identifying conditions under which application videos effectively reduce discrimination.
By bridging the gap between written evaluations and interpersonal stages of recruitment, this research advances vignette methodologies and provides actionable insights for designing fairer hiring practices. Application videos are a promising tool for mitigating biases in recruitment while enhancing the realism and robustness of experimental approaches to studying discrimination.
Miss Chia-Jung Tsai (GESIS – Leibniz Institute for the Social Sciences) - Presenting Author
Dr R. Gordon Rinderknecht (RAND Corporation)
Professor John Palmer (Pompeu Fabra University)
Professor Emilio Zagheni (Max-Planck Institute for Demographic Research)
This study investigates the extent to which the sex ratio of refugee populations can shape attitudes toward the refugee influx in Germany by using an online vignette survey experiment. We investigate whether the sex ratio of the refugee population explains differences in people's non-acceptance of refugee arrivals and different types of perceived threats: job competition, mate competition, German culture, and safety. Respondents are recruited via Meta targeted online advertisement in the German state of Mecklenburg–Western Pomerania. Our results suggest that the non-acceptance of a refugee group increases in line with that group’s ratio of men to women. Increases in the perceived threats toward German culture and safety appear to be especially pronounced as the ratio of men to women increases among non-white refugees. Female respondents tended to show more tolerant attitudes than male respondents toward refugee groups with lower male-to-female ratios. However, female respondents also perceived higher threats to culture and mate competition from non-white male refugees. Male respondents perceived higher job competition from non-white male refugees. Both female and male respondents perceived higher threats to safety from non-white male refugees. This study clarifies the underlying mechanisms driving negative attitudes toward refugee arrivals in Germany, focusing on the demographic composition of the refugee population.
Professor Giovanni Busetta (University of Messina) - Presenting Author
Professor Maria Gabriella Campolo (University of Messina)
Dr Giovanni Maria Ficarra (University of Messina)
Professor Alessandra Trimarchi (University of Messina)
Discrimination based on gender and ethnicity remains a persistent obstacle to achieving equity in labor markets. Despite significant advances in understanding discriminatory practices, much of the existing research is limited to the early stages of recruitment, often neglecting how these biases evolve throughout the hiring process and their impact on career advancement. Traditional methodologies, such as field experiments, provide valuable insights but fall short of capturing the complexities of multifaceted biases, especially when gender and ethnicity intersect.
This study addresses these gaps by employing a factorial survey experiment to investigate hiring discrimination in the Italian labor market. Factorial surveys, also known as vignette experiments, allow for a more nuanced examination of employer decision-making by simulating realistic hiring scenarios. This approach enables us to assess whether discriminatory practices arise from taste-based biases, driven by personal preferences, or from statistical discrimination, rooted in generalized stereotypes about certain groups.
The research focuses on understanding the interplay between gender and ethnicity in recruitment practices, exploring how these factors independently and jointly influence perceptions of candidate suitability. By adopting an intersectional perspective, this study contributes to a deeper understanding of the mechanisms underlying discrimination, while providing a flexible methodological framework applicable to diverse contexts.
In addition to examining hiring practices, we emphasize the importance of expanding experimental approaches to include additional stages of the employment process, such as interviews and career advancement opportunities. This broader perspective allows for a more comprehensive analysis of labor market inequalities.
By focusing on the Italian labor market, this research offers context-specific insights into employer behavior, with implications for broader discussions on diversity and inclusion in workplaces. The study aims to contribute to the development of policies that foster equity, promote inclusivity, and reduce discriminatory barriers in employment.