All time references are in CEST
Issues in Survey Methods and Analysis about LGBTI+ Communities |
|
| Session Organisers | Dr Angelo Moretti (Utrecht University) Dr Katharina Meitinger (Utrecht University) Mr Tom Dörr (Independent Equality, Diversity and Inclusion management ) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
Data collection and analyses focusing on LGBTI+ communities, which are vital for providing reliable information to policymakers, suffer from important issues. In particular, data collection efforts often fall short in addressing the wide range of issues affecting LGBTI+ populations. This data lack impacts policy effectiveness, significantly affecting the lives and well-being of LGBTI+ people.
These populations are ‘hard-to-reach’ populations, making them difficult for survey researchers to access; thus, carefully designed sampling methods are necessary. Additionally, significant methodological challenges arise when analysing collected data, particularly concerning confidentiality and privacy, as well as representativeness issues. These concerns can lead to measurement errors, such as underreporting of discrimination and victimisation. Therefore, these issues must be considered when developing and applying statistical methods. Moreover, due to the unique nature of the subject, variable distributions may feature a large number of zeros, making standard statistical approaches, especially based on models, inadequate.
We invite substantive and methodological papers addressing survey methods and analysis issues related to LGBTI+ populations. Methodological papers may cover sampling designs, data integration involving new data forms, weighting approaches, and statistical modeling approaches for robust analysis.
Keywords: LGBTI+, inclusion, diversity, sampling, total survey error
Dr Tamara Bosshardt (University of Lucerne) - Presenting Author
Mr Leo Theissing (University of Lucerne)
During the last decade, Switzerland has made significant progress in LGBTQIA+ rights. The revision of adoption law, the legalization of gay marriage, and the extension of anti-discrimination laws have led to improvements for rainbow families - at least at a legal level. However, the only survey focusing on rainbow families in Switzerland was conducted in 2017 before these major changes occurred. Unfortunately, large-scale family surveys in Switzerland too often only address heteronormative (nuclear) family constellations. They exclude non-biological parents and systematically overlook the realities of queer family constellations.
As part of the mixed-method project “Rainbow Families in Switzerland”, we conduct a larger online survey to investigate the “hard-to-reach” population of LGBTQIA+ families. The survey aims to explore (1) modes of family formation, (2) past and current family constellations as well as parenting and caring arrangements, and (3) specific challenges and needs of rainbow families who are confronted with institutions, that are designed to fit the needs of heteronormative families.
The survey is designed to build a bridge between LGBTQIA+ research in Switzerland, rarely focusing on family practices and previous family surveys that overlook LGBTQIA+ realities. Thinking intersectional, we investigate how gender, sexuality, race, class, and disability influence modes of family formation or interactions with institutions. We want to highlight forms of everyday resistance and queer joys beyond a focus on discrimination and minority stress. In this contribution, we discuss how to balance operationalizing complex queer modes of family formation and practices conceptually clearly distinct while keeping queer possibilities for contingency, fluidity, and contradictions.
Professor Pier Francesco Perri (Department of Economics, Statistics and Finance “Giovanni Anania”, University of Calabria ) - Presenting Author
Dr Shu-Hui Hsieh (Center for Survey Research, Research Center for Humanities and Social Sciences, Academia Sinica)
Dr Adrian Hoffmann (Department of Experimental Psychology, University of Duesseldorf )
When do individuals feel comfortable enough to provide honest responses to sensitive questions, particularly if they refer to highly personal issues such as sexual identity? To address this question, the current study uses a probabilistic sample of the Taiwanese population to investigate how different questioning techniques influence respondents’ willingness to disclose their sexual orientation in a self-report survey. Given the social pressure towards heteronormativity in Taiwan, we expected self-reports especially by non-heterosexual individuals to be influenced by social desirability bias (SDB). This bias can lead to underreporting of non-heterosexual orientations due to the social stigma associated with them, and thus to insufficient validity of prevalence estimates for the proportion of non-heterosexual individuals in the population. To face this issue, we conducted a survey about sexual identity in Taiwan and applied two innovative indirect questioning techniques, the Cheating Detection Triangular Model (CDTRM) and the Two-Stage Multilevel Randomized Response Technique (TMRRT), which have both proven effective in reducing the influence of self-protective response behavior attributable to SDB compared to conventional direct questioning approaches.
Our findings indicate that both the CDTRM and the TMRRT significantly mitigate the validity-threatening influence of SDB on the results of self-report surveys on sexual identity. Compared to conventional direct questioning, these techniques provide higher and thus potentially more valid prevalence estimates for non-heterosexual orientation. This higher validity is presumably achieved by granting full confidentiality of individual answers, thereby enhancing respondents’ willingness to provide truthful responses, and additionally adjusting for cheaters, i.e. survey participants not adhering to the instructions prescribed by the techniques.
In the talk we discuss the survey design, and show prevalence estimates for the proportion of non-heterosexual individuals in Taiwan obtained with the CDTRM, the TMRRT, and the traditional direct questioning approach.
Dr Ceylan Engin (Bogazici University) - Presenting Author
In Turkey, nationally representative surveys that capture the LGBTI+ population are non-existent, and even research on societal attitudes toward sexual minorities remains extremely limited. In countries where nationally representative data on gender and sexual minorities is unavailable, examining public attitudes toward sexual minorities becomes essential. To address this gap, I investigate both the prevalence and the determinants of intolerant attitudes toward gays and lesbians in Turkish society. The analysis draws on novel data from the 2022 Turkish COVID-19 Values Study (TCVS), conducted with 1500 adults across 12 NUTS-1 regions.
The findings confirm the widespread intolerance toward gays and lesbians in Turkish society. 40% of the population believe that homosexuality is a threat to social values while an equal proportion say they would feel embarrassed if they had a gay or lesbian family member. Another 40% disagree that gays and lesbians should be able to live their lives freely as they wish, and an additional 20% remain neutral on this issue. Higher levels of religiosity, political conservatism, and trust in government are significantly associated with greater intolerance toward sexual minorities. Intolerance toward gays and lesbians is also more prevalent among men, less educated individuals, and married people compared to women, those who are higher educated, and single individuals.
These results have several important implications. Intolerant attitudes often mirror the violence, discrimination, and prejudice gender and sexual minorities encounter in various social, economic, political, and health-related settings. Moreover, high intolerance can negatively affect LGBTI+ population rates of disclosure in surveys, thereby hindering our ability to accurately estimate the size, characteristics, and experiences of the LGBTI+ population in future surveys. These findings highlight the urgent need for laws and policies that promote acceptance and safeguard the rights of the LGBTI+ community in Turkey.
Dr Angelo Moretti (Utrecht University) - Presenting Author
Dr Katharina Meitinger (Utrecht University)
Dr Camilla Salvatore (Utrecht University)
Previous research revealed a puzzling finding in the measurement of attitudes towards homosexuality: In countries with extensive discrimination towards homosexuals, respondents were not aware of discrimination. In contrast, respondents in more tolerant countries report more frequently discrimination. At the same time, the countries also showed regional variability in the perceived discrimination of homosexuals. Respondents across and within countries might interpret the questions differently and survey measures are potentially misleading. Web probing is a crucial tool to reveal variations in respondents’ associations and/or silent misinterpretations when answering survey items. So far, web probing studies compared respondents’ associations across countries but disregarded potential regional variabilities of results. The combination of web probing with small area estimation provides a novel approach to study within and across country variability of respondents’ associations. Small area estimation approaches are able to deal with producing reliable analyses of relevant indicators when the survey is not representative at a subnational level.
However, our study is based on a quota (non-probabilistic) sample; therefore, important points need to be addressed here. This presentation will present the study from a total survey error framework perspective. In particular, we will pay a particular attention to the data editing and weighting steps. We will compare different weighting approaches we adopted in a cross-country context. These are based on European probabilistic sample surveys and/or known population benchmarks available via Official Statistics archive.
This project received funding from the Dutch Research Council (NWO).
Ms Christina Pao (Princeton University) - Presenting Author
A longstanding challenge in the social measurement of identities is deciding “how many” and “which” categories should be included for study; this decision has downstream consequences for monitoring and intervening on inequality. Though these categories have historically been imposed and coded by survey administrators (e.g., interviewer-coded sex and race), many surveys have since turned to principles of self-identification. Nonetheless, due to prior methodological challenges, there have been few social identities that utilize the most flexible form of self-identification on a survey: open-ended text boxes. Open-ended measures provide flexibility for the respondents, which is particularly useful for understanding marginalized populations or populations currently in flux; however, open-ended measures are challenging to code and difficult to weigh for costs and benefits. In this study, I compare traditional closed-ended measures and fully open-ended measures for gender identity and sexual orientation, two identities currently undergoing large demographic change, with an original survey fielded in the U.S. and U.K. in 2022 (n = 2,623). I propose a novel tool, the Open-ended Text Utility Index (OTUI), that summarizes the tradeoffs of using an open-ended versus a closed-ended measure for a sociodemographic identity into a single numeric value. My OTUI calculations for my case studies indicate that gender identity measures might be easily translated into a fully open-ended format at a larger scale, but sexuality measures may not. This paper encourages researchers to challenge the categories taken for granted by prior social measurement and provides the OTUI as a tool to gauge the feasibility.
Mr Tom Dörr (Freelance Equality, Diversity and Inclusion management )
Dr Angelo Moretti (Utrecht University) - Presenting Author
Lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ+) nurses represent one of the largest subgroups within the nursing profession in Germany and many other countries. Despite this, there is a notable lack of empirical survey research in nursing literature, and issues such as workplace discrimination and exclusion are largely overlooked by nursing education and professional organizations. Preliminary findings indicate that LGBTQ+ nurses see an urgent need for a professional organisation to educate both the nursing profession and the general public on LGBTQ+ issues while addressing their advocacy and healthcare policy needs.
Improving the workplace climate for LGBTQ+ employees requires:
1. Revisions to workplace policies,
2. Comprehensive education for the healthcare workforce, and
3. Active advocacy from nursing professional organisations.
In this presentation, we share preliminary results—both methodological and substantive—of an online survey conducted in Germany. This project aimed to design and implement a survey to explore and address the issues outlined above, focusing on nurses and certain other healthcare professionals in Germany. Given the lack of a sampling frame, we employed a non probabilistic sampling approach, combining convenience and snowball sampling techniques. Our findings have the potential to create a positive impact not only in Germany but also globally, as there is currently a significant gap in survey-based research on these critical topics.