Moving beyond the binary - Exploring the potentials, challenges and consequences of measuring sex and gender alternatively in surveys - I |
|
| Session Organisers | Professor Stephanie Steinmetz (University of Lausanne) Dr Mirjam Fischer (University of Cologne) Dr Léïla Eisner (University of Zurich) Dr Verena Ortmanns (GESIS) |
| Time | Friday 9 July, 15:00 - 16:30 |
There is an increasing interest among survey providers and population registers/census bureaus to better accommodate the reality of gender diversity in their data collections. Considering that 39 of 47 Council of Europe member countries currently have a legal gender recognition procedure in place that provides trans and gender-nonconforming people (TGNC) with some degree of legal gender recognition, this is a necessary conversation to have. Tackling this in a survey comes along with challenges for the survey design and instruments on respondents’ sex and gender identity, and it might also affect respondents’ answer behavior on other questions. The four contributions in this session discuss strategies, benefits and risks of accommodating gender diversity in surveys and official register/census data. They provide new insights from current research and advice to survey providers and researchers interested in ‘moving beyond the binary’ to maintain the validity of their gender measures.
Keywords: sex and gender identity, LGBT populations, self-categorizing; gender scales
Dr Verena Ortmanns (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Dr Ranjit Singh (GESIS - Leibniz Institute for the Social Sciences)
Dr Patricia Hadler (GESIS - Leibniz Institute for the Social Sciences)
Dr Cornelia Neuert (GESIS - Leibniz Institute for the Social Sciences)
Recent legal and administrative changes in Germany have made rethinking the survey question “What is your sex?” a pressing issue. More and more surveys in Germany start to include a third, non-binary answer category. However, there are uncertainties about whether including a non-binary sex category might affect the data quality of surveys. Adding this category might, for instance, increase item and unit nonresponse or affect responses to subsequent attitudinal questions because the confrontation with a non-binary sex category makes gender-related attitudes more salient.
To test this, we conducted a split ballot experiment using an online survey. In three conditions, we randomly varied which sex (and gender) questions respondents saw. The first condition was modeled after conventional sex questions: Respondents were only asked if they were “male” or “female.” In the second condition, respondents could also choose a third sex category: The new administrative term capturing non-binary sex expressions (“divers”). In the third condition, respondents could also select the third sex category “divers” but were additionally asked about their gender via two questions capturing their self-ascribed masculinity and femininity. After the sex (and gender) questions, respondents were shown constructs correlated with gender-related attitudes and roles: gender roles, social dominance orientation and conservatism. The order of these constructs was randomized.
To determine whether the different ways of capturing sex (and gender) affect data quality, we examine item and unit nonresponse across the three conditions. Furthermore, we want to test if adding a non-binary sex category biases response to attitudinal measurement instruments. To this end, we will compare response distributions for the concepts across conditions. Since the concepts were measured with multi-item instruments, we will also test for measurement invariance with a multi-group confirmatory factor analysis across conditions.
Our presentation is of interest to researchers focusing on gender and sex issues, survey design researchers, and data producers. While the research focuses on the German context, it may be a first step in confirming or allying fears of data producers regarding the inclusion of non-binary sex and gender questions and might inspire methodologists to conduct similar experiments in other countries, languages and cultural contexts.
Mr Elias Markstedt (University of Gothenburg )
Dr Maria Solevid (University of Gothenburg )
Professor Lena Wängnerud (University of Gothenburg ) - Presenting Author
Professor Monika Djerf-Pierre (University of Gothenburg )
The rationale for this study is that self-categorizing rating scales are becoming increasingly popular in large-scale survey research moving beyond binary ways of measuring gender. We are referring here to the use of rating scales that are similar to graded scales capturing left–right or liberal–conservative political ideology, that is, scales that do not include pre-definitions of the core concepts (femininity/masculinity as compared to left/right or liberal/conservative). Yet, previous studies including such non-binary gender measures have paid little attention to potential effects of survey designs. In the empirical part of the study, we use an experimental set-up with two randomized treatment groups where the sequencing of the questions is varied. Respondents are assigned to receive (1) gender self-categorization rating scales first, followed by an open-ended question on the content of feminine and masculine traits (n1 = 708) or (2) vice versa, the open-ended question preceding the question on self-categorization (n2 = 781). We also investigate effect heterogeneity by comparing demographic groups such as age, education, and respondents’ sex. Our study results in two main strands of findings. First, men are especially affected by our treatments and rate themselves as significantly ‘less masculine’ when prompted to reflect on the meaning of gender before self-categorization on scales measuring degrees of femininity versus masculinity. Second, both men and women give a more biological meaning to gender when first asked to self-categorize rather than being asked about the meaning of gender and then asked to rate themselves. The research instrument, the sequencing of gender measurements, is affecting the results in significant ways. Scholars using self-categorizing gender rating scales should bear this in mind in the construction of surveys and the interpretation of results.
Professor Elisabeth Gidengil (McGill University) - Presenting Author
Professor Dietlind Stolle (McGill University)
This study compares two different measures of gender identity. Drawing on an online survey conducted with a representative sample of Americans that included both a single bipolar scale and separate masculinity and femininity scales, we compare how the same people respond depending on how gender identity is measured. The results of validation tests suggest that a single bipolar scale performs just as well in differentiating among those with sex typical gender identities, but separate masculinity and femininity scales are needed for studies investigating the political behavior and preferences of those with sex atypical identities.
Mr Will Beischel (University of Michigan) - Presenting Author
Dr Zach C. Schudson (California State University)
Dr Rhea Ashley Hoskin (University of Waterloo)
Professor Sari van Anders (Queen's University)
Most measurements of gender/sex ask participants
only about their sex category as female or male. Such an assessment does not include those who are neither (e.g., nonbinary people) or allow specifications of transgender and cisgender status. Combining open-ended gender/sex questions with categorical ones
can address these limitations, but coding these data can be challenging without a common analytic method. Across two studies, we tested the potential of the Gender/Sex 3x3, a novel framework for thinking about and measuring gender/sex. This framework represents
two intersecting dimensions: gender trajectory,
with categories of cisgender, transgender, and allogender (i.e., neither cisgender nor allogender), and binary
relation, with categories of binary, nonbinary, and allobinary (i.e., neither binary nor nonbinary). While some of the locations in the Gender/Sex 3x3 have been recognized in previous studies (e.g., transgender binary, transgender nonbinary, cisgender binary), others have largely been missing (e.g., cisgender nonbinary, transgender allobinary, allogender nonbinary). In Study 1, we compared how six different forms of coding open-ended and categorical gender/sex demographic data, including with the Gender/Sex 3x3, led to similar or different ways of categorizing 242 participants with diverse gender/sexes. Results demonstrated that using different analytical frameworks with the same data resulted in very different sample descriptions. For example, the number of women and men coded as transgender or cisgender varied greatly by method. The results also suggested utility for the Gender/Sex 3x3, as coders had the highest level of agreement with this method and noted its efficiency. In Study 2, we revised the gender/sex questions to address limitations of Study 1 and asked 737 gender/sex-diverse
participants to directly evaluate them. We found that all nine locations of the Gender/Sex 3x3 were populated. Quantitative feedback indicated sufficient comprehensibility and inclusivity of the questions and qualitative feedback suggested further wording changes.
We conclude by providing recommendations for questions about gender/sex that are inclusive of all nine locations in the Gender/Sex 3x3 and useful for flexible categorization of gender/sexes and the empirical reality of gender/sex diversity.