All time references are in CEST
Measuring sex and gender identity in social surveys: the next phase 1 |
|
| Session Organisers |
Dr Soazig Clifton (National Centre for Social Research) Mrs Lisa Rutherford (National Centre for Social Research and University College London) |
| Time | Wednesday 19 July, 11:00 - 12:30 |
| Room | U6-07 |
The way in which data on sex and gender identity are collected in official statistics and social surveys has been in the spotlight in recent years. This attention has resulted in many social surveys, either formally or informally, reviewing and what data they collected on the topic, and how. Outputs from these reviews are beginning to emerge. For official statistics this has sometimes been in the form of guidance on how official data on sex and gender identity should be collected.
This period of review has taken place against the backdrop of the COVID-19 global pandemic. The requirement for social surveys to quickly deviate from standard data collection methods has been well documented and there is a broad consensus that the pandemic will have long-term implications for how social surveys are conducted. The shift away from traditional face to face interviewing also has implications for what sex and gender identity data can be collected.
The topic reviews and pandemic followed broadly similar, independent, timelines. ESRA 2023 represents an ideal opportunity, and forum, for social survey researchers to collectively reflect upon how events of the last few years have shaped the future direction of sex and gender identity data collection.
Keywords: measurement, questionnaire, sex, gender identity, LGBT populations, self-categorizing, gender scales
Ms Christina Pao (Princeton University) - Presenting Author
Presently, many large-scale surveys reuse longstanding sociodemographic questions for reasons of data concordance across time (e.g., a binary sex question on the census), but several of these measures may be too coarse to capture current social realities of gender, and particularly gender expression. I argue that gradational gender questions—a set of two unipolar scales (0 to 6) that ask for self-rated masculinity and femininity are a parsimonious and easily implementable solution.
Using an original survey (N=2477) fielded in the UK and US, I demonstrate that gradational gender questions not only maintain prior data quality on gender, but also disentangle previously underexplored heterogeneity in gender expression—not only among trans and nonbinary respondents, but also cisgender respondents. For my analysis, I provide three case studies of topics of sociological interest (gender inequalities in income, depressive status, and social connectivity) and show how gradational gender questions can: i) parsimoniously replicate prior findings that used longer indices, ii) reveal within-gender identity heterogeneity when combined with two-step gender measures, and iii) be manipulated in conjunction with two-step gender measures to reveal penalties of gender nonconformity, even for cisgender people, and iv) potentially functionally replace two-step gender measures for surveys in the future.
Mrs Soazig Clifton (University College London / NatCen Social Research) - Presenting Author
Professor Catherine Mercer (University College London)
Dr Kate Nambiar (Welsh Gender Service, Cardiff & Vale University Health Board)
Dr Lorraine McDonagh (University College London)
Dr Jo Gibbs (University College London)
Professor Pam Sonnenberg (University College London)
Professor Nigel Field (University College London)
Introduction
Many surveys are reviewing how gender and sex are measured. Here, we describe development of measures of gender and sex for Britain’s fourth National Survey of Sexual Attitudes and Lifestyles (‘Natsal-4’): a probability-sample interviewer-administered survey requiring robust measures for routing participants to appropriate questions/biosamples, and understanding risk behaviours.
Methods
We reviewed existing measures, and engaged with subject and questionnaire design experts to create a draft questions. We then undertook: 1) cognitive testing (n=30, including 3 people who identified as transgender or non-binary (‘trans’), 2) community/lay review of the full questionnaire including from a trans perspective, 3) two pilot studies (age range 16-59y; total n=261), using a combination of interviewer-administered and self-completion questions. Questionnaire development was undertaken June-November2019, piloting in June-Aug 2021 and Feb-March2022.
Results
Several existing questions had previously undergone testing, however, no existing question set met the requirements for a detailed bio-social sexual health survey. Stakeholder engagement provided essential input into measurement concepts, question wording, and how best to administer the questions. The Natsal draft questions captured gender identity, sex described at birth, and trans identity/history. Cognitive testing found the questions to be well understood and acceptable, and identified minor changes to wording to improve clarity. Pilot findings indicated the questions were acceptable (no item non-response, no negative feedback). However, pilot data suggested likely mis-coding of some participants as trans, resulting in changes to response options and additional checks in the final questionnaire.
Discussion
We used a multi-stage process to identify questions that balance the needs of a diverse group of participants and end users. Testing found our questions were acceptable and comprehensible. This adds to the literature on measurement at a critical time for this topic.
Dr Celine Wuyts (KU Leuven) - Presenting Author
Survey researchers have rarely failed to collect data about respondents’ gender/sex, and have historically done so, without second thought or debate, by a binary measure that implies gender is understood as fixed, perfectly aligned with biological sex, and well observable by an interviewer either by sight or by ear. Over the past 10 years, more inclusive gender measurements have been developed and are increasingly more widely advocated. Some research institutions and agencies are in the process of reviewing their approaches accordingly. At the same time, the move towards self-completion survey modes, long underway and recently intensifying by the COVID-19 pandemic and the restrictions on in-person contacts, has made survey conditions more amenable to the proposed inclusive gender measurements, which depend critically on self-report.
However, the move towards inclusive gender measurement is slow in practice, and many survey practitioners remain reluctant to make the switch, arguing that inclusive gender measurements would confuse or offend many respondents. The Belgian population is one of the most uncertainty avoiding in the world. People are generally uncomfortable with changing the way they think and practice, and intolerant of deviant persons and ideas. How uncomfortable are people really with changing the way they are presented one short question in forms and questionnaires?
The aim of this study is twofold. First, to describe the current use of gender measurements in Belgium relative to international best practices and recommendations, and survey practitioners’ reluctance. Second, to analyze attitudes of the general population as a potential impediment, and to identify its antecedents. We make use of a direct measure of respondents’ preference regarding gender measurement in forms and questionnaires, and beliefs about gender measures, in a probability-based general-population web panel in Belgium.
Ms Roxane Mordasini (FORS) - Presenting Author
Mr Max Felder (FORS)
Mr Nicolas Pekari (FORS)
Non-binary gender identities and their inclusion in traditional gender measures are slowly gaining recognition and acceptance in survey methodology. However, most researchers continue to rely on sex assigned at birth as a proxy for measuring gender. Even when gender identity is measured, participants are often still presented with a dichotomous scale, i.e. a non-exhaustive response scale. Several studies have shown that, dichotomous measures can be poor proxies for measuring gender identity, while using finer-grained gender measures allow for a more nuanced and accurate analysis of gender gaps in social and behavioural predictors.
The main aim of this study is to explore better suited gender measurements for a more comprehensive understanding of gender differences in attitudes and behaviours. Participants from an online Swiss panel survey are asked for their sex assigned at birth, their gender identity (through a non-dichotomous categorical variable) and their degree of gender expression (through a bi-dimensional scale). Additionally, attitudes towards politics, health, and technology are measured and the results are compared using the three types of sex and gender measures. The potential framing effect of gender measures is tested through randomised presentation order of the sex and gender measurements. Building on previous work, we expect to expand on the added value of finer-grained gender measures in online population surveys.
With this study, we wish to present an overview of the current state of research on the measurement of gender identities as well as validate recent fuzzy gender measures. We hope to offer a guideline for the use of broader and more precise gender measures, allowing for the inclusion of non-binary and non-conforming gender identities in surveys.