Engendering Survey Research |
|
| Session Organisers |
Dr Mónica Méndez Lago (Centro de Investigaciones Sociológicas (Spain)) Dr Sara Pasadas del Amo (Instituto de Estudios Sociales Avanzados-CSIC (Spain)) |
| Time | Friday 19th July, 11:00 - 12:30 |
| Room | D16 |
There are different aspects in which designing surveys is related to gender. It has become an issue of concern in questionnaire design when drafting questions about gender or gender identity. Whereas gender used to be a straightforward variable, that was not even asked but directly observed by face to face interviewers, now it is beginning to be explicitly asked and increasingly using formats that are different from the traditional binary man/woman responses. There is also an increasing interest in different ways to ask about gender identity.
Also related to questionnaire design, there is the issue of grammatical gender and how this is being dealt with in a social context demanding a more inclusive language. In many of the “grammatical gendered” languages, those which assign a gender to nouns and their modifiers, the masculine form is used to refer both to masculine and feminine objects/people. Increasing social contestation of the generic masculine is pushing towards asking questions in a different way, i.e. explicitly differentiating masculine and feminine grammatical forms. For instance, asking someone in Spanish about their evaluation of doctors as professionals is supposed to include both male and female doctors (at least from a linguistic standpoint). However, from a cognitive perspective, it is unclear whether respondents interpret the question having in mind male and female doctors, or just male doctors.
This leads to the third aspect addressed in this session: gender differences in answering survey questions. Some evidence suggests that men and women carry out the survey response process differently in questions dealing with diverse topics, from political knowledge (Rapoport, 1981; Ferrin et al, 2017) to the number of past sexual partners (Brown & Sinclair, 1999; Brown et al. 2017), and that the question design and survey features might explain some of those differences.
To sum up, this session invites papers that address the different ways in which gender may be affecting survey research design and results. Examples include:
1) Experiences from national or cross national surveys in the way to formulate questions about respondents´ gender/gender identity.
2) Experiences on how to deal with the design of questions in which grammatical gender might make a difference, in national and cross-national surveys, where the need of translation/combining different languages complicates the picture.
3) Experiments/analysis of survey results about gender differences in responding to survey questions.
Keywords: Gender, Questionnaire design, Response patterns, Measurement error
Mrs Kyley McGeeney (PSB) - Presenting Author
Mr Brian Kriz (PSB)
Clients are asking how to measure non-binary gender identity on surveys. Currently there’s no standard question wording approved by the government’s Office of Management and Budget (OMB) and it’s not asked on federal surveys typically used for weighting (e.g. CPS, ACS). Many researchers use The Williams Institute’s “Two-Step” approach from their publication, “Best Practices for Asking Questions to Identify Transgender and Other Gender Minority Respondents on Population-Based Surveys.” This involves asking respondents their sex assigned at birth followed by their current gender identity. There are three potential issues with this approach: it adds an additional survey item, the majority of people give the same answer to both questions, and some clients fear the “sex at birth” question is off-putting. We therefore endeavored to find alternative ways to ask non-binary gender as a single, more neutral survey question.
Using GfK’s KnowledgePanel, we tested three approaches for asking gender. All panelists are asked binary gender (male/female) annually. On a separate annual survey, all panelists are asked a modified two-step approach. This includes the sex at birth question recommended by the Williams Institute and a follow up gender identity question with slightly different response options (“something else, please specify” rather than “do not identify as female, male, or transgender). Using GfK’s omnibus survey we then asked respondents a single gender identity question using the exact wording recommended by the Williams Institute.
Our research questions related to measurement were whether disclosure of gender minority status increased with the two-step approach over a single non-binary item and whether the response option “something else, please specify” solicited higher endorsement than, “do not identify as female, male, or transgender.” We also wanted to know how to weight the non-binary responses.
Dr Clelia Cascella (University of Manchester) - Presenting Author
Dr Maria Pampaka (University of Manchester)
Professor Julian Williams (University of Manchester)
Mathematics education has substantially addressed equity issues related to gender differences internationally. But in fact, although research on gender differences in Mathematics has received increasing attention in the last fifty years, it has as yet failed to explain important patterns across cultures and geographies. In order to understand if, how, and how much social and cultural characteristics of the environment can explain differences in mathematics achievement between boys and girls, we used secondary Italian data (N=38120) from students aged 16 years old, on average.
In contrast with previous similar studies, we performed our analysis at intra-national levels in order to understand if the (statistically significant but small) gender differences typically shown to exist at national level exists also (and if so to what extent) at provincial, regional, and macro-area levels. This choice is based on the hypothesis that social conditioning and gender-biased environments can have effects on education (and Mathematics test performance in particular) with the hypothesis that gender differences can vary also within the same country, at regional and/or provincial levels.
A multilevel regression analysis explored the differential attainment of boys and girls in mathematics test scores considering locality/regional as well as classroom and school levels.
Results showed that, in Italy, the significant intra-national gender differences are between macro areas widely understood to be culturally and socio-economically ‘different’ areas of Italy. This confirms our hypothesis and highlights the importance of carrying out comparative analyses also at intra-national level rather that only at national or inter-national level.
Dr Marta Fraile Maldonado (Institute of Public Goods and Policies (IPP/CSIC))
Dr Mónica Méndez Lago (Centro de Investigaciones Sociológicas (CIS)) - Presenting Author
Dr Sara Pasadas del Amo (Institute for Advanced Social Studies (IESA/CSIC))
We will present the main results of a survey experiment designed to measure the impact of grammatical gender and gender related attitudes in relation to the evaluation of occupational prestige. The main aim of the experiment is to disentangle the effects of grammatical gender and those related to gender bias, which are usually intertwined when using grammatical gendered languages, such as Spanish. The data for the experiment stems from a cross-sectional online survey addressed to a full probability sample of the resident population in Spain, achieved through an offline push2web procedure.
Three random sub-groups were drawn. Respondents in one group were asked to rate the prestige of different occupations worded in the neutral or in the (generic) masculine form, whereas in the two other groups respondents were given a list of neutral and explicitly feminine/masculine forms of the same occupations (using both terminations “o/a” in one these groups and “a/o” in the other).
The occupations were selected using three main criteria:
1) Choosing occupations that are grammatically gender neutral and others that are masculine or feminine regarding grammatical gender;
2) Mixing occupations that are traditionally male dominated with others that are traditionally female dominated, and some mixed cases;
3) Including a combination of low, medium and high prestige occupations according to previous survey results.
The experiment included a second question asking respondents to say whether they thought each of the occupation was mostly carried out by men, mostly by women or approximately equally by both.
This experiment will shed light on important methodological questions related to grammatical gender, which is relevant to the field of questionnaire design. For cross-national survey design it is also relevant because of the different implications of comparing (grammatically) gendered versions with the original English version.