ESRA 2025 Preliminary Program
All time references are in CEST
Surveying Diversity: Methodological Issues and Best Practices |
| Session Organisers |
Dr Lisa de Vries (Bielefeld University)
Mrs Zaza Zindel (German Centre for Integration and Migration Research; Bielefeld University)
|
| Time | Thursday 17 July, 14:00 - 15:00 |
| Room |
Ruppert C - 0.23 |
Diversity is a critical topic across society, encompassing dimensions such as age, gender, ethnicity, and socioeconomic status. As societies develop more complex and heterogeneous, inclusive research that adequately captures this diversity is essential. The task of surveying a “super-diverse” society, in which individuals embody multiple intersecting identities and experiences, presents unique challenges for survey design, data collection, and analysis.
This complexity not only opens new directions for survey research but also introduces significant methodological challenges. Traditional survey instruments may struggle to capture the full range of social diversity, and classic sampling techniques may fail to reach specific subpopulations. Furthermore, data security and research ethics become increasingly complex in this context. Addressing these challenges requires innovative approaches to sampling, questionnaire design, and data integration. Advances in technology and new data sources offer promising opportunities to better reflect the diversity of modern societies in research.
This session invites researchers and professionals from survey institutes, census bureaus, and related fields to discuss recent developments, as well as both methodological and practical challenges of surveying increasingly diverse populations. Given the societal relevance and its relatively recent emergence in survey research, we are open to a wide range of contributions that capture diversity in survey research. We especially welcome contributions that provide insights and perspectives from different communities (e.g., LGBTQI+, ethnic or religious minorities, or people with disabilities).
Relevant topics include (but are not limited to):
• Measurement instruments that reflect the complexity of a diverse society (e.g., intersectionality, multidimensional scales, tailored question formats)
• Sampling strategies that capture the full range of societal diversity, including hard-to-reach or underrepresented groups
• Balancing the need for detailed demographic information with concerns about respondent burden, privacy, and research ethics
• Integrating diversity across all survey stages, from question development to data analysis (e.g., participatory research
Keywords: diversity, inclusion, survey research, social heterogeneity, intersectionality
Papers
Surveying Student Diversity: The Case of the Student Survey in Germany
Mrs Ulrike Schwabe (German Centre for Higher Education Research and Science Studies) - Presenting Author
Mrs Frederike Gerdes (German Centre for Higher Education Research and Science Studies)
Dr Julia Steinkühler (German Centre for Higher Education Research and Science Studies)
Mrs Theresa Schommer (German Centre for Higher Education Research and Science Studies)
Dr Nancy Kracke (German Centre for Higher Education Research and Science Studies)
In recent decades, the student population in many countries has become more diverse (Kerst et al., 2024; Middendorff 2015; Middendorff & Wolter, 2021). In order to capture this diversity in all its dimensions, survey-designers have to develop an awareness for these complexities (Gottburgsen & Jungbauer-Gans, 2024). This contribution uses the Student Survey in Germany as a case study to demonstrate how student diversity is assessed in a nationwide, representative self-administered web-survey (Beuße et al., 2022).
Based on a general multi-layered concept of diversity in higher education, we first outline the measurement instruments employed, such as methods capturing gender, sexual orientation or prior educational pathways. Second, we describe our sampling strategy and weighting procedures. Particular attention is given to the challenges associated with (a) the role of higher education institutions as key gatekeepers for field access, (ii) the inclusion of students with health impairments, and (iii) the use of official statistics for post-stratification. We close our contribution with current empirical findings for Germany.
Can Filter Questions Filter Too Much?: Improving Measurement of Disability and Impairment
Professor Randall K. Thomas (Ipsos Public Affairs) - Presenting Author
Ms Megan A. Hendrich (Ipsos Public Affairs)
Survey researchers commonly use short filter questions to screen for individuals so they can be asked more in-depth questions. For instance, participants may be asked a general question like ‘Do you have any disability or impairment?’ If yes, the researcher can drill down to the type and extent of disability or impairment. However, this simple question may not sufficiently identify all people who might be of interest, including people who may have corrections for their impairment (e.g., hearing aids, crutches). We report on two studies (n=5,923; n=4,592) where we randomly assigned participants to no filter or a filter question (randomly using either a Dichotomous Response Format (DRF – ‘Yes-No’) filter or a Multiple Response Format (MRF – ‘Select all’) filter). All were then asked for their degree of impairment (ranging from ‘No impairment’ to ‘Complete impairment’), regardless of their filter response. The second study repeated the same manipulations as in the first study, but we followed up with items to determine the severity of the impairment more precisely. Both studies found a much higher impairment rate (hearing, seeing, movement, cognitive abilities, and mental health) for the ‘degree’ of impairment question than when using only a filter question. Though we identified more people with impairment, we found that the average rating of severity of impairment was less for those who did not indicate impairment in the filter question. In addition, the second study found that many who had lower levels of severity used aids for their impairment that were associated with a lack of identification of impairment in the filter questions. We conclude that simpler filter questions can significantly affect both the prevalence and severity of impairment in a population of interest. For more accurate impairment assessment, we recommend using graded impairment measures without filter questions.
