ESRA 2025 Preliminary Program
All time references are in CEST
Making surveys more inclusive and equitable: strategies, actions and challenges 3 |
| Session Organisers |
Dr Véronique SIEGLER (Office for National Statistics (ONS))
Line Knudsen (National Centre for Social Research (Natcen))
|
| Time | Wednesday 16 July, 11:00 - 12:15 |
| Room |
Ruppert D - 0.22 |
In today's diverse society, ensuring inclusivity in social surveys is essential for generating accurate, representative, and meaningful insights that can shape policies. Without a comprehensive approach to inclusivity, there is a risk of omitting important voices and producing skewed findings. Additionally, inclusivity carries an ethical responsibility to ensure that all groups, particularly marginalised ones, are heard. This session will explore how researchers can adopt a holistic approach to inclusivity across all stages of the survey process, from respondent engagement , sampling and questionnaire design to data analysis requirements. Key to this discussion is the concept of intersectionality—people’s experiences are shaped by multiple, interconnected factors, and ignoring this complexity can lead to oversimplified approaches to inclusivity.
Barriers to inclusivity in social surveys are wide-ranging, including accessibility issues, cultural insensitivity, technological challenges, and lack of trust amongst certain communities or disenfranchised individuals. The aim of this session is to reflect on how to overcome these barriers, to share successes, challenges and lessons learned. The session will provide a platform for researchers to present innovative strategies and approaches that ensure inclusivity, contributing to a more equitable future for social surveys.
We invite papers that address a variety of topics, including but not limited to:
• Inclusivity for individuals experiencing physical or mental health issues, or neurodiversity.
• Overcoming demographic or identity-based barriers.
• Engaging hard-to-reach groups.
• Addressing the complexities of intersectionality in social surveys.
• Building trust in social surveys.
• Leveraging alternative data to address inclusivity challenges.
• Inclusivity in the context of non-traditional data sources.
We particularly welcome submissions from researchers across different sectors—academia, national statistics, and private research organisations. This session aims to inform future strategies and best practices for enhancing inclusivity in social surveys.
Keywords: inclusivity, equity, accessibility, intersectionality, hard-to-reach, trust, barriers, alternative data
Papers
Designing an inclusive approach to an ethnic minority boost in a large face-to-face survey
Mrs Beverley Bates (National Centre for Social Research)
Mrs Sarah Morris (National Centre for Social Research)
Mrs Mari Toomse-Smith (National Centre for Social Research) - Presenting Author
Adult Psychiatric Morbidity Survey (APMS) is a flagship mental health survey in England. Our client on the survey, NHS England, wished to add an ethnic minority boost to the survey to improve the information that is available on the mental health of specific ethnic groups. We worked with a communications agency focused on minority communities to create an inclusive and culturally sensitive communications and fieldwork strategy. In the presentation, we will give an overview of what worked about this approach in particular and what were wider issues that eventually resulted in the ethnic minority boost not meeting its aims. We will also explore whether an even more inclusive approach would be feasible in the context of a large survey and ask the wider question of whether boosts of any small group are still viable in face-to-face surveys.
Designed for inclusivity: Findings from a feasibility study for a new UK birth cohort
Dr Alyce Raybould (University College London) - Presenting Author
Professor Lisa Calderwood (University College London)
Professor Alissa Goodman (University College London)
Professor Pasco Fearon (University College London and University of Cambridge)
Dr Erica Wong (University College London)
Ms Karen Dennison (University College London)
Birth cohort studies are vital for understanding the development of successive generations of children, though there is increasing recognition that often those families who are of most interest from a research and policy perspective are less likely to be recruited and retained in these studies. Therefore, designing an inclusive survey that can effectively recruit and retain hard-to-reach populations is a key challenge.
The Early Life Cohort Feasibility Study (ELC-FS) aimed to test the feasibility of conducting a UK-birth cohort study, collecting information about several thousand babies and their families between 8-12 months old in 2023-24. The ELC-FS has a very strong focus on inclusivity and was designed to maximise representation of ‘less often heard’ groups. It includes sample boosts of babies from disadvantaged and ethnic minority families, as well as Scotland, Wales and Northern Ireland, and direct recruitment of fathers including those living in their own households. The study also made concerted efforts to build accessibility into the study’s design.
The study involved 30-60 minute interviews with mothers and fathers. We also collected record linkage consents and saliva samples for DNA extraction. Interviews were carried out primarily face-to-face, with web, phone and video interviewing also used. We also tested the effectiveness of targeted differential incentives and different conditional incentives.
This paper will give an overview of the inclusive and accessible design of ELC-FS. We will present some early findings relating to representativity, response rates, and the experimental components of the study: we achieved a high proportion of lower-income families in the sample, response rates among ethnic minority groups tended to be higher than others, and we found differences in consent rates to data linkages and saliva provision among lower income and ethnic minority families.
How do you feel about surveys? A qualitative perspective on survey participation
Dr Marieke Haan (University of Groningen) - Presenting Author
Dr Yfke Ongena (University of Groningen)
Dr Rita Smaniotto (University of Groningen)
Ms Demi Stadens (Municipality of Ermelo)
The decline in survey response rates across all survey modes poses critical challenges for survey research. Traditionally, insights into non-participation have been drawn from paradata and brief non-response surveys administered post-refusal.
To deepen our understanding of survey (non)-participation, we conducted qualitative semi-structured interviews with both respondents and non-respondents. Guided by established survey theories and practical insights, we designed an interview guide to explore experiences with and reasons for (non)-participation. The guide included questions on motivation, survey design, survey topics, communication, and privacy (Dillman, 2020), as well as vignettes and photos to encourage conversation. Recruitment proved challenging (see Haan et al. 2024), but through personal professional networks and snowball sampling, we recruited 24 participants between September 2023 and July 2024 — 12 respondents and 12 non-respondents. Interviews were followed by an assessment of non-respondent types: occasional non-respondents and "hard" non-respondents being very resistant to participate.
Thematic analysis of the transcribed data revealed that negative past survey experiences have a lasting impact, reducing willingness to participate. While positive experiences, such as gaining insights or reflecting on personal views, increases engagement. We will present these findings, discuss improvements for the survey process and discuss, with an eye on the results, how surveys can potentially be made more inclusive.
References:
Dillman, D. (2020). Towards survey response rate theories that no longer pass each other like strangers in the night. In P.S. Brenner, Understanding Survey Methodology. Sociological Theory and Applications (pp. 15-44). Springer.
Haan, M., Toepoel, V., Ongena, Y.P., & Janssen, B. (2024). Recruiting non-respondents for a conversation about reasons for non-response: A description and evaluation. Survey Practice 17 (March).
A Multifaceted Approach for Inclusivity: Representing a Diverse Population through the California Health Interview Survey
Mr Todd Hughes (University of California Los Angeles) - Presenting Author
Dr Ninez Ponce (University of California Los Angeles)
Since its inception in 2001, a central objective of the California Health Interview Survey (CHIS) has been to reflect the very diverse population of the state of California, including sufficient representation of race and ethnic minorities, those with language needs other than English, persons with different levels of education and literacy, sexual and gender minorities, and other under-represented groups. The CHIS has employed a multifaceted set of strategies to address these challenges, including predictive modeling during address-based sampling, multi-mode data collection, supplemental sampling frames including prepaid cell phones, multiple strategies for recruitment and data collection in languages other than English, linguistic simplification review of survey questions and materials, oversampling of specific race and ethnic groups, and follow-on studies of understudied groups. In recognition of these efforts, the CHIS was chosen as the winner of the 2024 American Association of Public Opinion Research (AAPOR) Inclusive Voices Award, which recognizes important data sets, research, and survey methods that have improved the ability to study complex social phenomena related to understudied populations. This presentation will describe the various methods used in the CHIS to address inclusivity for the diverse population it studies, as well as some ideas to consider for future strategies.
