ESRA 2025 Preliminary Program
All time references are in CEST
Making surveys more inclusive and equitable: strategies, actions and challenges 2 |
| Session Organisers |
Dr Véronique SIEGLER (Office for National Statistics (ONS))
Line Knudsen (National Centre for Social Research (Natcen))
|
| Time | Wednesday 16 July, 13:30 - 15:00 |
| Room |
Ruppert Blauw - 0.53 |
In today's diverse society, ensuring inclusivity in social surveys is essential for generating accurate, representative, and meaningful insights that can shape policies. Without a comprehensive approach to inclusivity, there is a risk of omitting important voices and producing skewed findings. Additionally, inclusivity carries an ethical responsibility to ensure that all groups, particularly marginalised ones, are heard. This session will explore how researchers can adopt a holistic approach to inclusivity across all stages of the survey process, from respondent engagement , sampling and questionnaire design to data analysis requirements. Key to this discussion is the concept of intersectionality—people’s experiences are shaped by multiple, interconnected factors, and ignoring this complexity can lead to oversimplified approaches to inclusivity.
Barriers to inclusivity in social surveys are wide-ranging, including accessibility issues, cultural insensitivity, technological challenges, and lack of trust amongst certain communities or disenfranchised individuals. The aim of this session is to reflect on how to overcome these barriers, to share successes, challenges and lessons learned. The session will provide a platform for researchers to present innovative strategies and approaches that ensure inclusivity, contributing to a more equitable future for social surveys.
We invite papers that address a variety of topics, including but not limited to:
• Inclusivity for individuals experiencing physical or mental health issues, or neurodiversity.
• Overcoming demographic or identity-based barriers.
• Engaging hard-to-reach groups.
• Addressing the complexities of intersectionality in social surveys.
• Building trust in social surveys.
• Leveraging alternative data to address inclusivity challenges.
• Inclusivity in the context of non-traditional data sources.
We particularly welcome submissions from researchers across different sectors—academia, national statistics, and private research organisations. This session aims to inform future strategies and best practices for enhancing inclusivity in social surveys.
Keywords: inclusivity, equity, accessibility, intersectionality, hard-to-reach, trust, barriers, alternative data
Papers
Harmonised Standards: Understanding the barriers to survey completion in the disability and impairment space.
Dr Alice Toms (Office for National Statistics) - Presenting Author
The UK Government Statistical Service Harmonisation team within the Office for National Statistics (ONS), is currently working on updating the disability and impairment harmonised questions (‘standards’). Disability is a complex topic, and so significant work has been undertaken to understand the problem space and gather a broad range of user needs and requirements. This talk will detail our qualitative approach to discovery, including focus groups, workshops, and cognitive testing of the existing and alternate versions of the disability and impairment standards.
The work is part of a wider suite of research within the ONS Future Population and Migration Statistics programme. The disability workstream focuses on several questions, including the long-lasting health conditions and illness standard, the activity restriction standard, and the impairment standard. These standards are designed to align with the Equality Act 2010 or the Disability Discrimination Act (1995) for Northern Ireland: ensuring comprehensive data collection that reflects the diverse experiences of individuals who are disabled. In particular, they capture the core disabled population according to UK legislation. The update aims to ensure increased inclusivity of the wider disabled population, whilst ensuring coherence, comparability and consistency of our data and statistics.
A recent review of the standards by the ONS Harmonisation team showed that user requirements have changed significantly over the last decade. There is an increasing argument for viewing disability in line with the social model of disability, rather than according to legislation. There are also data gaps where complementary surveys that captured additional progressive health conditions and other non-core elements of the disability standard have either stopped running or changed over time. Updating the standards is pivotal in working towards standardizing the collection of data to enhance the accuracy and comparability of statistics across the UK, as well as improving the inclusivity of our data.
Findings from the inclusive 2022-23 UK Veterans Survey using a self-select and data linkage method
Dr Tansy Arthur (non-member)
Ms Hayley Forrest (non-member) - Presenting Author
Project Overview
This project is about inclusion for a hard-to-reach population group. The Office for Veterans’ Affairs (OVA) commissioned the Office for National Statistics (ONS) to carry out the Veterans’ Survey to support the aims of their Veterans’ Strategy Action Plan, to make the UK the best place for veterans to live by 2028.
The research aim was to understand the unique experiences of the UK veteran population and their families living in the UK to inform future government policy.
We included a diverse group of stakeholders. Having broadened the scope from England to the UK, we worked with the Welsh Government, Scottish Government and Northern Ireland Veterans’ Support Office. Charities and Veterans’ Organisations supported the project.
Sample
A self-select sample was used for the veterans’ survey as a cost effective and inclusive method. Veterans are hard to reach, and a random household sample would include many households with no veterans, incurring additional costs. Data Linkage with the 2021Census would enable us to compare responses to population estimates.
Survey Development and Operation
The accessible survey was online first, with paper for those with no internet access. Questions were designed and tested collaboratively with veterans and other stakeholders. The collection ran from Armistice Day (11 November 2022) for 12 weeks. The survey promotion was through veterans’ charities and organisations. The overall response rate was over 24,000 and well above the target of 13,000.
Data Linkage
The processed data was linked to the 2021Census to measure representativeness. This showed a match of 98.4%, with differences in 2 regions (one higher, one lower), and fewer of the oldest veterans participating.
This paper will discuss the self-select survey method enabled by UK veterans charities and organisations, the data linkage to the 2021Census, and the published results.
Have hard-to-reach groups become harder to reach? Patterns of decline in response rates among the three major Scottish surveys and impact on representativeness
Mr Chris Martin (Ipsos) - Presenting Author
Ms Rachel Ormston (Ipsos)
This paper will analyse historic outcome data of the three major surveys in Scotland to examine whether hard to reach groups have become harder-to-reach, the potential impact on non-response bias, and the implications for future survey design. Major face-to-face random preselected surveys in Scotland, like their counterparts in the rest of Europe, play a crucial role in informing public policy. Face-to-face designs have been the preferred design partly because they achieve higher response rates and better representativeness than other modes. However, falling response rates and increasing costs have meant greater consideration has been given to alternative designs.
While overall trends in response rates show a clear decline over time, what is less clear is the impact that this has had on nonresponse bias. Although response rates have typically been used as a key indicator of survey quality and a measure of potential bias, the relationship between response rates and nonresponse bias is complex and a high response rates do not necessarily equate to low nonresponse bias (nor low response rates to high nonresponse bias.)
Particular types of groups have traditionally been under-represented in face-to-face surveys. These include younger adults, people from minority ethnic backgrounds, those with lower levels of education, those with lower literacy levels, and those living in flats. However, it is less clear if, relative to the overall population, those groups that have been traditionally hard-to-reach are becoming harder-to-reach over time, or whether they are sharing equally in the overall decline in response rates. The findings will contribute to boarder discussions of the future of survey methodologies and value for money on traditional face-to-face approaches.
Survey Accessibility to ensure Inclusiveness for Young People with Additional Needs.
Dr Wendy van Rijswijk (Department for Education) - Presenting Author
Mr Joas Flynn (Department for Education)
Ms Laura Hewitson (Department for Education)
Ms Samantha Spencer (National Centre for Social Research)
Ms Line Knudsen (National Centre of Social Research)
Over eighteen per cent of children and young people in England have identified Special Educational Needs or Disabilities (SEND). A targeted longitudinal study would help us understand their experiences and subsequent educational and adult outcomes.
We conducted a feasibility phase of the SEND Futures longitudinal study to:
1. assess whether representative participation and adequate levels of engagement could be achieved
2. identify design and implementation challenges to providing an inclusive survey for a population with heterogenous needs
This presentation will outline findings on engagement and attrition, drawing on both qualitative and quantitative data from parents/carers, young people, and interviewers.
The discovery phase involved two waves of fieldwork with young people with additional needs and their parent/carer. Wave 1 used face-to-face surveys for underrepresented groups and trialled an online survey with the wider SEND population. Wave 2 explored a web-first, mixed-mode design, which additionally offered telephone interviews to test whether overall engagement would increase and to provide an alternative means of participation to increase accessibility.
We will explore how types of additional needs were associated with participation across different modes, and why some young people were unable to participate. Furthermore, we will present new findings on perceived question comprehension and the level of support provided to young people to complete their surveys, and how this varied by needs.
We will reflect on the implications of these findings for understanding accessibility and how we can and should support young people to take part. We will set out potential challenges that arise from this, such as how the presence of and/or support from others might affect comparability and validity of the collected data. Finally, we will discuss how to balance inclusivity and accessibility with practical decisions (e.g., resources and timelines) in the design.
