All time references are in CEST
Conventional and Innovative Sampling Methods for Hard-to-Survey Populations |
|
| Session Organiser | Dr Melike Saraç (Hacettepe University Institute of Population Studies) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
Hard-to-survey populations typically refer to sub-groups of the total population that are difficult to define, cover, select, interview, and adjust for post-survey procedures in surveys. It is important to include these sub-groups in surveys so that policy makers can develop evidence-based policies in many areas (e.g., health, nutirition, disability, education, labor) for these groups relying on survey or registration data. Tourangeau (2014) discusses the terminology of hard-to-survey populations in the survey context. He elaborates on the distinction between hard-to-sample, hard-to-identify, hard-to-find and hard-to-interview in detail.
This session aims to discuss conventional and innovative sampling and data collection methods, in particular to cover hard-to-survey populations targeted in social surveys. The session welcomes studies dealing with methods for the sample frame, sample selection (probabilistic vs non-probabilistics), data collection, fieldwork supervision, and post-survey adjustments including weighting and calibration. The pros and cons of new sampling ideas in the digital age will be discussed in addition to traditional methods. Comparative or experimental studies as well as studies based on mathematical calculations are therefore also to be welcomed. Researchers from different countries would contribute by sharing their survey experience of sampling and collecting data from hard-to-survey populations.
Keywords: sampling, hard-to-survey populations, coverage, weighting
Professor Katherine McLaughlin (Department of Statistics, Oregon State University) - Presenting Author
Respondent-driven sampling (RDS) has been adopted worldwide as an important method for sampling vulnerable and high-risk populations to enable vital decisions about resource allocation and program planning and implementation. However, RDS relies on many assumptions about the population and sample dynamics, and continued innovation is needed to ensure that RDS surveys are implemented correctly to meet these assumptions. In this paper, we introduce the directed seed method as a modification to RDS where seeds work with an interviewer to enumerate their potential recruits across a number of important characteristics using a diversity recruitment grid and are then instructed on which of these people they should recruit. The directed seed method shows promise for enhancing the recruitment of additional diverse individuals and overcoming potential bottlenecks in the population network by concentrating bias in the seeds and their recruits and then allowing the RDS sample to proceed without interference. We found that directing seeds to recruit across identified bottlenecks can be helpful to reach a more diverse group of population members sooner and to try and ensure that all important subgroups in a population are included in the final sample. We provide an example from surveys conducted among Venezuelan refugees and migrants in Colombia in 2020 to introduce the steps needed to implement the directed seed method. We assess the method using a variety of existing and novel diagnostic tools, including visual diagnostic techniques like recruitment chain, bottleneck, and all points plots and metric-based diagnostic techniques like recruitment homophily, expected gain in wave 1 diverse participants, and standard deviation of seed-based characteristic estimates based on a threshold. Finally, we discuss best practices for implementation.
Dr Tansy Arthur (Non-member (at the moment)) - Presenting Author
Project Overview
The Office for Veterans’ Affairs (OVA) commissioned the Office for National Statistics (ONS) to carry out the Veterans’ Survey to support the aims of their veterans’ Strategy Action Plan, which is to make the UK the best place for veterans to live by 2028.
Stakeholders on the project included the Welsh Government, Scottish Government and Northern Ireland Veterans’ Support Office. Charities and Veterans’ Organisations such as COBSEO: The Confederation of Service Charities supported the project.
Background Rationale
The UK Veterans’ Survey was the first project of its kind in the ONS. The research aim was to understand the unique experiences of the UK veteran population and their families living in the UK to help inform future policy created by the government.
Sample
A self-select, respondent driven sample was used for the veterans’ survey. Veterans are hard to reach, and a random household sample would risk targeting households that did not have veterans, incurring additional costs, and low response due to missing veterans’ households.
Survey Development and Operation
The survey was online first, with paper for those with no internet access. Questions were designed and tested collaboratively. The collection ran from Armistice Day (11 November 2022) for 12 weeks. The survey promotion was through veterans’ charities and organisations. The overall response rate was over 24,000 and well above the target of 13,000.
Data Linkage
Once the data was processed it was linked to the 2021 Census to measure representativeness. This showed a match of 98.4%, with differences in 2 regions (one higher, one lower), and fewer of the oldest veterans participating.
This paper will discuss the self-select survey operation enabled by UK veterans charities and organisations, the data linkage to the 2021 Census, and the published results.
Mr Patrick Brock (WB/UNHCR Joint Data Centre)
Mr Andrej Kveder (UNHCR) - Presenting Author
Mrs Anqi Pan (WB/UNHCR Joint Data Centre)
Collecting socioeconomic data in forced displacement contexts presents multiple methodological challenges and sampling in particular. Those affected by forced displacement are often vulnerable and hard to reach for the purposes of household surveys, and sampling frames are rare and of low quality. Therefore, many surveys in these contexts use less-than-ideal convenience-based sampling. As intervention planning, programming and policymaking in response to rising global forced displacement intensifies, the need for high-quality and comparable socioeconomic data on forcibly displaced grows.
This research explores the use of open data to respond to these challenges. An innovative sampling approach will be compared to alternative list-based sampling approaches, and we discuss the implications for survey implementation in terms of cost, logistics, and statistical estimation. The proposed paper discusses how combining UNHCR publicly available data with satellite imagery and Google Open Buildings data made an integrated sampling approach for the UNHCR Forced Displacement Survey in South Sudan possible.
The second challenge discussed in this paper is the comparability of data collected on forcibly displaced persons with those from the communities where they live. The main challenge lies in finding a generalizable definition of host communities, for example, in the degree to which host nationals are affected by the presence of refugees, such that it can be operationalized into a concrete sampling approach. The paper demonstrates the example of gravity sampling using Euclidian distance between host community dwellings and refugee camp boundaries in combination with a gravity coefficient. The paper also uses sensitivity analysis to explore how this approach overlaps with alternatives, which aim to more directly incorporate the interaction between refugees and host communities.
In conclusion we will investigate biases as well as representativeness of the samples and will elaborate on how they can be addressed through different weighting strategies.