All time references are in CEST
Quantitative and qualitative methods to survey hard-to-reach populations 2 |
|
| Session Organisers |
Dr Alessandra Gaia (University of Milano-Bicocca) Dr Daniele Zaccaria (University of Applied Sciences and Arts of Southern Switzerland (SUPSI)) |
| Time | Wednesday 19 July, 16:00 - 17:30 |
| Room | U6-03 |
Survey researchers often face the challenge to collect data on so called “hard-to-reach population” or “hard-to-survey” populations. These terms refer to population subgroups that are rare, marginal, hidden, elusive or excluded from mainstream society and thus hard-to-locate, sample, contact or interview. Examples include sex workers, illegal immigrants, victims of trafficking, drug users, displaced populations, homeless, institutionalised people, but also groups that – while not being excluded or marginalised – are rare and elusive (e.g. elites), hard-to-persuade to take part in surveys or hard-to-interview (for example due to lower cognitive abilities, like the oldest-old).
Exclusion of hard-to-reach population subgroups from data collection may lead to biased estimates on topics of relevance for social science research (e.g. poverty, inequalities, physical and mental health, social care, housing, migration, wellbeing, and social exclusion); ultimately, lack of information on hard-to-reach population subgroups lead to policy agendas which may not take into account the needs of the most vulnerable in society, and exacerbate social inequality and conflict.
However, meeting the need for high quality data is complex. To overcome this challenge, a number of quantitative and qualitative research methods have been developed, including: techniques to estimate the size of hard-to-reach populations (e.g. capture-recapture), sampling strategies (e.g. Respondent Driven Sampling), and data collection methods to ask questions about sensitive topics, including indirect questioning techniques (e.g. the Item Count Technique), adoption of proxy respondents, passive data collection through new technologies, participatory mapping, visual methods, etc.
We welcome submission on empirical or theoretical comparison of different research techniques, theoretical discussion of challenges faced by social researchers in surveying hard-to-reach populations and elaborations on the ethical principles guiding research on these population subgroups.
Keywords: Hard-to-reach populations, hard-to-survey populations, indirect questionning techniques, Respondent Driven Sampling, Passive data collection
Mr Jeremy Binz (IMPACT Initiatives) - Presenting Author
Mr Joeri Smits (IMPACT Initiatives)
Ms Nayana Das (IMPACT Initiatives)
In dynamic humanitarian emergencies, aid actors require a consistent flow of timely, reliable information about populations most in need to inform the response. However, information gaps appear because ‘gold standard’ representative household surveys are not always feasible, either due to insufficient resources, or inaccessibility due to insecurity or logistical difficulties. In response, IMPACT Initiatives developed the ‘area of knowledge’ (AoK) methodology, originating in 2014 in the Syrian conflict and implemented in 14 countries in 2023.
The AoK approach produces information about needs of crisis-affected populations by interviewing key informants (KIs) with recent knowledge of the situation in areas of interest. KIs are either interviewed where they reside, or they have recently been displaced from hard-to-reach areas, or have recently visited these areas (e.g. local traders, aid workers). Interviews can also be administered by phone where telecommunications are available. The key limitation of the AoK methodology is its reliance on aggregating the knowledge of a limited number of KIs, yielding indicative findings only.
This novel AoK methodology is often applied in areas where no other comparable data exists to enable triangulation, raising questions about data reliability. To address this, in 2021 IMPACT launched a validation study to assess the association of AoK data with superior sources of data. This paper will present results of the study's first phase, where data was collected through both AoK methods and representative household surveys in accessible and overlapping geographic areas in the Central African Republic and South Sudan. Here we present the results of the cross-sectional evaluation of the degree of association between AoK and aforementioned, superior comparators. In phase 2, from which some results may be ready for presentation, we aim to evaluate the ability of AoK data to capture longitudinal trends in humanitarian needs and conditions.
Dr Akuffo Amankwah (World Bank Group)
Dr Hibret Belete Maemir (World Bank Group)
Dr Pauline Castaing (World Bank Group) - Presenting Author
Dr David C Francis (World Bank Group)
Dr Amparo Palacios-Lopez (World Bank Group)
Dr Diego Zardetto (World Bank Group)
The informal sector represents a key source of livelihoods and accounts for around 70% of employment in the developing world. Despite the size and importance of the sector, informal businesses are mostly missing from official records, listings of active businesses and firm-level surveys. Recognition of this, and the increasing need to measure the activity and characteristics of the sector, has driven the development of methodologies to generate representative samples of informal businesses. As a result, two alternative approaches have been implemented to better capture the unique environment in which informal businesses operate: household surveys collecting very-detailed information on business operations, and enterprise surveys relying on area-based adaptive cluster sampling. The main advantage of the household survey is to collect key information on household demographic and economic characteristics that can inform intra-household analysis of determinants and consequences of informality. However, the sampling strategy is not designed to represent business density in the enumeration areas selected for the study, putting at risk the representativeness of the informal sector. Conversely, the area-based adaptive cluster sampling approach allows to render a representative sample of these business but provides few indicators of household livelihoods. By implementing the two approaches simultaneously in the same urban centers in Ghana, our paper illustrates both the potential and limitations of each of these approaches and explores the complementarity between the two. In addition to methodological contributions, our study provides recent estimates that helps to profile the informal sector in urban Ghana.
Ms Alex Bogdan (Ipsos UK) - Presenting Author
The Hunger in the UK research is a mixed method research programme commissioned by the Trussell Trust to support its mission to end the need for food banks in the UK.
This required a survey of a representative sample of people referred to food banks. Previous surveys had done this face to face in food banks, but during COVID a number of food banks switched to a delivery only model. A new method was required to enable us to interview a representative sample of this hard-to-reach group.
Our paper addresses several challenges and learnings from all stages of the research. For the food bank survey, we developed a stratified sample of food banks, using information on the number of referrals, placed paper questionnaires (along with instructions, online links, a translation sheet and freephone telephone) in parcels. We will review our sampling strategy and the results achieved. The survey was logistically challenging, requiring food bank staff to distribute invitations using strict protocols. An initial stage was recruiting food banks (that often had little or no paid staff) and preparing them to administer the survey.
A key stage of the research was engaging with stakeholders. We will share findings from interviews with area managers and food bank staff and how these translated into specific research design choices.
A parallel survey was conducted through an online random probability panel. Results provided a comparison for the survey of people referred to food banks.
Following from the referral to food bank survey, we carried out qualitative research with people to further understand their lived experiences, and drivers for food bank use. We will describe our approach to and learnings from qualitative engagement which involved design of materials, flexibility over participation, inclusion of breaks and split interviews and consideration of participant and interviewer wellbeing.