ESRA 2025 Preliminary Program
All time references are in CEST
Methods for Within-household Selection in Surveys without Field Interviewers |
| Session Organisers |
Dr Nhlanhla Ndebele (European Social Survey HQ, City St George's, University of London)
Dr Ruxandra Comanaru (European Social Survey HQ, City St George's, University of London)
Professor Peter Lynn (Institute for Social and Economic Research, University of Essex)
|
| Time | Wednesday 16 July, 11:00 - 12:00 |
| Room |
Ruppert 005 |
The impact of the COVID-19 pandemic, along with the declining quality of interviewer-led and the promise of self-completion survey data collection has led social surveys to transition to surveys without field interviewers. However, this is not without its challenges. For probability-based surveys, the random selection of participants within households is challenging in the absence of field interviewers. This is particularly the case for sampling frames with no named lists of individuals, such as address-based sampling frames. Interviewer-led surveys often use the Kish method. However, this method is too complex to implement in the absence of field interviewers and quasi-random methods are used instead, with implications for participant selection.
This session aims to investigate methods and the impact of participant selection within households in surveys without field interviewers. The session welcomes paper submissions on various aspects related to within-household selection methods in probability-based self-administration or mixed-mode (with a component of self-administration) surveys. Topics may include, but are not restricted to, the following areas:
• Evaluation of the success or accuracy of within-household selection methods;
• Comparisons of alternative approaches to within-household selection, including between interviewer-led and self-administration surveys;
• New or novel methods for within-household selection.
We encourage papers from researchers from different disciplines and sectors, including academia, national statistics, and research agencies. We particularly welcome contributions that use experimental designs, and/or other designs that can inform future strategies for surveys conducted without field interviewers.
The session is proposed by Research Strand 4 of the Survey Futures project, “Surveys without Field Interviewers”. Survey Futures (surveyfutures.net) is a multi-institution collaboration research programme funded by the UKRI-ESRC, focusing on ensuring large-scale social surveys in the UK can innovate and adapt in a changing environment.
Keywords: social surveys, self-administered surveys, within-household selection, selection accuracy, sample representativeness, response rates
Papers
Testing two approaches to within-household selection in a postal push-to-web survey: findings from the 2024 European Working Conditions Survey
Dr Gijs van Houten (Eurofound) - Presenting Author
Dr Tanja Kimova (Verian)
Mr Christopher White (Eurofound)
Ms Hajar Gad (Verian)
Mr Jamie Burnett (Verian)
The European Foundation for the Improvement of Living and Working Conditions (Eurofound) commissioned Verian to conduct the eighth edition of the European Working Conditions Survey (EWCS) in Spring 2024 in 37 countries. As part of Eurofound’s strategy for future-proofing its surveys, the EWCS 2024 was conducted both face-to-face and online in all EU Member States.
In six countries a postal push-to-web approach was applied for the online segment, while using an address-level sampling frame. Here, a key challenge is the approach to within-household selection. To avoid having to ask respondents to apply a rule for randomly selecting a single person within the household, two different “census” approaches were used: inviting any two or any four eligible people in the household to participate in the survey. As the survey targets workers, the “any 2” approach invites all eligible respondents in around 90% of households and the “any 4” approach invites all eligible respondents in 99% of households. There is a first practical reason for limiting the number of invited respondents, which is that the invitation letter needs to include login details for each potential respondent and this list cannot be too long. A second reason is that the perceived burden on the household is likely to increase with the number of people that are invited to participate.
To test whether and how this affects the survey results, respondents in the six countries were randomly allocated to the “any 2“ or “any 4” condition. This paper will discuss the results of this test, assessing the differences between the two conditions in terms of yield, sample composition, and data quality, exploring the various trade-offs between cost-efficiency, self-selection biases, sample clustering, and sample composition.
Within-Household Clustering of Attitudes and Implications for Survey Sample Design When Only Address-Based Frames Are Available
Dr Nhlanhla Ndebele (European Social Survey HQ, City St George's, University of London) - Presenting Author
Professor Peter Lynn (Institute for Social and Economic Research, University of Essex)
The Covid-19 pandemic led many social surveys to transition, or consider transitioning, from interviewer-administered to self-completion methods. This raised debates on the best way to select individuals within households in the absence of field interviewers, with implications for survey sample design. This is particularly relevant in situations where individual-level sampling frames are unavailable, and it is necessary to use address-based frames. Considerations pertinent to this debate include the extent to which survey estimates are less precise if more than one respondent is selected from each household relative to independently sampled individuals, or whether additional interviews within the same household are any more informative relative to the first interview.
For surveys focusing on subjective measures, like attitudes, opinions, and values, there is little or no empirical evidence on the similarity of views within households. This makes it difficult to develop evidence-based survey sample designs. This paper aims to fill the gap by exploiting a data source from the United Kingdom that enables estimation of intra-household correlations across various subjective measures.
The paper will use data from the 2021 British Social Attitudes (BSA) survey, which is an important barometer of public attitudes on a variety of topics. These range from views on the left-right, libertarian-authoritarian, and welfarism scales, to the European Union referendum, migration, the economy, and culture. The BSA has a nationally representative sample, sampled from a postcode address file, with any two adults invited from each household and a concurrent push-to-web design alongside opt-in CATI. The within-household selection method allows for clustering analysis and enables the estimation of intra-household correlations. The paper will present estimates of the intracluster correlations and discuss implications for the choice of within-household selection method in attitudinal surveys.
An experimental comparison of within-household selection methods for self-completion surveys
Mr Nathan Reece (European Social Survey HQ (City St George's, University of London))
Mr Peter Lynn (Institute for Social and Economic Research, University of Essex) - Presenting Author
For self-administered postal push-to-web surveys in countries such as the UK, where only address frames are available, it is challenging to devise a method for within-household respondent selection that has both desirable statistical properties and high compliance levels. Several methods are currently used, with no consensus about their relative merits. This presentation reports a research project that aims to provide much-needed evidence about the relative effectiveness of alternative methods of within-household selection in the absence of an interviewer.
A commonly-used nonrandom method with cost advantages will be implemented alongside a random method, with sample units randomly allocated to each experimental group. Each is believed to be the most practical of its kind. The nonrandom method requests that up to two adults from each household respond to the survey, meaning that in households with more than two adults selection is not random and the probability of selection is unknown. The random method requests that the adult with the next birthday complete the questionnaire. Research questions address differences between the methods in population representation, precision of estimation, compliance, and costs.
Measuring the Impact of Within-Household Selection Accuracy on Representation in Cross-National Self-Administered Surveys
Mr Nathan Reece (European Social Survey HQ (City St George's, University of London)) - Presenting Author
Where address-based sampling frames are used, surveys rely on random within-household selection methods to implement probability sampling. In self-administered surveys, respondent selection must be conducted by household members, rather than a trained interviewer, which results in a notable proportion of incorrect selections (Olson et al., 2014). Little research has been conducted to assess whether incorrect selections are driven more by self-selection bias, disregard for instructions, or inability to understand instructions, nor whether the presence of incorrect selections in these surveys has a negative impact on representation.
The European Social Survey has implemented self-administered surveys in ten countries where the next-birthday method has been used for within-household selection. This study assesses the accuracy of the method in each country using data collected in the household grid to determine the approximate birthday of all eligible household members. Selection outcomes are determined to be correct, incorrect, or inconclusive. Where sample sizes are large enough, the impact of the accuracy of selections on representativeness is assessed by determining whether sample composition across multiple indicators is significantly improved by removing incorrect or inconclusive selections from the final sample.
Overall, selection accuracy decreases as average household sizes get larger and the task of having the person in the household with the next birthday complete the questionnaire becomes more complex. For multi-person households, differences between the selection outcome groups are generally not statistically significant. The impact of retaining incorrect selections in the final sample appears to be minor, and in most instances does not significantly worsen representation. For some indicators such as citizenship, retaining incorrect selections potentially improves representation, suggesting self-selection bias may not be a major cause of incorrect selections.
