Representation Error: Linking Sampling Design and Fieldwork Practices 1 |
|
| Session Organisers |
Dr Kathrin Thomas (Princeton University) Dr Salima Douhou (City, University of London) |
| Time | Friday 19th July, 09:00 - 10:30 |
| Room | D20 |
This section focuses on the representation side of the Total Survey Error framework linking fieldwork practices to probability sampling designs. Previous research has investigated the impact of sampling error on the “representativeness” of a survey’s target population. While confidence in good coverage of the target population is crucial, correct random sampling is also a highly dependent on the survey interviewers and their supervisors, as they select the dwellings, households, and respondents in the field. Depending on the cultural context, the availability of solid sampling frames, field organisations’ general guidelines, and other aspects, fieldwork practice may largely influence a survey’s representativeness.
We invite papers studying the extent to which fieldwork practices and sampling designs affect representation error using existing survey data. We are also interested in papers discussing coverage issues of (special) populations and in approaches mitigating these during fieldwork, including applied solutions to monitor, reduce, and tackle fieldwork influences on sampling designs. In addition, we welcome applied and novel approaches to estimate the impact of fieldwork practices on random sampling, instruments that may circumvent this problem, such as further automated respondent selection or random walks with geo-fencing, and other techniques to control random selection at all stages. Finally, we aim to attract papers addressing how fieldwork practices affect random sampling more generally.
Potential topics could include, but are not limited to the following:
• How can we best deal with the lack of sampling frames?
• Under what circumstances should special populations or subgroups of a population (i.e. internally displaced persons) be excluded? When do we consider under-coverage?
• What can we do, when the best available sampling frame is not good enough? Is it appropriate to apply additional methods to ensure good coverage? How would these look like?
• What fieldwork practices are harmful to the quality of the survey sample? How can we monitor these?
• Are there ways to implement a random walk in a way that sampling error is minimised?
• How can we improve methods at the doorstep to reduce sampling error, i.e., tackle interviewers or respondents (self-)selection?
• How is the data quality affected by different fieldwork practices in comparative or longitudinal studies? How much could/should we harmonise?
• How do house changes affect fieldwork?
We welcome contributions from various cultural contexts, survey practitioners, secondary survey data users, and academic researchers.
Keywords: sampling, fieldwork practice, representativeness, special populations, random walk
Mr Robin Houben (Robert Koch Institute) - Presenting Author
Mr Robert Hoffmann (Robert Koch Institute)
Mr Michael Lange (Robert Koch Institute)
Mr Hans Butschalowsky (Robert Koch Institute)
Dr Antje Goesswald (Robert Koch Institute)
Introduction
Over the last years, it has become increasingly difficult to convince individuals into participation in studies solely using written/postal invitations ("early response"). To achieve a representative net sample composition, further costly measures are necessary, e.g., F2F-recruitment, especially for hard-to-reach target individuals. The KiGGS Wave 2 data will therefore be used to identify indicators for future studies that predict in advance at which locations higher effort may be expected. KiGGS Wave 2 itself took place in 167 primary sampling units (PSU).
Methods
The analysis is performed for individuals of the KiGGS Wave 2 cross-sectional health examination survey in Germany (age group 3-17 years). The unadjusted gross sample comprises 9,230 persons. By means of logistic regression, the following independent variables were checked for their explanatory content for an early initial response (yes/no): age group, sex, nationality, region according to Nielsen and proportion of foreigners in the municipality/city.
Results
First analyses show that adolescents in the age groups 11-13 and 14-17 have a higher chance of early response than children in the younger age groups. Female individuals have a higher spontaneous willingness to participate. Target persons with German citizenship have a significantly higher odds ratio to participate spontaneously than people with a non-German citizenship. Region-specific differences in early response could not been detected. Detailed analyses will be presented at the ESRA conference.
Discussion
A logistic regression model based on freely accessible data on social structure may allow to predict response probabilities for different types of PSUs. This model can be used to support sample planning and personnel assignment to PSUs in future studies. Hence, resources could be used more efficiently, especially for hard-to-reach target individuals. The extent to which this model could also be transferred to surveys dealing with adults should be examined more in detail.
Professor Marcel Das (CentERdata)
Dr Merel Griffith-Lendering (MGL)
Ms Elly Hellings (IndigoBlue Consult)
Ms Marije Oudejans (CentERdata)
Dr Renske Pin (RE-Quest Research)
Miss Corrie Vis (CentERdata) - Presenting Author
Ms Jasmira Wiersma (University of Groningen)
Marcel Das, Merel Griffith-Lendering, Elly Hellings, Marije Oudejans, Renske Pin, Corrie Vis, Jasmira Wiersma
SONORO project: the sampling and recruitment process to build SONORO community (II)
The SONORO project aims to explore the relationship between financial literacy and health literacy in Curacao, with a focus on social relations and cultural values. To support health literacy and financial literacy research longitudinally, a panel environment will be implemented. Separate from the research questions there will be attention for other topics. In order to stimulate participation and continuation among the members these topics are discussed in interaction with the researchers. For this reason we speak of SONORO community instead of a panel.
The community is composed of individuals, representative for the whole population. For statistical reasons we need a random sample of Curacao addresses, but there are problems with the address registry: there are no postal codes, streets may have a new name, and addresses may have no number. To cope with these omissions we use a repository of global open address data combined with geographic coordinates.
A random sample of coordinates is uploaded in a digital map, which interviewers use on a tablet. The interviewers go to pointed addresses and try to contact the occupants. In order to gain insight in (non-)response the most important general characteristics are collected. At the end of the questionnaire respondents are asked to join SONORO community. Members receive an online follow-up questionnaire about their households.
In this presentation the results are shown of the sampling and the recruitment process for SONORO community and the pros and cons of the chosen method will be discussed.
Mr Stefaan Demarest (Sciensano) - Presenting Author
Mrs Finaba Berette (Sciensano)
Mr Youri Baeyens (Statbel)
Professor Geert Molenberghs (Hasselt University)
Dr Rana Charafeddine (Sciensano)
Miss Elise Braekman (Sciensano)
Dr Herman Van Oyen (Sciensano)
Professor Guido Van Hal (University of Antwerp)
Introduction
Although reviled as sampling technique, field substitution is used in the Belgian Health Interview (BHIS). The number of participants is predefined and set at 10,000 individuals. Based on data derived from the National Register (NR), non-participating households are substituted by a maximum of 3 households matched on district, age-group of the households’ reference person and household size. No information was available in the NR to enable substitution by socio-economic status. The aim of this study is to assess the impact of field substitution on the socio-economic composition of the BHIS 2013 net sample.
Methods
The educational level of the household’ reference person was used as a proxy for socio-economic status. Data on the educational level of the reference persons from initial selected households and their substitute households were derived from the Census 2011 and linked to the BHIS 2013 sample. Given the high level of missing data on the educational level (16%) in the Census 2011, regression based multiple imputations were applied. Participation rates by educational level for every stage of the substitution process were calculated. Differences in participation rates were assessed by applying the Delta method.
Results
At every stage of the substitution process, the participation rate was the lowest in low educated households (households with a low educated reference person) and significantly higher in middle or high educated households. Throughout the substitution process, the participation rate dropped from 51.6% to 42.7% in low educated households and from 61.7% to 46.3% in high educated households. The share of the participating households according to the educational level was similar before and after field substitution was applied.
Conclusion
Field substitution does not affect the socio-economic composition of the BHIS net sample.