Recruiting and surveying special populations 1 |
|
| Chair | Dr Anna B. Sandoval Girón (Center for Survey Measurement, U.S. Census Bureau ) |
| Coordinator 1 | Dr Susanne Vogl (University of Vienna) |
As Miranda (2004) has noted it is clear that surveying difficult-to-reach populations is a universal problem.” “It is universal in several senses. First, some populations, sub-groups, and individuals will always be hard to survey. Second, the reasons for populations and individual cases being challenging are quite diverse and complex. Difficulty is not simple, nor uni-dimensional. Finally, difficulty spans the globe. Wherever surveys are done, difficulties arise. But the specific mix of impediments can be very society specific and likewise the steps need to overcome them must be geared to the realities and complications that prevail in each survey in every country.
The total survey error paradigm can be used to examine the challenges in surveying hard-to-reach/hidden populations. Major components of total survey error that specifically relate to hard-to-reach/hidden populations include 1) sampling problems such as the need for special sample frames, 2) non-coverage or under-coverage, 3) nonresponse, and 4) measurement error such as misreporting.
First, sampling problems include sub- groups that are part of the target population, but which are not-covered or under-covered by available sample frames. If the sub-group is the focus of the study, an alternative sampling design often must be developed. Even if not under-covered, using a sampling design for the general population and screening down to the targeted sub-group may be inefficient and impractical due to cost. Dual or multiple frames or various follow-up or referral sampling methods will often be needed to adequately and efficiently sample small, hard-to-reach populations (i.e. “rare” populations). What can be done for a given target population will vary greatly from country-to-country depending on what information is available for sampling frames. For example, if a national census or administrative records fail to identity a particular sub-group or rare population, then sample frames based on those sources cannot target these sub-groups.
Second, while nonresponse is a problem for all surveys and all population groups , it is especially difficult for many hard-to-reach populations. Some of these populations involve groups that wish to avoid detection (e.g. sex workers, undocumented aliens), that are difficult to locate or contact (e.g. the homeless, nomads), or that are less able to do interviews (e.g. drug addicts, the mentally ill, alcoholics). These traits tend to lower response rates for these sub-groups. Moreover, response rates vary notably across countries.
Finally, measurement error from misreports contributes to the omission of hard-to-reach respondents. Rather than refusing or avoiding interviews, hard-to-reach respondents may thwart surveys by misreporting their status. For example, sex workers may deny engaging in prostitution, bankrupts may fail to report their insolvency , or undocumented aliens may report legal residence in their adopted country.
This paper outlines the methodological and ethical issues raised from a mixed-method study that focused on educational outcomes and careers of young people who grew up in poverty. The main aim of the study was determine the factors that contribute to successful transition of young people raised in poverty from education to work and exit from poverty. The sequential explanatory design was applied. Target population were young people aged 19 to 28, who lived at least three years in a household which was a welfare recipient during their high school age. Results show that 85,6% families were welfare recipients more than five years. In quantitative phase of study, three stage cluster sample was used while in qualitative phase extreme case sampling was used. Regarding the very low response rate in quantitative part of study, in this paper challenges associated with participants mistrust of the research process, social risks to participation, resource constraints and collaboration for the various parties involved in the study are discussed. Challenges in recruitment participants for qualitative part of study are also problematised. The dilemmas encountered are discussed in a way that illustrates the complexity of the processes of negotiating initial and ongoing consent as well as achieving benefits both for the young people and for the research.
Germany has experienced a significant influx of refugees since summer 2015, especially from Syria. Surveying this special population, special attention has to be paid to the target groups’ linguistic and cultural background as well as the specific situation of these refugees.
We assume the motive of a severe labor market orientation and the willingness to integrate into the labor market could be interpreted as return to the receiving and here especially the asylum giving country. Thus, we expect higher levels of reported work orientation or work attitudes compared to that of residents’ pattern.
To test this hypothesis we use data from two related surveys conducted in the year 2016, the IAB-Study „Youth unemployment, mental health and labor market outcome“ and the IAB study “WELLCOME”. Both surveys sampled young people 18 to 25 years of age, who entered the German unemployment register for the first time shortly before the interview. While the survey „Youth unemployment, mental health and labor market outcome” was designed for CATI interviews, the WELLCOME-Study employed both CATI and CAWI techniques, to reduce nonresponse bias by integrating refugees without telephone numbers in the survey.
The results indicate significant differences in respondents´ work commitment (Warr scale) between Syrian refugees and German residents. Syrian refugees report a significant higher work commitment compared to German residents.
Furthermore, we identified a significant mode-effect in answering behavior of work commitment. CATI respondents from the refugee sample (WELLCOME) report higher work commitment compared to respondents of the self-administered online-subsample (WELLCOME-CAWI population).
We assume, part of the mode effect could be connected to the interaction between respondent and interviewer. We expect that respondents reflect expectation of the receiving society, represented by the interviewer. These findings seem to be in line with the fact that WELLCOME-respondents’ (both male and female) report a higher level of work commitment in case of a male interviewer compared to female interviewer, given the fact that males report higher levels of work commitment in general. We apply logit models to control for individual and migrant-specific covariates and additional interviewer characteristics.
The question of sampling hard-to-access and special populations partly overlaps with that of rare populations for which there is generally no sampling frame. The traditional methods that allow this type of populations to be sampled, such as two-stage samples (with numerous variations) have many limitations since they are expensive to implement and can only be used when the population is fairly stable and easily identifiable. Collecting survey data from victims of dictatorships years after the end of those regimes is problematic because these populations tend not to be stable or easily identifiable. This paper addresses the difficulties in implementing a web survey and building a sample from a sub-population characterized by individuals who were politically active during an authoritarian regime – the Portuguese Estado Novo (1933-1974) – having been either arrested or imprisoned, exiled, underground, removed from civil service, and/or deserters or draft dodgers for ideological reasons. Our sub-population comprises a small share of the general population for which there is no contact list. Also, there are no official up-to-date numbers regarding the size of this sub-population or its socio-demographics, a number that diminishes by the year. Moreover, a sufficient sample of victims in Portugal would have been difficult to achieve via general population surveys, given that the population of interest has relatively low numbers and is geographically dispersed, which makes an investigation throughout the general population very expensive. So what to do to reach a sub-population with such characteristics?
We turned to this question during our research and we resorted to two strategies: first, we implemented a Multiphase Mixed-Method design combining three individual-level data collection methods: focus groups, followed by a web survey, followed by qualitative interviews; second, we also implemented a sampling strategy adapted to the existing context, targeting associations and organizations in Portugal, which was essential for the collection of the survey data. Taking into account the general development of ICTs, the widespread adoption of the internet in Portugal, the continuous decline in ICT-related costs and the salient characteristics shared by our sub-population, we have chosen to use web surveys as our primary data collection technique. Web surveys can be a vital tool to reach sub-populations that share a salient characteristic such as the victims of dictatorship. We will use our research experience of sampling a hard-to-reach and special subpopulation through a mixed-method design, including webs surveys as a primary data collection technique, to highlight the methodological challenges faced by researchers when using this type of surveys to collect data in the contexts of transitional justice.
Between February and November 2015, the French Institute for Demographic Studies (INED) conducted a quantitative survey on violence and gender relations (Virage) on a sample of 27,268 men and women aged 20-69 living in ordinary households in metropolitan France. Among these respondents, 503 reported being homosexual or bisexual. Small sample sizes are problematic for the statistical analysis of minority phenomena such as intimate partner violence and sexual violence. Moreover, the absence of sampling base for LGBT populations in France makes it impossible to correct for their small numbers in the general population.
To complement the representative sample of the general population, a targeted search was organized via the internet to increase the number of lesbian, gay, bisexual and transg (LGBT) respondents. From December 2015 to March 2016, LGBT individuals aged 18 or above were invited to take part in the Virage-LGBT online survey on a voluntary basis, whatever their practices or their definition of themselves:men who have sex with men, women who have sex with women, all persons defining themselves as gay, lesbian, bisexual or trans person. In a departure from the main Virage survey, for Virage-LGBT we launched a communication campaign which involved posting information on community websites, leaving brochures in bars and clubs, and sending emails to members of LGBT associations. The LGBT questionnaire, available exclusively online, was the same as for the main survey. A total of 10,612 people visited the questionnaire site, and 7,148 answered all the survey questions, giving a completion rate of 67%.
The paper aims to compare the advantages and drawbacks of these two data collection methods for analysing LGBT populations, and to examine the pertinence and comparability of the two samples.To this end, we compare their respective characteristics. We postulate that these characteristics are very diverse (practices, attraction, identification) and cannot be used to establish clear-cut categories; and that the LGBT populations are captured rather differently by these two collection methods.Questions about collection strategies and comparability are dissociable from those on data representativeness, and hence on the construction of LGBT groups.
To shed light on this point, we will compare the two samples on the basis of their sociodemographic characteristics (age, education, place of residence, town/city size, occupational category), their sexual trajectory and their lifestyle (identification, number of sexual partners, membership of community groups, social activity), and their health (consumption of psychoactive substances, mental health, HIV status). Several questions answered by the sample of volunteers can be used to trace the respondents' origins and, in the case of certain partners that advertised the survey (associations, information and dating sites, etc.), the size of the organization and of its membership.