Recruiting and surveying special populations 2 |
|
| Chair | Dr Anna B. Sandoval Girón (Center for Survey Measurement, U.S. Census Bureau ) |
| Coordinator 1 | Dr Susanne Vogl (University of Vienna) |
The Consolidated Appropriations Act of 2014 required the US Bureau of Justice Statistics (BJS)/Department of Justice to examine whether Uniform Crime Reports, the National Crime Victimization Survey, or other relevant data series should collect and report data on honor-based violence (HBV). Westat was tasked by BJS to examine issues related to estimating the prevalence of HBV in the US. The initial report from this study concluded that representative surveys of the general population would have to be extremely large to identify victims of this rare crime and that asking victims to respond to such a survey in their homes would be extremely dangerous for those victims (Helba et al., 2015).
This paper reports on the reliability of another source of estimates concerning the prevalence of honor-based crimes: databases created through standardized searches of media and legal resources. In particular, it considers the problem of the exhaustiveness or completeness of the information sources; and, therefore, the databases. Such databases are subject to limitations stemming from the availability of information, i.e., cases that are covered in the media or reach the legal sector; and the definition or criteria used by the researchers for the identification of potential cases. Case reporting is affected by the perceived “newsworthiness” of a case (e.g., is violent or “exotic”), media stereotypes of HBV, and the information available to the media or the legal system (i.e., the case is reported, reporting is not restricted by medical or legal regulations). Case definition is affected by the researchers’ understanding and stereotypes of HBV, including the gender, race/ethnicity, and religion of the victims and perpetrators, and the behaviors or acts considered. This paper compares several databases developed for the estimation of HBV, including three databases from the US and three databases from Europe.
The French random telephone survey Virage conducted in 2015 aims at describing violence ; among the 27,268 respondents aged 20-69, only 500 described themselves as Gay, Lesbian or Bisexual (LGB, i.e. 1.6 % of the weighted sample). This number is relatively small for analyses stratified by gender. This proportion may in addition be underestimated because of underreporting due to inteviewer effect and undercoverage of a part of the LGB population that would not answer the phone. We present a method aiming at partially correcting the undercoverage and increasing the analytic sample size of the LGB. Because there were comparability constraints with older similar surveys, the method prevents any measurement difference due to the data collection mode on the main variables of interest.
In partnership with the major LGB associations of the country and with an extensive communication plan in LGB places and cultural events, in the social networks and in the LGB online communities, we replicated the Virage survey online on a volunteer basis. We obtained 7100 LGB web questionnaires among which 5700 were in the target population (20-69 years old living in ordinary dwelling in metropolitan France). The phone and web questionnaires were identical (average completion time=1h07 mn). Internet LGB were younger, more educated and more often victims of violence than their telephone counterparts.
Targeting LGB that go out in LGB places in both surveys and knowing the proportion of internet LGB that do not answer the phone when the calling number is unknown led us to estimate that 10.5% of the LGB population was not covered by the telephone survey, yielding a more realistic estimate of the LGB population (1.8%).
In a second step, we conducted a propensity analysis of the LGB questionnaires only, modelling internet or phone answer. We ran two models separately : one for the internet LGB that were reachable by phone, and one for those who were not. The models included gender, age, some sociodemographic and 4 variables that synthetise the primary interest of the survey, violence. These 4 interest variables are the number of offences during the last 12 months that were: sexual, physical, psychological or psychological with a sexual content. Internet questionnaires were matched 1:2 on the propensity score and balance between samples was checked.
The method allowed us to include 2000 LGB internet questionnaires into the random telephone survey. A pseudo sampling weight was computed for these additional internet questionnaires and the total (telephone and internet) sample was calibrated using the additional margin built to compensate for the LGB undercoverage.
Advantages and limits are discussed.
Methods for sampling rare populations have been extensively researched in the past decades. The focus of the research has been mostly on reducing the cost of locating and sampling rare youth populations when no lists are available. In such cases, probability-based sampling techniques usually require a substantial amount of screening to locate eligible cases, making it cost prohibitive for most surveys to reach adequate sample sizes for the desired analyses. This is even more challenging for developing countries, given their survey environments and the availability of resources. Innovative approaches that make sampling rare populations logistically practical and financially more affordable for developing countries are the main focus of this presentation.
This presentation will focus on sample designs for the Programme for the International Student Assessment for Development (PISA-D) Strand C, in which the target population is 14 to 16 year olds who are out-of-school or in lower than seventh grade. We will discuss stratification, disproportionate sampling, use of multiple frames, alternative designs to the traditional probability-based methods (such as location sampling, snowballing, link-tracing, etc.), combining probability and non-probability samples for analyses, evaluating potential bias for non-probability sampling, screening and misclassification, and different and creative data sources as sampling frames. The sample design for the field trial from six countries will be summarized.
Declining response rates and increasing data collection costs are adversely affecting probability-based surveys. Hence, online-based sampling and recruitment methodologies have emerged and have been examined as an alternative approach to address cost, non-response, and coverage concerns related to the probability-based sampling strategies. The dramatic national and global increase in social media use and its potential to lower cost per complete allow this platform to be considered as a potential sample recruitment venue. Previous research has shown that social media sites can be used for recruitment to web surveys yet the findings are equivocal and suffer from two shortcomings. First, they tend to rely on only one social media site (e.g., Facebook). Second, they rarely examine the usefulness of social media platforms for their ability to reach targeted populations. Our paper reports findings from two studies that used social media and search engine ad campaigns to recruit targeted populations for web-based surveys. In our first study, we employed targeted ad campaigns via Google, Facebook, and Twitter to recruit 18-25 year olds to complete a web survey regarding health and social life. Given the success of Facebook reaching targeted populations in our first study, our second study focused on recruiting another targeted population (Asian-Americans) to a web survey on financial well-being via Facebook-based ad campaigns. We examine differences in response rates by platform, as well as the substantive responses of our respondents in comparison to similar questions asked in general population probability based surveys. We further explore how each of the social media and search engine platforms perform relative to each other. Ultimately, the results expand our understanding of using social media and search engine ads for targeted survey recruitment both in terms of their strengths and shortcomings.
Recruitment via social networks is becoming increasingly popular. It is tempting of having access to millions of their members and specialized agencies offer possibilities to implement a focussed targeting on gender, race or age group etc. in a fast and cheap way.
But just one of the problems recruiting participants via Facebook is the validity of the resulting data.
The first experiment on collecting data via social media for the Robert Koch Institut was conducted between December 2012 and January 2013. We recruited young women (18-25 yrs.) for an online survey related to HPV-immunization within German Facebook. The result was presented at the ESRA 2013 conference. Since then we conducted another survey using Facebook for recruitment (between May and July 2015); subject was the parental acceptance of seasonal influenza vaccination for children. We recruited participants on Facebook and additionally as a reference on a traditional random-digit dialing telephone survey. For this, we only asked for information about households with one or more children. On the household level we did not use random sampling in the telephone survey. On Facebook we recruited 1,419 participants and 518 on the telephone. In particular woman participated in both roll outs (Facebook: 86%; Telephone: 73%). One of the interesting points is that the number of a migration background in the telephone survey is with 31% nearly double than in the Facebook arm.
The method aim and question of this study was to find out about possible differences between target populations for the outcome variables and also to compare the standard demographic characteristics between Facebook and telephone. The presentation gives a review about the two samples, about the cost and the results in terms of demographic characteristics. Additionally we can provide an insight of possibilities to control the survey via Facebook with distributed adds.