All time references are in CEST
Social Media Ads as a tool for targeted survey sampling 1 |
|
| Session Organisers |
Dr Michael Zoorob (Meta Platforms) Dr Steffen Pötschke (GESIS - Leibniz Institute for the Social Sciences) Dr Bernd Weiß (GESIS - Leibniz Institute for the Social Sciences) |
| Time | Thursday 20 July, 09:00 - 10:30 |
| Room | U6-05 |
The sampling of hard-to-reach populations remains a challenge in survey research. For example, conventional survey methods are often incapable of reaching very mobile groups, such as international migrants, or employees of specific industries, who are defined by an attribute that is usually not included in readily available sampling frames in most countries. Additional challenges occur in the field of comparative social research when a researcher must grapple with multiple national sampling systems that prevent consistency in sampling design.
Though not without its problems and challenges, the wide reach and detailed targeting options made available by social media advertising provide a potential solution to some of these core-problems in survey research. A nascent but growing suite of research tools, including automated Chatbots that provide customized messaging and automated survey response collection, has developed to enhance this approach. Recent research has underlined that social media constitute a useful recruitment tool allowing timely and cross-national data collections on relevant issues such as (forced) migration and the COVID-19 pandemic. This panel will discuss recent empirical research that samples via targeted social media advertising sampling in the absence of feasible alternatives. Preference will be given to contributions that, besides demonstrating the added value of this sampling approach, address additional methodological issues in terms of survey quality, e.g., sample composition in comparison to appropriate benchmarks, or nonresponse bias. We also welcome contributions that discuss issues of research ethics when applying this sampling approach.
Keywords: Sampling, sampling frames, social media, advertisements, hard-to-reach populations
Ms Jessica Donzowa (Max Planck Institute for Demographic Research) - Presenting Author
Dr Daniela Perrotta (Max Planck Institute for Demographic Research)
Professor Emilio Zagheni (Max Planck Institute for Demographic Research)
During the COVID-19 pandemic, we have witnessed many primary survey collection efforts, especially relying on advertisements using social media recruitment. However, such surveys may be biased due to self-selection depending on interest in the study subject. In this study, we assess how the display of the survey topic in advertisements during recruitment affects responses. For this, we use the survey data collected via the “COVID- 19 Health Behavior Survey” that ran between March and August 2020 in eight countries in Europe and North America (N = 133,069). Respondents’ recruitment took place via Facebook advertisements that differed in the image shown in the ad. Here we investigate whether survey responses resulting from images with stronger (or weaker) relation to COVID-19 are associated with higher (or lower) compliance with recommended preventive behaviors, such as wearing a face mask, increasing hand hygiene, or perceived threat of the pandemic. We observed no image effect on the outcomes in the majority of the countries. Where an effect was found, there is no clear association between higher topic salience and the adoption of protective behaviors or higher threat perception. Our findings are reassuring regarding the low impact of self-selection bias on survey responses based on topic interest, thus strengthening the reliability of surveys using social media recruitment during health crises.
Mr Björn Rohr (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Dr Barbara Felderer (GESIS - Leibniz Institute for the Social Sciences)
Dr Steffen Pötzschke (GESIS - Leibniz Institute for the Social Sciences)
Dr Jan Priebe (Bernhard Nocht Institute for Tropical Medicine)
Dr Henning Silber (GESIS - Leibniz Institute for the Social Sciences)
Dr Bernd Weiß (GESIS - Leibniz Institute for the Social Sciences)
Sampling respondents via ads on social media is a promising new approach to reach respondents both in hard-to-sample regions and at a cross-national level. As one option, Meta optimizes their advertisement to reach the most clicks on a provided link (e.g., the survey link). The optimizing algorithm includes the layout of the ad and user characteristics. Knowing this, multiple targeting strategies are possible, for example, targeting all Meta users of a selected group or within separate strata of that group. Yet, the influence of the targeting strategy on the sample composition is less studied.
In this study, we evaluate the sample composition of surveys we conduct using Meta ads in Sub-Saharan Africa with varying Internet coverage. Specifically, we ask which targeting strategy leads to a survey respondents' composition that comes closest to the respective benchmarks of the general population. As benchmarks, we use socio-demographics and health-related variables such as vaccination prevalence from the Demographics and Health Survey (DHS), the Afrobarometer, and administrative data. Our target sample consists of Facebook and Instagram users from three African countries (Ghana, Kenya, and South Africa). We vary the two sampling strategies over the course of several weeks. Our stratified targeting strategy groups the countries by region, age, and gender, whereas the second sampling strategy is without demographic targeting. The study results inform researchers on how a Meta survey of African countries compares to surveys realized through a probability-based approach and provides insights into the advantages and disadvantages of employing stratified targeting as a sampling strategy.
Mrs Zaza Zindel (Bielefeld University) - Presenting Author
Mr Simon Kühne (Bielefeld University)
Mr Emilio Zagheni (Max Planck Institute for Demographic Research)
Mrs Daniela Perrotta (Max Planck Institute for Demographic Research)
A growing number of research projects are using social networks such as Facebook to recruit (infrequent) populations to participate in online surveys. These social media platforms are based on an advertising revenue model that allows researchers to recruit survey participants by buying ads. However, researchers need more information about how users view the ads, perceive them, and why they ultimately participate in a survey. As a result, different ads can lead to polarized response patterns that favor, for example, users with very strong opinions on certain survey topics.
To date, there is no evidence in survey research on how different ad designs affect the composition of survey respondents.
Our presentation includes a Facebook ad experiment conducted as part of a web survey project on climate change and migration in 2022-2023. We contrast different ad images and examine whether the polarizing features of the ads are also reflected in the survey data. The talk thus provides valuable insights into data quality hurdles, such as sampling errors in survey data generated in this way. Moreover, our examples can serve as a starting point for further research.