All time references are in CEST
Education Bias in general population surveys |
|
| Session Organisers | Professor Tobias Gummer (GESIS - Leibniz Institute for the Social Sciences) Dr Elias Naumann (GESIS - Leibniz Institute for the Social Sciences) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
Nonresponse bias is a challenge for general population surveys in the social sciences. When the determinants of participation also relate to variables of interest, results drawn from survey data might be biased. Education is a key variable in the social sciences, frequently used in various research fields and correlating with many other variables of interest. Prior research hints at an underrepresentation of persons with lower education in general population surveys. Moreover, the recent transition from face-to-face to self-administered modes seems to have worsened the participation among respondents of lower education. Against this background, the present session aims to assess the magnitude of education bias for general population surveys, understand the underlying mechanism of nonresponse with respect to education, and stimulate the development of methods to improve participation among lower educated persons.
We invite submissions that address the challenge of education bias in general population surveys. Submissions should focus on one or more of the following aspects:
- Overviews of education bias in general population surveys to gauge the magnitude of the problem from a cross-national or longitudinal perspective
- Assessment of the mechanisms behind the underrepresentation of lower educated persons
- Comparison of survey modes and their impact on the participation of respondents with different education levels
- Analyses of the connection between education bias and nonresponse of other hard-to reach-groups (e.g., migrants) to arrive at a more pronounced definition of risk groups
- Empirical evidence (esp. from experiments) on the feasibility of methods to improve participation among lower educated persons
Keywords: education bias, nonresponse, general population surveys, participation
Mrs Leïla Saboni (Santé publique France) - Presenting Author
Mrs Noémie Soullier (Santé publique France)
Mrs Axelle Quiviger (Santé publique France)
Mrs Maria El Haddad (Santé publique France)
Mr Jean-Baptiste Richard (Santé publique France)
The reluctance to participate of people with lower education levels remains a significant challenge in general population surveys, particularly with the increasing use of self-administered web questionnaires. This issue is especially problematic in surveys on health opinions and behaviours, where education level is also a key determinant of the main variables of interest. To address potential biases, it is crucial to explore strategies for reaching underrepresented groups.
The French Health Barometer is a cross-sectional repeated survey, conducted by the French Public Health Agency, interviewing the population living in France about their opinions, behaviours and knowledge related to health. The 2024 survey was mostly collected with a sequential mixed-mode design, starting with a web-questionnaire followed by phone interviews. Benefiting from results of a pilot study conducted in 2023, we included a mail questionnaire as a last resort, with the aim of increasing participation especially from underrepresented populations. The mail questionnaire was designed to look like the web-questionnaire, but was also shorter.
Preliminary results show a clear association between education level and survey response mode: the higher the level of education, the greater the likelihood of responding online, while lower levels of education are associated with higher participation via the paper questionnaire. We will further study to which extent the inclusion of a paper questionnaire helps with reducing education bias. We will also investigate the specific profiles of respondents opting for this mode, regarding socio-demographic characteristics and health-related key variables.
This study underscores the importance of mode diversification in reaching underrepresented groups and provides insights into the profiles of lower-educated individuals who participate via paper questionnaires.
Ms Katrien Vandenbroeck (KU Leuven) - Presenting Author
Dr Audrey Vandeleene (Université Libre de Bruxelles)
Professor Bart Meuleman (KU Leuven)
Individuals with lower levels of education are often less willing to participate in surveys (Groves, 2005; Nicaise & Schockaert, 2014), a challenge also evident in probability panels such as LISS in the Netherlands (Knoef & de Vos, 2009). This issue similarly affects our newly established online probability panel in Belgium, The Social Study, where lower-educated individuals are underrepresented at recruitment and exhibit higher attrition rates over time. These patterns undermine the fundamental goal of research in probability panels, i.e. achieving a representative depiction of society.
While previous studies (e.g., Nicaise & Schockaert, 2014) have suggested that a general lack of trust may act as a barrier to survey participation, to our knowledge, little research has examined trust as a mediating factor.
This study fills this gap by examining trust as a mediator, using data collected during recruitment and early participation in The Social Study. We explore how different dimensions of trust—specifically, concerns about sharing sensitive personal information in general, trust in how public institutions handle such information, and trust in science overall—mediate the link between education level and participation rates and attrition. The findings aim to inform the development of strategies to reduce the underrepresentation of lower-educated individuals in probability panels, thereby contributing to more accurate and representative research outcomes.
Groves, R. M. (2005). Survey errors and survey costs. John Wiley & Sons. Chapter 5: Probing the Cause of Non-response.
Knoef, M., & de Vos, K. (2009). The representativeness of LISS, an online probability panel. CentER Data, Tilburg, 1-29.
Nicaise, I., & Schockaert, I. (2014). The hard-to-reach among the poor in Europe. Lessons from Eurostat’s EU-SILC survey in Belgium. In Hard-to-survey populations (pp. 541-554). Cambridge University Press.
Dr Mustafa Coban (Institute for Employment Research) - Presenting Author
Ms Christine Distler (Institute for Employment Research)
Professor Mark Trappmann (Institute for Employment Research)
The COVID-19 pandemic has accelerated a trend in survey research to use online data collection for general population samples. High-quality web surveys recently achieved response rates comparable to or even exceeding those of telephone surveys. However, selection bias concerning education is often more pronounced. To address this issue, we analyze complete employment biographies of both respondents and non-respondents and focus on three main research questions: (1) How do the different stages of the recruitment process for an online panel contribute to education bias? (2) Are there specific subgroups within the low-educated population who are even less likely to participate? (3) Are there interaction effects between education and other predictors of nonresponse?
In 2023, the Institute for Employment Research in Germany launched a new online panel survey of the German workforce (IAB-OPAL) using a push-to-web approach. Addresses were sampled from a powerful database comprising compulsory social insurance notifications by employers as well as unemployment insurance and welfare benefit records. We utilize this unique opportunity of a sampling frame containing detailed individual-level information on complete employment biographies.
Our findings indicate that educational bias accumulates at every stage of the recruitment process. We observed that unit nonresponse is more pronounced among individuals with lower education levels, particularly for respondents aged 50 and older and foreign nationals. Additionally, nationality appears to have a greater impact on highly educated individuals, and women are less likely to participate unless they hold advanced degrees.
Using a detailed sampling frame that includes individual-level information from complete employment histories enables us to evaluate how educational bias emerges throughout the recruitment process. It also allows us to determine if response tendencies within different educational strata vary based on typically unobserved factors, such as experience with benefit receipt, occupations, or wages.
Miss Julia Rokos (NIDI) - Presenting Author
Nonresponse bias is a persistent issue in population surveys. Previous studies suggest that individuals with lower education are often underrepresented, yet education levels influence a range of demographic, social, and economic outcomes. This makes educational bias in population surveys a significant problem for social science research. This issue may be exacerbated by the shift from face-to-face interviews to self-administered survey modes, which can discourage participation among lower-educated respondents.
The Generations and Gender Survey (GGS) provides an extensive dataset for examining demographic indicators, family dynamics, and attitudes. The GGS is currently in its second round of data collection (GGS-II) and has thus far been conducted in over 20 countries. Its high-quality data underpins a diverse array of research and policy analyses, contributing to addressing demographic challenges and improving societal well-being. However, like many large-scale population surveys, it may be vulnerable to educational bias. Since 2017, the GGS has been experimenting with conducting surveys in computer-assisted web interviewing (CAWI), and today most countries choose to field solely in CAWI, with some using face-to-face interviewing as a backup mode.
This paper aims to understand whether the shift to CAWI data collection introduces an educational bias in the data representativity of various GGS-II countries. Previous studies indicate that CAWI surveys may lead to underrepresentation on the grounds of self-selection in some populations. Our findings show that in general there appears to be a slight over-representation of medium higher educated individuals. However, this is stratified and differs across GGS-II countries, with some countries exhibiting a much higher discrepancy in distribution compared to the population than others. Furthermore, the probability of completing the survey after agreeing to participate is lower for individuals with medium or lower education levels.