All time references are in CEST
Cross-national probability-based web or mixed mode panels 2 |
|
| Session Organisers |
Dr Gianmaria Bottoni (ESS ERIC HQ, City University of London) Professor Rory Fitzgerald (ESS ERIC HQ, City University of London) |
| Time | Thursday 20 July, 16:00 - 17:30 |
| Room | U6-06 |
Conducting high quality scientific social surveys in a national setting is per se a complex, costly and time-consuming task, but cross-national surveys face significant additional challenges. The historical development of the social survey reflects the ongoing tension between trying to maintain or reduce costs whilst also trying producing high quality data. The development of Information and Communication Technology has opened a new chapter in that complex interrelationship. Web surveys have several characteristics that make them an attractive alternative to more traditional modes of data collection and in particular to face-to-face surveys which are becoming increasingly expensive. With web surveys there are no costs of paying and training interviewers and the interviewer effect is removed from the equation. Answers to the questionnaire are captured in real time and reminders can be sent in a digital format once respondents are identified. In addition, the gap between questionnaire design and the start of fieldworks is substantially reduced.
For these reasons, web surveys are particularly suitable for implementing panel studies as once willing respondents are recruited their responses can be repeatedly captured without further in-person contact.
In the last years, several countries in Europe, Australia and USA have established on-line or mixed mode panels with a web component based upon probability samples.
This session is open to anyone who would like to present methodological findings from existing or planned cross-national web panels. Amongst the other topics, papers might cover: challenges linked to cross-national setting (e.g. input-harmonisation, translation issues, questionnaire design) representativeness, recruitment strategy, cost analyses, contact mode effects, incentive strategies, effects of device on measurement, mode effect, efforts to improve survey completion respondent behaviour, strategies to improve response rate, and the impact of including off-liners through alternative modes of data collection.
Keywords: Online survey methods, Cross-national web panels
Dr Eva Aizpurua ( ) - Presenting Author
Professor Gianmaria Bottoni (ESS HQ)
Previous research highlights the importance of contextual effects, showing how earlier questions on a survey can affect responses to subsequent questions. This is because prior items can prime certain beliefs, or serve as a standard of comparison for later questions.
In this study, we analyse the impact of question order on a series of items measuring the importance of multiple areas in one's life (i.e., work, family, friends, leisure time, politics and religion). For this, we use data from CRONOS, a probability-based online panel implemented in Estonia, Slovenia, and Great Britain during Round 8 of the European Social Survey (2016/2017). Using a between-subjects design, respondents were randomly assigned to one of two conditions: seeing family in second place, or seeing family in last place. To examine the impact of the order in which the family question was asked, we analysed the distributions of the variables included in the set, as well as their intercorrelations. We also explored the possibility of heterogeneous effects across countries and education levels. Based on previous research (Stark et al., 2020), we hypothesize that question order effects, if found, will be comparable across levels of education, but heterogeneous across countries. The results from this paper contribute to a growing body of research examining contextual effects in countries other than the U.S., administered in multiple languages. The implications of these findings for research and practice are discussed.
Mrs Živa Broder (University of Ljubljana, Faculty of Social Sciences) - Presenting Author
Mrs Tina Vovk (University of Ljubljana, Faculty of Social Sciences)
Dr Slavko Kurdija (University of Ljubljana, Faculty of Social Sciences)
In this paper, we analyze non-participants in social surveys. We are trying to find out which values are common to respondents who are willing to participate in research, and which to those who distance themselves from participation or have second thoughts. Our analysis will be based on data from the ESS R10 field survey, in which we recruited respondents to participate in the CRONOS2 online panel. Among the 1252 participants, who took part in the ESS field survey in Slovenia, a total of 80 % (1008) were eligible to participate in the CRONOS2 online survey. Approx. 70 % of them agreed to take part in online panel and approx. 30 % of them, despite participating in the field survey, refused to participate in the online panel. In this paper, we will try to extract the value profile of those who were ready to participate in the ESS basic research, but refused or did not respond to the invitation to participate in the online panel. Based on Putnam's theory of social capital, who refers to social capital as 'features of social organizations, such as networks, norms and trust that facilitate action and cooperation for mutual benefit', we can assume that, in addition to some demographic variables, the non-participants probably also have some values in common, such as trust etc. Trust can be categorized at two levels: (1; variables A4-A6) interpersonal trust, which derives from the network of interpersonal connections and relationships in an individual's immediate environment, and (2; variables B6-B12a) institutional trust, which derives from an individual's level of trust in systemic institutions. We assume that individuals who have a higher level of trust both at the individual and at the system level are more willing to participate in research.
Miss Christina Stabourlos (Department of Epidemiology and public health, Sciensano, Brussels) - Presenting Author
Miss Céline van Bilsen (Department of Sexual Health, Infectious Diseases and Environmental Health, Public Health Service (GGD) South Limburg, Heerlen & Department of Social Medicine, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht )
Dr Stephanie Brinkhues (Department of Sexual Health, Infectious Diseases and Environmental Health, Public Health Service (GGD) South Limburg, Heerlen & Department of Social Medicine, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht )
Miss Chrissy Moonen (Department of Sexual Health, Infectious Diseases and Environmental Health, Public Health Service (GGD) South Limburg, Heerlen & Department of Social Medicine, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht )
Mr Stefaan Demarest (Department of Epidemiology and public health, Sciensano, Brussels)
Miss Daniëlle Hanssen (Department of Social Medicine, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht & Department of Medical Microbiology, Care and Public Health Research Institute (CAPHRI), Maastricht University Medical Centre, Maastricht)
Dr Inge van Loo (Department of Medical Microbiology, Care and Public Health Research Institute (CAPHRI), Maastricht University Medical Centre, Maastricht)
Dr Paul Savelkoul (Department of Medical Microbiology, Care and Public Health Research Institute (CAPHRI), Maastricht University Medical Centre, Maastricht)
Mr Dirk Philippsen (Gesundheitsberichterstattung, Gesundheitsamt Düren, Düren)
Miss Brigitte van der Zanden (Foundation euPrevent, Heerlen)
Background:
‘Impact of COVID-19 on the Meuse-Rhine Euroregion’ was a prospective longitudinal study conducted in the border area between Germany, Belgium and the Netherlands. Collection of comparative cross-border data enables to assess the pandemic response and the impact of infectious disease control measures in border regions. To facilitate a uniform approach on all sides of the borders in the Meuse-Rhine Euroregion (EMR), an interregional partnership was set up. In the spring of 2021, citizens were invited to collect a blood sample at home using a self-finger prick test and fill in an online questionnaire. In Autumn participants were invited again for a follow-up round. An online tool was developed to coordinate fieldwork procedures and the data collection process.
Results:
In the first round, 6.006 citizens in the EMR participated. 15.3% of the invited citizens in Belgium participated. In the Netherlands and Germany this was respectively 27% and 26.7%. In the follow-up round 71.4% participated. The participation rate was highest in the age group 50-69 years and lowest from > 80 in all sub regions of the EMR. In terms of gender, women were more likely to participate.
Conclusions:
Aligning the project’s objectives with national legislations was challenging and distorted simultaneous and uniform data collection in the first round. This hampers the comparability of cross-border data for indicators that are time-specific such as compliance with infection prevention measures and results of the SARS-CoV-2 antibody testing. The centralized online environment however led to cross-border collaboration on equal footing where transparency and a real-time monitoring of participation rates was possible for all stakeholders involved, while respecting GDPR regulations. The tool facilitated informed decision making processes such as correcting for lower participation by inviting reserve invitees.
Mr Diogo Dinis (Institute of Social Sciences, University of Lisbon) - Presenting Author
Dr Alice Ramos (Institute of Social Sciences, University of Lisbon)
Although face-to-face is proven to be the most reliable method of data collection, its implementation is becoming more and more difficult in many countries due to the number of resources needed. Moreover, response rates are decreasing, which represents a problem of the achieved samples’ representativeness. Thus, online surveys are quickly gaining ground; they are less expensive and allow researchers to cover larger samples. But they also have fragilities. In this presentation, we explore Portugal’s first run in a web panel survey, the Coss-National Online Survey (CRONOS) comparing ESS10 respondents that accepted and didn’t accept to join the panel, considering two points of view: 1) the characteristics regarding socio-demographic composition of the two samples; 2) their attitudes regarding social issues. Regarding samples’ composition, we found significant differences: panelists are younger (the only age group with no differences between groups is the 44-65 yrs), more educated (almost a difference of 2 years between groups). There are no differences in gender composition. Regarding attitudes towards social issues we found differences, but also similarities, which gives support to the quality of the panel. For instance panelist and non-panelists do not differ regarding political orientation, institutional trust, satisfaction with the economy and the government or perceptions of threat regarding immigrants. On the other side, panelists show higher means in openness to immigration, political interest or perception of political self-efficacy. However, the magnitude of the effects is very small in any of the cases. Since the evidence suggests that, at least in Portugal, panel composition may have an impact on social attitudes, this can be a valuable point of discussion in order to develop panel survey’s best practices, regarding sampling procedures and data analysis techniques. Limitations and directions for future research will also be addressed.
Ms Blanka Szeitl (Eötvös Loránd University) - Presenting Author
Ms Zita Fellner (Eötvös Loránd University)
Ms Annamária Tátrai (Eötvös Loránd University)
Online data collection may seem to be one of the new directions of survey research in times of crisis and thus, the quality of online survey is under a permanent pressure. With online-only data collection, a probability sample is not feasible, which may compromise the generalisability of the results and bias the estimates. Authors present a case study of Hungary, where internet access is far from reaching full penetration (89 percent) but considered average by European standards. The study focuses on two main points: (1) the extent to which estimates from online surveys are biased in general, and (2) the socio-demographic and attitudinal aspects relevant to the magnitude of the bias. The main method of the analysis is simulation, which is based on multiple data sources. Based on administrative data the demographic composition is modeled for both offline and online populations, for the attitude dimensions face-to-face survey data of the European Social Survey is used. The study evaluates estimates from simulated online and face-to-face data collections involving several post-stratification processes. The overall conclusion of the study is that although with relatively high internet penetration, online data collection seems to be a convenient data collection tool for social research, it is not yet a suitable method. The study found that even a minor data gap from the offline population leads to major error in the estimates: based on the characteristics of internet penetration in Hungary, in 76 percent of the cases erroneous estimates were obtained. For relevant research dimensions such as interest in politics, religiosity, health and marital status, the online data collection significantly under- or overestimates the presumably real population proportions. With the appropriate administrative data, the analysis can be carried out for other countries, providing an analytical tool to describe local capabilities.