Recent Challenges for Cross Cultural Research: Reflections on the International Social Survey Programme |
|
| Session Organisers | Dr Rene Bautista (NORC at the University of Chicago) Mr Ricardo Gonzalez (Universidad Adolfo Ibáñez) |
| Time | Friday 2 July, 15:00 - 16:30 |
The COVID-19 pandemic has posed challenges for survey researchers across the globe, more so for cross-cultural research programs such as the International Social Survey Programme (ISSP), whose aim is to achieve methodological comparability. Most ISSP modules are conducted face-to-face across countries, although some countries use alternative modes (due to the availability of adequate sampling frames and technology). The recent challenges due to the pandemic have varied by country, ranging from mode of administration, questionnaire adaptation, to sample balancing. Panelists from Asia, Europe and the Americas will share their experiences and lessons learned that serve to reflect on established practices, as well as innovations and new ideas for the future. Special attention will be given to sources of error of different data collection methods from a Total Survey Error perspective.
Keywords: ISSP, Comparative research, COVID19
Dr Evi Scholz (GESIS) - Presenting Author
Dr Oshrat Hochman (GESIS)
Over the last twenty years the German ISSP was fielded as self-completion part of a joint ALLBUS-ISSP survey. The ALLBUS is the biannual German General Social Survey based on a register sample of inhabitants with face-to-face interviews at the respondent’s home, usually fielded from spring to summer of the respective year. After the face-to-face ALLBUS interview, respondents were asked to complete the ISSP part of the survey themselves, i.e. in CASI mode with the interviewer attending. ALLBUS and ISSP thus use the same sample frame and design and the ISSP is integrated in the logistic organization of the ALLBUS survey.
Due to the covid-19 pandemic, a regular face-to-face survey could not be fielded in Germany where public life was locked down in March 2020 with unknown length of the pandemic and social and economic restrictions. To obtain data for the ISSP 2019 Social Inequality module, after discussion and approval of the ISSP Methodology Committee, the mode of the German ISSP 2019 survey was changed from CASI into mail and the survey run as stand-alone survey.
We will provide you with some information about consequences of this mode change with regard to questionnaire development and design, survey organization, and outcomes in order to spread information on how a probability based high quality survey could be conducted in Germany during pandemic times.
Dr Michèle Ernst Stähli (FORS (Swiss Centre of Expertise in the Social Sciences)) - Presenting Author
Mr Alexandre Pollien (FORS (Swiss Centre of Expertise in the Social Sciences))
Dr Michael Ochsner (FORS (Swiss Centre of Expertise in the Social Sciences))
Dr Marlène Sapin (FORS (Swiss Centre of Expertise in the Social Sciences))
Ms Karin Nisple (FORS (Swiss Centre of Expertise in the Social Sciences))
Dr Caroline Roberts (FORS (Swiss Centre of Expertise in the Social Sciences))
The Covid-19 pandemic and the related lockdown measures challenged many surveys in 2020. The general anxiety about the health crisis that was taking place transformed people's ordinary concerns. The media and conversations focused on the Covid issues, overshadowing other concerns (climate, politics). During the month of March, the general activity of the societies turned into a crisis economy. The lockdown was decreed, confining most individuals to their homes, closing schools and nurseries, dramatically changing people's daily lives, making people unemployed or homeworking. Time use and concerns were totally mixed up. Under these conditions, we can make the hypothesis that the response propensity to a general survey about a theme like the environment is upset. In this presentation, we will deepen this aspect, by showing how the pandemic affected the outcomes of the Swiss ISSP and its follow-up surveys.
In Switzerland, since 2018 the ISSP has been fielded in a self-completion survey project called MOSAiCH with a sequential web-paper design. This is probably the most viable data collection mode for a lockdown period such as in spring 2020. The switch from face-to-face to web-paper has, however, been implemented two years prior to Covid. The field of MOSAiCH 2020 started before the beginning of the first lockdown, and lasted until summer, when the measures were relieved. During the lockdown, we decided to add to the follow-up survey a newly developed questionnaire on COVID-19, which we repeated in autumn 2020 and spring 2021.
Capitalising on the two previous editions of MOSAiCH based on the same mode, the longitudinal design of the study in 2020, as well as the availability of administrative data for the whole gross sample, we investigate how the pandemic affected participation and the composition of the responding sample. We test whether the pandemic caused specific non-response biases related namely to time availability and whether the survey response propensity correlates with key measures of interest in the survey, on the topic of environment. As survey refusal is mostly argued by lack of time or of interest, we hypothesize that the potential effects of the pandemic are related to central concerns and to characteristics determining time availability, such as occupation and family situation (working in the health sector or for goods of primary necessity, having small children while day care centers and schools were closed). The presentation will focus namely on:
1) Design and basic outcomes of MOSAiCH 2020 (centred on ISSP 2020), a survey run during the COVID pandemic
2) Comparison of non-response bias between the ISSP 2020 edition and the two previous ones.
3) Detailed analysis of response by short time periods (as by pre- and after lockdown decision period, and contact procedure events).
Dr Gudbjorg Andrea Jonsdottir (University of Iceland) - Presenting Author
Dr Sigrun Olafsdottir (University of Iceland)
Mr Helgi Gudmundsson (University of Iceland)
Ms Gudny Bergthora Tryggvadottir (University of Iceland)
From the onset of the COVID-19 pandemic two ISSP modules have been fielded in Iceland. ISSP 2019 on Social Inequality was conducted as a face-to-face survey. This was done to secure comparability with the 2009 survey despite the fact that web-surveys have been the established mode of data collection for ISSP in Iceland for a number of years. The survey was conducted from spring 2020 to January 2021; and as it was known that people might be reluctant or unwilling to let interviewers into their homes, it was decided to give the choice of a video interview via a computer. Roughly half of the respondents (677) answered the survey in a face-to-face interview. Around 20% (222) were interviewed through a video interview. Towards the end of the data collection period, in an attempt to improve the response rate, two additional data collection modes were added, and respondents were offered to complete the survey as a telephone survey or a web-survey. Approximately 20% were contacted by telephone and were sent a link to the survey, which they completed by self-submission; and 4% (52) responded by telephone interview. The second module, the ISSP 2020 on Environment was fielded as a web-survey according to the research design that has been employed in Iceland since 2014. In both modules a random sample of around 3000 individuals was drawn from the National Population Register and advance letters sent by mail to introduce the surveys and their mode of administration. From the perspective of the Total Survey Error framework, we compare response rates of the two different approaches and explore the effects of data collection mode on representativeness of the samples and on data quality.
Mr Ricardo González (Universidad Adolfo Ibáñez) - Presenting Author
Chile has a strong reliance on face-to-face (F2F) surveys, even for electoral polls. They usually provide high-quality data, but face challenges in terms of coverage, increasing nonresponse rates and costs. Importantly, they are not possible to field when events disrupt the regular functioning of major cities in Chile, such as the social crisis in October 2019 and the COVID-19 pandemic since March 2020. Chile has been particularly affected by the pandemic. In August 2020, Chile was top 5 in terms of cases per million across the globe. To curb the contagion, the government set night-time curfews and targeted and dynamic lockdowns at the municipality (comuna) level, among other policies. In July 2020, roughly 60 percent of the population was on lockdown. Since early August lockdowns have been over in the center of the country, but came back over the weekends only in early December, affecting about 40 percent of the population. Around 20 percent of Chileans living in other areas are on full lockdown. For this reason, it has been impossible to field nationally representative F2F surveys to this day, such as the International Social Survey Programme (ISSP) module.
This presentation describes the processes, challenges and outcomes of setting up a mixed-mode probability-based panel in Chile over the COVID-19 pandemic. It presents the major deviations in terms of design from established longitudinal surveys in developed countries. Respondents are selected through probability sampling by random-digit dialing (RDD) of cellphone numbers across all regions and are asked to join the panel at the end of the interview. Online and offline households are included in the panel. Subsequent waves are conducted in a mixed-mode fashion, i.e. by web and phone. Recruitment, response and retention rates are analyzed under the two different incentive schemes: gift cards conditional on response and after collecting a certain number of points. The representativeness of the panel is assessed in each step. Differences across modes in substantive topics are discussed. Finally, the presentation addresses the potential of conducting ISSP modules using the mixed-mode probability-based panel and offers methodological insights for cross cultural research, particularly in Latin America.
Dr Pei-Shan Liao (Academia Sinica) - Presenting Author
COVID-19 has raised concerns in Taiwan since late January 2020, when the demand for face masks increased rapidly to prevent droplet infection. The Central Epidemic Command Center (CECC) soon announced guidelines on epidemic prevention measures for public transport, public gatherings, schools and academic institutions, and the scale of these measures has been enlarged gradually through April 30 when they tended to relax, while retaining minimal restrictions. Importantly, there have been no lockdown or stay-at-home restrictions in Taiwan. The Center for Survey Research (CSR) at Academia Sinica has been regularly conducting academic surveys for over two decades, mostly in-person and telephone interviews. Like in other countries, the scheduled surveys were disrupted by COVID-19. Despite the absence of strict lockdowns, the practices of fieldwork for CATI and CAPI in CSR were adjusted to reduce the risk to interviewers and respondents. This presentation discusses several anti-epidemic strategies taken by CSR in response to different phases of the COVID-19 outbreak in Taiwan, in particular for in-person interviews and the in-house CATI lab. This presentation will help provide perspective of survey research conducted in Asia and provide insights for cross cultural research.