All time references are in CEST
The International Social Survey Programme (ISSP) – Methodological innovations and substantive research in comparative perspective |
|
| Session Organisers | Professor Stephanie Steinmetz (University of Lausaane) Dr René Bautista ( NORC at the University of Chicago) Dr Ricardo Gonzalez (Universidad Adolfo Ibáñez) Professor Ioannis Andreadis (Aristotle University of Thessaloniki) Professor Markus Hadler (University of Graz) Professor Eglė Butkevičienė (Kaunas University of Technology) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
In an increasingly interconnected world, understanding public attitudes and behaviours around the world is crucial. While many cross-national surveys are limited to specific topics or conducted sporadically, the International Social Survey Programme (ISSP) stands out as a continuous, large-scale initiative that has provided high-quality cross-national data and long-term time series since 1985. The ISSP's unique organisational structure, based on democratic cooperation among equal partners, promotes adaptability and responsiveness to new challenges. As a result, the ISSP covers a wide range of topics and provides valuable insights into social science research.
A key strength of the ISSP is its rigorous methodological approach and a dedicated committee of experts that ensures that national sample designs meet the program's strict quality standards before they are approved, thus helping to maintain data comparability across countries. In addition, ISSP data, questionnaires, and related documentation - such as data collection and questionnaire development materials - are freely available to researchers. This makes the ISSP an invaluable resource for those seeking high-quality global survey data that meet rigorous methodological standards.
This session aims to facilitate the exchange of knowledge about the ISSP among data providers and researchers who already use or plan to use ISSP data in their work. The focus will be on methodological challenges related to the quality of cross-national survey data, although papers addressing substantive issues related to one or more ISSP modules are also welcome.
Keywords: ISSP, cross-cultural research, comparability, data quality, measurement, survey mode, questionnaire development, cognitive testing
Dr Anja Eder (University of Graz) - Presenting Author
Mr Matthias Penker (University of Graz)
Professor Markus Hadler (University of Graz)
Declining response rates, rising costs, a shortage of interviewers, and other challenges have led many national and international surveys to transition from interviewer-administered modes to self-administered modes, particularly Computer-Assisted Web Interviewing (CAWI). This growing prevalence of self-administered surveys in international studies notwithstanding, challenges such as higher rates of item non-response compared to interviewer-administered methods persist. Nonetheless, previous research has demonstrated that lower rates of nonresponse are observed in the absence of interviewers, particularly in cases of sensitive inquiries.
Accompanying research on item non-response has primarily focused on national or small-scale cross-country data, leaving cross-national heterogeneity in item non-response rates and survey mode differences insufficiently explored—especially from a longitudinal perspective. We address these gaps by analyzing data from 22 countries participating in waves one (2011) and two (2021) of the International Social Survey Programme (ISSP) Health Module. In the initial wave of the survey, only four of the 22 countries administered a web-based survey. However, in the subsequent wave, this figure rose to ten countries conducting CAWI (several of them combined with mail mode). Using descriptive analyses and multilevel modeling, we evaluate item non-response rates over time, across countries, and between survey modes.
Dr Petra Raudenská (The Czech Academy of Sciences, Institute of Sociology, Prague) - Presenting Author
Background: Searching for health information online has become an integral aspect of personal health management and a proactive approach to well-being. Despite its popularity as a research topic in many disciplines, there remains a significant gap in examining the measurement tools used to study health information seeking and their comparability across different cultural and social contexts.
Objective: The aim of this study was to provide an overview of how online health information seeking is currently measured, to specify the instrument for measuring online health information seeking developed in the most recent health module of the ISSP, and to assess its cross-national measurement invariance across 19 countries.
Methods: This study used data from the health module of the ISSP, conducted in 2021-2023 in 19 countries (N = 27,276) in different regions. This study examines the cross-country measurement invariance of two constructs reflecting general health-related internet use and the perceived usefulness of searching for health information online. The latest Bayesian approximation approach was used.
Results: The general factor of health-related Internet use was found to be approximately metric and scalar non-invariant, with deviations in almost all countries. Conversely, the second factor reflecting the perceived usefulness of searching for health information online was observed to be approximately invariant across participating countries. The results highlight problems with frequently used single-item questions about the frequency of searching for health information online, which should not be used for cross-country comparisons in the International Social Survey Program. The study concludes by warning against double-barreled questions, which are problematic on both theoretical and methodological levels. The second factor, reflecting the perceived usefulness of searching for health information online, showed much better results, suggesting that its means and regression coefficients can be compared internationally.
Miss Nikoletta Sivenou (School of Political Sciences, Aristotle University of Thessaloniki) - Presenting Author
Professor Ioannis Andreadis (School of Political Sciences, Aristotle University of Thessaloniki)
The paper explores the evolution of social attitudes towards non-traditional family structures, focusing on children raised by single parents, same-sex male couples and children without marriage. Drawing on longitudinal data from the International Social Survey Programme (ISSP) spanning from 1988, it investigates how political ideologies, party preferences and elite discourse influence these attitudes across various national contexts. Specifically, the study addresses three research questions: 1. How have attitudes towards single parenthood, same-sex male parenting and children without marriage evolved over time? 2. What roles do political ideologies, such as conservatism and progressivism, and party affiliations play in shaping these attitudes? 3. How does congruence between elite political discourse and voter preferences impact societal attitudes, with Greece as a case study?
In particular, the paper employs multilevel modeling to examine trends across countries, analyzing the interplay between respondents' ideological self-placement, party choices and their attitudes towards non-traditional families. It also assesses how the political context and elite rhetoric shape public opinion, highlighting variations in these dynamics across different nations. The Greek case study provides a closer examination of the alignment between political elites and voter attitudes regarding non-traditional family structures.
The findings are expected to offer compelling evidence of how political ideologies and party preferences shape societal norms over time. By analyzing the influence of political rhetoric on attitudes towards single parenthood, same-sex parenting and children without marriage, the paper aims to deepen understanding of the mechanisms driving social change.
Finally, it contributes to broader discussions about the role of political elites in shaping social norms and fostering equality and social inclusion, as part of the "Data for Inclusive Societies" (DATIS) project.
Dr Marlène Sapin (FORS) - Presenting Author
Professor Christof Wolf (GESIS)
The literature has highlighted the association between different aspects of social networks and health. At the same time, research has underscored the impact of social structures and socioeconomic inequality on health. While various explanations have been proposed regarding the mechanisms linking social networks to health, these mechanisms still require further exploration within individual societies and across different ones. The way this relationship is embedded in broader social frameworks and influenced by national contexts demands a more thorough examination. A substantial body of work has examined the role of social capital as a resource within networks with health, but comparative research on the topic remains limited. This study focuses on the meso-level hierarchical context in which individuals navigate their daily lives, specifically the resources they access through their social connections within the status hierarchy. The research has two main objectives: first, to examine whether network diversity, extensiveness, upper-reachability, average reachability, and a global social capital index contribute to individual health protection, and second, to explore how this protective effect may differ across societies according to income inequality. Using data from the 2017 module of the International Social Survey Programme on Social Networks and Social Resources, which includes representative samples from 30 countries, we demonstrate that various network indices and relational resources are significantly linked to self-rated health. Additionally, we explore how economic inequality influences the relationship between individual social capital and health. Finally, we discuss the institutional and cultural factors that may explain why the connection between network measures and health varies across different nations.
Dr Markus Quandt (GESIS Leibniz Institute for the Social Sciences) - Presenting Author
There appears to be a widespread perception among social scientists that the ethnicity of their subjects is among the core 'background' variables for describing the social situation of individuals, with potential relevance in many if not most current research contexts. The ISSP has therefore decided to add a measure of subjective ethnic belonging to their list of "standard background variables" that are to be collected with every national sample, alongside variables such as education, age, occupational status etc. This has been implemented in ISSP 2011 for the first time.
While the societal relevance of the concept of ethnicity seems evident, with for example political debates around increasing migration movements also being led with reference to ethnic frames, or with ethnic origin being an established predictor of labour-market discrimination experiences, the concept in itself poses many obstacles for a unified implementation in a comparative survey program such as the ISSP.
ISSP is therefore currently conducting an evaluation exercise for its ETHNICITY background variables that is looking both at issues around the practical implementation as performed by the ISSP members and at the realization of any research benefits that were to be expected from the availability of those data. The presentation will present the concept pursued by the ISSP for measuring subjective ethnic belonging, and summarize and discuss the first results of the internal evaluation with the intent of garnering feedback from a wider community.
Miss Dörte Naber (Universidad de Granada)
Mr Ricardo Gonzalez (Universidad Adolfo Ibañez) - Presenting Author
This article addresses the challenges faced by General Social Surveys (GSS), especially in measuring household composition within the International Social Survey Programme (ISSP), as they transition from interviewer- to self-administered modes. We document high nonresponse rates of household composition questions, comparable to those for income questions, and moderate under-reporting of household members in several countries where these GSS-type surveys have been fielded in self-completion mode. Using the question-and-answer process model and Satisficing Theory, we design and assess new survey questions aimed at reducing both nonresponse and inconsistent reporting drawing on three distinct ISSP waves conducted in Germany (2019, 2021, and 2022/2023) in self-completion mode, each featuring different designs. The 2019 survey employed a traditional group design, requiring respondents to categorize household members into three age groups. In contrast, the 2021 and 2023 surveys used list designs, where respondents were asked to list the ages of all household members, excluding themselves in 2021 and including themselves in 2023. We find that list designs significantly reduce nonresponse rates and mitigate issues related to survey fatigue, which typically contributes to increased nonresponse and under-reporting, especially towards the end of surveys, where household composition questions are typically placed. The 2023 list design particularly demonstrates its effectiveness by lowering both over-reporting, prevalent in the 2021 design, and under-reporting, noticeable in the 2019 design. These findings are relevant for survey practitioners and researchers, demonstrating that thoughtful question design can substantially improve data quality in self-administered GSS-type surveys.
Professor Ioannis Andreadis (Aristotle University of Thessaloniki) - Presenting Author
This paper explores how to improve the quality of survey data, focusing on new methods developed by the DATIS research project for collecting data for the International Social Survey Programme (ISSP). Regarding data collection, the project involved using text messages (SMS) as the primary mode of contact, which proved effective due to the near-universal access to mobile phones in Greece. This push-to-web method allowed for efficient data collection, reducing costs compared to traditional postal mail methods. To ensure data quality, the paper details several indicators used to measure and enhance response quality, such as assessing the speed of responses and checking for satisficing behavior (e.g., straight-lining), that are implemented in the R package, SurveyDataQuality. Overall, the paper highlights the importance of a comprehensive approach to data quality in surveys, combining innovative data collection methods with rigorous quality checks to ensure reliable results.
Dr Pei-shan Liao (Academia Sinica) - Presenting Author
Cross-national survey data offer valuable opportunities for comparative studies in the social and behavioral sciences. However, it is crucial to determine whether differences in sample estimates are influenced by response bias. Response bias refers to a systematic tendency to answer questionnaire items based on factors unrelated to the item content. For example, response style manifests when individuals consistently exhibit a bias across different times and situations. Response styles are commonly observed in attitudinal scales, which can lead to measurement errors and affect the accuracy of estimates. Previous research has identified various types of response styles using cross-national or cross-cultural surveys. However, many of these studies relied on non-probability samples, which limits the validity of their findings.
This study investigates response styles in cross-national surveys, with a particular focus on acquiescence response style (ARS) in East Asian countries and extreme response style (ERS) in European countries. Using data from the ISSP 2018 Religion module, the study compares ARS and ERS across these regions. Standardized questionnaires in cross-national survey projects, such as the ISSP, play a crucial role in minimizing the influence of survey design on response styles. The findings aim to improve researchers' understanding of how response styles affect the outcomes of comparative studies using cross-national survey data.