Survey Data Harmonisation: Potentials and Challenges 2 |
|
| Session Organisers |
Dr Ilona Wysmulek (IFiS PAN) Dr Irina Tomescu-Dubrow (IFiS PAN and CONSIRT) |
| Time | Friday 19th July, 11:00 - 12:30 |
| Room | D21 |
Survey data harmonization - its theory and methodology - is growing into a new scientific field that pushes forward the methods of survey data analysis while emphasizing the continuous relevance of surveys for understanding society. Depending on whether researchers intend to design a study to collect comparable data, or use existing data not designed a priori as comparative, the literature distinguishes between input and ex-ante output harmonization, and ex-post output, or, just ex-post, harmonization. Applied ex-ante, harmonization facilitates comparability of survey data collected in multinational, multiregional and multicultural contexts (3MC, www.csdiworkshop.org). Applied ex-post, harmonization enhances the effective use of extant surveys and represents a way to overcome limited time and space coverage inherent in any single comparative project. In both its forms, ex-ante and ex-post, harmonization is a complex, labor-intensive and multistage process, which poses numerous challenges at different stages of the survey lifecycle.
This session welcomes papers on both opportunities and difficulties inherent in ex-ante and ex-post survey data harmonization. We invite theoretical and empirical contributions that deal, among others, with (a) transparency of harmonization procedures, (b) variability in source data quality, (c) minimizing information loss in harmonization, (d) measurement equivalence, and (e) substantive analyses on survey data harmonized ex-post.
Keywords: Data harmonization, transparency of harmonization process, comparability of survey data, data quality
Dr Dara O'Neill (UCL) - Presenting Author
Surveys often differ in the measurement tools and strategies they adopt, impacting upon the longitudinal and cross-study comparability of the information collected. Such dissimilarities may reflect pertinent differences in research aims and cross-cultural considerations, but they can also arise as a result of idiosyncrasies in assessment strategy, interdisciplinary variations, or changes in theory and measurement practice over time. Endeavours to make such data more equivalent and amenable to joint analysis can open up new research opportunities, as well as offering learning opportunities that facilitate more prospectively harmonised data collection moving forward.
CLOSER, a consortium of UK longitudinal studies, coordinates a number of harmonisation work packages that has to date covered a wide range of topic areas, including measures of socio-economic status, body size, and childhood environment. Through this work, we have been able to observe and document the diverse obstacles and opportunities such efforts offer. This talk will address issues around data management, including the importance of documentation and its role in the subsequent utility of harmonised data outputs. Particular focus will be given to considerations around longitudinal data harmonisation, as well as the relevance of interdisciplinary differences. Finally, the importance of engagement and the challenges of effective dissemination, both of harmonised data outputs and the learning obtained through harmonisation exercises, will be discussed.
Dr Sarah Martin (Scottish Government) - Presenting Author
Mr Jamie Robertson (Scottish Government)
Mr Michael Davidson (Scottish Government)
Mr Alex Stannard (Scottish Government)
Ms Sara Grainger (Scottish Government)
Mrs Julie Wilson (Scottish Government)
In a large-scale transformation project, the four face-to-face population surveys in Scotland were recently methodologically harmonized and pooled, releasing the unprecedented potential of the combined sample size and resulting survey power. The harmonized subset of questions is drawn from effectively independent surveys and is now published as the Scottish Surveys Core Questions, which for the first time enables the statistical reporting of outcomes for minorities such as specific ethnic, religious and non-heterosexual groups, and also spatial data for small geographies within Scotland.
We present the full process of challenges and compromises to date:
- the harmonization of questions and attempts to harmonise ex-post,
- the coordination of the sampling strategy of three quite differently-conceived population surveys,
- the development of a multi-stage pooled weighting methodology,
- the study/order effects on estimate variability between source surveys.
We also present the ongoing results of this unique project, which span multivariate analyses of multi-year datasets and a number of official statistics publications and topic reports, including the first official statistics on LGB groups in Scotland.
www.gov.scot/sscq
Professor Ferruccio Biolcati Rinaldi (University of Milan) - Presenting Author
Professor Marco Maraffi (University of Milan)
Dr Francesco Molteni (University of Milan)
Professor Cristiano Vezzoni (University of Milan)
Despite the long-lasting interest on religious change, debates on the topic have been heated and are far from being settled. For what concern Europe, all the main theories about religious change hypothesize a decrease of religious practice across the countries. One of the main tasks for scholar dealing with this topic is to identify if there is a common pattern of religious decline across European countries. This task is mainly tackled by identifying clusters of countries sharing the same trend and by comparing them. From the methodological point of view, the study of these long-term secularization processes currently under way in Europe as well as their clustering requires reliable data covering the wider possible observation window. To do so, it is possible to count on a cumulating strategy of high-quality comparative studies. CARPE (Church Attendance and Religious change Pooled European dataset) is a cumulative dataset of the main international surveys containing items concerning religiosity (Eurobarometer, ESS, EVS, ISSP, WVS). The dataset covers 46 European countries and the period 1970-2016. The aim of this presentation is to introduce a way of clustering European countries according to church attendance by means of a group-based trajectory model. This modelling will be performed on the complete CARPE dataset. While doing so, we will also present the harmonization procedure we have adopted, the strategy we used to combine the datasets and the reliability tests we have performed.
Dr Markus Quandt (GESIS Leibniz Institute for the Social Sciences) - Presenting Author
National Identity is often distinguished into civic and ethnic types, sometimes a cultural type is added. The historical origins of this classification were in institutional analysis at the level of whole states and were designed to assign those states into distinct regime types. However, with the advent of comparative survey data, a number of researchers have sought to confirm this type assignment by investigating what the dominant types in the individual level attitudes of the respective populations are. Since this has first been attempted, it is well established that most states are characterized by a mix of types at the individual level, and often no clear assignment of types at the state level is possible.
Based on work conducted in the project ONBound (funded by the German national science foundation), which seeks to build a large database of survey data on topics related to national identity, ethnicity, and religion, we will revisit the question of classifying countries by their populations' distributions of national identity types. In contrast to most existing work, we will attempt to combine data from different survey projects (for example ISSP, World Values Survey, or Pew), which employ somewhat different versions of a question battery on criteria of belonging to a nation. We will check how much these lead to similar results for the same countries, such that a macro level dataset for these countries can be constructed. According to the conceptual definition of the ethnic/civic types, we will use a classification approach instead of a factor analytical approach to measure the construct of interest and estimate the state-level distributions of the attitude types.