All time references are in CEST
Using Central Management Strategies and Workflows in Large-Scale Surveys to Reduce Total Survey Error and Implement Methodological Changes |
|
| Session Organisers | Mr Niccolo Ghirelli (ESS HQ (City St George's, University of London)) Ms Victoria Salinero-Bevins (ESS HQ (City St George's, University of London)) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
In large-scale and cross-national social surveys, minimizing total survey error (TSE) is crucial for ensuring high-quality, comparable data. This session will focus on the management, monitoring, and research support techniques that can reduce TSE, particularly in the context of methodological changes. As surveys evolve to adapt to new technologies, data collection modes, and shifting respondent behaviours, maintaining standardization across diverse national or regional settings becomes increasingly challenging.
The session aims to bring together insights from different survey contexts to highlight how survey teams have successfully faced methodological and operational challenges while maintaining data quality and comparability. A key focus will be on how effective implementation support, centralized monitoring systems and quality control can ensure that methodological updates – such as changes in the mode of data collection – are applied uniformly across countries or regions.
Efficient communication workflows between central coordinators and national or regional teams are essential in this process, ensuring that all stakeholders understand and adhere to the new standards. This includes clear protocols for feedback loops, active monitoring, and ongoing training and support to address country-specific issues while upholding a unified methodological framework.
By sharing case studies and experiences, presentations could outline how methodological and practical challenges in implementing survey projects have been addressed through management strategies, communication workflows, and quality control techniques.
By collating experiences from various survey contexts, the session aims to provide participants with actionable insights on how to successfully manage methodological and operational changes, minimizing TSE and biases.
Keywords: Total survey error, survey project management, large-scale survey, data collection harmonisation, new methods implementation
Mrs Kim De Cuyper (Ipsos)
Mrs Sara Gysen (Ipsos) - Presenting Author
This presentation examines the centralized management strategies and workflows implemented in the European Survey of Enterprises on New and Emerging Risks (ESENER), coordinated by Ipsos European Public Affairs (EPA) across 30 European countries. Drawing from extensive field experience, the presentation will showcase a comprehensive framework for maintaining data quality and comparability across diverse national settings.
Key elements of our centralized approach include:
• Establishment of a highly specialized central team that is also responsible for selecting experienced field agencies.
• Highly centralized survey management strategy with the central, team overseeing all critical aspects such as sampling, scripting, translation, briefing, call design, field monitoring, and quality control, while remaining receptive to local input and regional nuances.
• Meticulous sample frame selection, sample monitoring and release during fieldwork, with additional screening and enrichment during fieldwork to ensure comprehensive representation.
• Multi-mode data collection strategy, incorporating telephone and online methods to maximize participation and address changing respondent preferences.
• In-depth briefing of local supervisors and project managers through a two-day in-person seminar and detailed materials.
• Robust communication systems, including daily detailed field reports, a secure central document exchange platform, and targeted personal contact with local teams.
• A three-step questionnaire refinement approach to ensure cross-country comparability: translatability assessment, cognitive testing, and piloting.
• Continuous fieldwork monitoring for progress, achieved sample composition, and data quality, utilizing various metrics and comparative analysis with previous survey rounds and across modes.
We will discuss how these strategies have helped maintain data quality and comparability while adapting to new technologies and shifting respondent behaviors. Our experience demonstrates the effectiveness of centralized management in standardizing methodological updates across diverse national settings, ultimately contributing to the reduction of Total Survey Error in large-scale, cross-national surveys. By sharing these strategies, the session aims to provide actionable insights for survey practitioners
Dr Olga Grünwald (GGP Central Coordination Team, NIDI) - Presenting Author
The Generations and Gender Programme (GGP) is an international survey infrastructure dedicated to collecting and disseminating cross-nationally comparable data on family and life-course dynamics. Over the past two decades, the GGP has transitioned from a predominantly decentralized operational model during the first round of data collection of the Generations and Gender Survey (GGS-I) to a more centralized structure in its second round of data collection (GGS-II). Since 2020, GGS-II has been implemented using mixed-mode data collection in over twenty countries.
This presentation will evaluate these five years of GGS-II data collection, highlighting lessons learned in balancing flexibility in fieldwork with the need for high-quality data and cross-national comparability. Recognizing that the needs of national teams vary significantly, our current practice revolves around tailoring support to meet these needs. This flexibility addresses specific challenges faced by national teams and contributes to reducing total survey error. For example, in some countries, additional data quality checks were introduced during fieldwork to monitor progress and ensure data accuracy. In others, the use of paradata has been explored as a tool to improve fieldwork monitoring by providing insights into respondent behavior. At the same time, we have begun developing a standardized reporting framework to streamline fieldwork evaluation and ensure consistency across countries.
By reflecting on these strategies and their outcomes, this presentation aims to provide valuable insights into managing mixed-mode data collection in large-scale, cross-national surveys. It emphasizes the importance of adapting to national contexts while maintaining methodological rigor to minimize total survey error and deliver high-quality, comparable data.