All time references are in CEST
Digital technologies for the data lifecycle of large-scale surveys 1 |
|
| Session Organisers | Dr Diana Zavala-Rojas (Universitat Pompeu Fabra, European Social Survey ERIC, RECSM ) Mr Niccolò Ghirelli (European Social Survey ERIC HQ, City, University of London ) |
| Time | Thursday 20 July, 16:00 - 17:30 |
| Room | U6-03 |
At the different stages of the survey data lifecycle, process-generated or user-generated content, meta- and para-data play an increasingly important role. As large-scale surveys rely more on the implementation of highly standardised processes, project management and data lifecycles utilise more digital tools to ensure both data quality and maintainability
Research infrastructures, and survey data providers are developing new digital technologies to enhance greater consistency, and improve communication between the different stakeholders involved.
This session aims to bring together those working with data dashboards and survey tools to share their experiences. We welcome presentations on tools used during the survey data lifecycle (e.g., questionnaire design, sampling, translation, data collection and data processing) from different stakeholders (fieldwork agencies, research infrastructures, academic institutes, etc.) and/or studies (national and cross-national, cross-sectional, and longitudinal, conducted in any mode).
Presentations could outline how methodological and practical issues have been addressed and show how digital technologies have an impact on survey implementation and appraisal of data quality. Presentations may also provide systematic reviews of performance indicators, monitoring dashboards, and studies testing tools.
Keywords: survey tools, survey project management, metadata, total survey error, digital technologies
Miss Sophie Hensgen (Institute of Employment Research) - Presenting Author
Miss Marieke Volkert (Institute of Employment Research)
Dr Joseph Sakshaug (Institute of Employment Research)
Paradata have become increasingly popular in recent years as a tool for assessing interviewer behavior, controlling questionnaire functionality, and monitoring the field phase. Most of these monitoring and quality control checks are under the control of the contracted survey institute and outside the control of the researchers or study sponsors. To provide more transparency during the field phase and to allow researchers and sponsors to monitor the work of the survey institute, as well as to perform their own quality checks and play a more active role in the data collection process, the use of researcher-oriented dashboards is a viable option. We have created a dashboard for the IAB Establishment Panel using Shiny Dashboards in the R software. Within this dashboard, it is possible to monitor the current field phase and use a number of different indicators to monitor interviewer performance to identify potential problems, such as non-functioning questions in the questionnaire or suspiciously short interview/screen times. In our presentation, we will present a selection of indicators and charts used to analyze the field phase, case studies that show where these indicators identified potential problems with data collection, and other useful aspects of Shiny Dashboards, as well as lessons learned in developing the dashboard.
Mr Maurice Martens (centerdata) - Presenting Author
The Data CTRL suite is a set of tools that support the Survey life cycle for large international Survey studies. In round 11 of the European Social Survey a self-completion survey platform is developed to support a number of participating countries with testing their delivery of the ESS through self-completion. To facilitate this, Centerdata extends the existing functionalities already present in the Data CTRL suite to support multiple modes, thus providing a centralised workflow to develop, implement, field and monitor this endeavor. Self completion presents several new challenges, how to register, track and monitor the physical paper questionnaire. How to join the data for these modes and how to visualise these questions in self completion mode.
This presentation will explain the challenges and solution presented per step in the development process and discuss the implemented adaptations with focus on:
- web and paper & pencil questionnaire development and testing; departing from the face to face questionnaire already implemented in the CTRL suite, an online version will be implemented for self completion
- localisation of paper & pencil placeholder version in InDesign, based on a source version developed in InDesign, a placeholder version will be generated, that will be linked to the translation platform TranslationCTRL. The texts will then be dynamically loaded in
- extending the translation process in TranslationCTRL to accommodate multiple modes; a new interface on TranslationCTRL will be defined that can display various mode specific translations of similar question items, which helps with guarantying consistency of items over modes
- the choice between tailoring SurveyCTRL or extending SampleCTRL to accommodate this process; focusing on the SurveyCTRL system, forces a web first mind set, while focussing on SampleCTRL departs from a sample perspective
- adjusting data delivery for multiple modes
Mr Iggy van der Wielen (Centerdata) - Presenting Author
Mr Nick Wondergem (Centerdata)
Managing the survey lifecycle of large international surveys has been and will continue to be a complicated puzzle. It is a major challenge to support many countries and organizations working together towards the common goal of fielding questionnaires and collecting data.
Centerdata is experienced in supporting the full survey life cycle for many international surveys. To streamline the survey processes we developed the DataCTRL suite, a collection of tools that support various parts of the survey lifecycle from a centralized approach.
In this presentation we will take a look at some of the tools the Survey of Health, Aging and Retirement (SHARE) and the European Social Survey (ESS) use during their data collection. These tools are CaseCTRL, SampleCTRL and FieldworkCTRL.
CaseCTRL and SampleCTRL focus on collection of interviews and managing fieldwork, while FieldworkCTRL is used to monitor all fieldwork processes from an aggregated perspective.
We will showcase how the CTRL suite has been integrated in the ESS to monitor fieldwork and how it preemptively detects issues during fieldwork and improves data quality overall.