ESRA 2025 Preliminary Program
All time references are in CEST
The Implementation of PIAAC Cycle 2: Challenges and Lessons Learnt 2 |
| Session Organisers |
Ms Anouk Zabal (GESIS – Leibniz Institute for the Social Sciences)
Ms Silke Martin (GESIS – Leibniz Institute for the Social Sciences)
|
| Time | Friday 18 July, 11:00 - 12:30 |
| Room |
Ruppert D - 0.24 |
PIAAC, the Programme for the International Assessment of Adult Competencies, is a multi-cycle international survey that aims at measuring key skills in the adult populations of the participating countries. PIAAC Cycle 2 is designed to obtain international comparable data of the highest quality possible, and to be comparable to the first cycle of PIAAC. Thirty-one countries participated in the second cycle of PIAAC and collected data within an elaborate quality assurance and control framework, adhering to gold standards in survey methodology.
The second cycle of PIAAC introduced innovative elements in various areas, such as in the instrumentation (e.g., the inclusion of social-emotional skills in the background questionnaire or the introduction of the new assessment domain adaptative problem solving in the cognitive assessment), the interview mode of the cognitive assessment (tablet-based), or survey operations (e.g., introduction of a dashboard as a tool to monitor fieldwork, adaptive survey design). The PIAAC survey had many implementational challenges stemming from its unique design and specifications, with the additional challenge of finding a balance between innovation and trend. Implementing PIAAC within the national settings often required new national solutions to meet international standards. In addition, the COVID-19 pandemic impacted on the data collection of both the field study, carried out in 2021, and the main survey, carried out in 2022/2023. Many countries faced difficulties during fieldwork, such as declining response rates, and had to rise to the challenge of adapting to changing situations and constraints. This session will focus on methodological issues and aims to contribute to extending the survey-methodological body of knowledge and offer insights pertinent to other international surveys and large-scale assessments.
Keywords: PIAAC, international comparability, survey operations, data quality, nonresponse
Papers
Is a doorstep interview effective in addressing literacy-related nonresponse? Looking back on the implementation in PIAAC Germany
Mrs Silke Martin (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Mrs Anouk Zabal (GESIS - Leibniz Institute for the Social Sciences)
The Programme for the International Assessment of Adult Competencies, PIAAC, is a multi-cycle international comparative assessment of literacy, numeracy, and problem solving. This large-scale survey comprises an interviewer-administered background questionnaire and a self-administered assessment conducted in the official country language(s). PIAAC collects data from the 16- to 65-year-old population residing in the participating country, regardless of citizenship, status, or knowledge of the official country language(s). The aim is to obtain participation from as many selected target persons as possible in order to draw reliable conclusions. Target persons with language barriers are often unable to participate because their language skills are insufficient, and they may also be reluctant to interact with interviewers. The participation of these persons is crucial for the valid measurement of skills in the population. In PIAAC, the best option is to have an interpreter assist respondents with language barriers when completing the background questionnaire. If this is not possible, rather than obtaining a refusal, interviewers can attempt to conduct a doorstep interview. This interview consists of six questions that respondents can answer in their own language at the doorstep of their homes. It provides minimal key information on characteristics of these doorstep respondents (e.g., age, gender, employment status), and, importantly, sufficient information for the estimation of proficiencies. The doorstep interview was developed for PIAAC Cycle 2 and is an innovative approach to reduce literacy-related nonresponse. Despite the benefits of the doorstep interview, we encountered some challenges in implementing this non-standard instrument. The present contribution will illustrate and summarize experiences from implementing the doorstep interview in PIAAC Germany and reflect on issues of international comparability.
The Gift Card Experiment in Norway PIAAC Cycle 2
Mrs Elisabeth Falnes-Dalheim (Statistics Norway) - Presenting Author
The Gift Card Experiment
Statistics Norway has registry information containing the information of the highest, completed education of the citizens of Norway. In the database of Higher education in Norway there is data and information from the universities and all the other higher educational institutions in Norway. The Statistics Act for Statistics Norway gives Statistics Norway the right to use this database and therefore the educational level of almost all the respondents in the Norwegian PIAAC sample are known to Statistics Norway.
Norway, as many other countries, faced difficulties during fieldwork of PIAAC cycle 2. Such as declining response rates among the lower educated respondents. In addition, the data collection was too slow. To meet these challenges, an experiment was constructed.
The experiment aimed at speeding up the data collection and raising the response rate among the lower educated respondents ensuring high quality in the collected data, and better representation for all educational groups in Norway.
During fieldwork the response rates among the respondents with high (completed master’s degree) and very high education (Ph.D.) were higher than the response rate among the respondents with education beneath master (bachelor’s degree and beneath). In particularly, the goal was to achieve higher response rate for this group of respondents and fight biases in the PIAAC data. The experiment shows how raising the amount of the gift card impacted the willingness to take part in the survey. Prior to the experiment every respondent participating was offered a gift card of 500 Norwegian kroner (NOK). In the experiment some of the refusals and non-contacts were offered higher amounts (800 NOK or 1100 NOK) and the control group still got the same amount as the higher educated respondents.
The experiment produced many interesting findings and useful knowledge for other data collection teams.
