Panel attrition 1 |
|
| Chair | Mr Joaquin Prieto (Department of Social Policy, The London School of Economics ) |
Compared to cross-sectional studies, setting up and conducting panel surveys pose some additional challenges for the primary researcher: Apart from panel effects on response quality, panel attrition is the most crucial methodological problem. Research on panel maintaining strategies shows that efforts mostly aim to decrease panel attrition due to spatial mobility of respondents and panel attrition due to respondents’ refusal to participate as they are approached for the follow-up interview. However, some studies are required to query respondent’s panel consent in the preceding wave and are thus faced with another reason for panel attrition.
In this study, we investigate how post-survey panel maintenance strategies influence panel consent rates after the first wave and how these subsequently affect panel attrition in the follow-up wave. As panel attrition literature suggests that the mode of data collection affects attrition rates, we investigate if consent rates differ between survey modes by comparing a random sample with face-to-face interviews and a random sample with self-administered interviews. We, furthermore, study possible panel consent biases by respondents’ age, sex, migration background, and education. Beyond that, we investigate the effectiveness of two post-survey panel maintenance strategies (mail and telephone calls) aimed at increasing panel consent rates in self-administered interviews and additionally analyze if these panel maintaining procedures influence panel consent biases.
We use data from the first and second wave of the German multi-mode panel survey “Legitimation of Inequality Over the Life Span” (LINOS). The first wave (LINOS-1) was conducted during winter 2012/2013, the second wave (LINOS-2) is scheduled for January 2017.
Data of over 4,500 respondents reveal remarkable differences between survey modes: Panel consent rates were much higher (about 97 percent) in the face-to-face mode compared to self-administered interviews (46 percent). However, post-survey panel maintenance increased the consent rates of respondents in the self-administered survey mode from 46 to 75 percent. Further analyses revealed that the post-survey panel maintenance substantially decreased the consent bias regarding age, migration background, and education. As soon as the data for the second wave are available further analyses will reveal how the efforts to increase panel consent affect panel attrition in the follow-up wave.
The UK Household Longitudinal Study, also known as Understanding Society, had used face to face interviewing since it began in 2009. In 2016, wave 8 of the survey took a leap into mixed mode interviewing with 40% of the sample being asked to participate online initially, before being followed up face to face.
This paper will look at the overall effects of the move to web on response rate (and therefore attrition) as well as whether there are particular groups for whom the move to web has increased or decreased response, with a focus on groups that have tended to have higher levels of attrition on the face to face waves of the study. The paper will also discuss the particular strategies used by the study to boost response in a mixed mode context and the relative success of these.
By summer 2017 we will also have early response data from wave 9 of the study, so will have some initial findings on whether the mode of response at one wave appears to affect participation at the following wave.
The minimization of panel mortality between two waves is a challenge in every longitudinal study. This is particularly true when the period between the inter-rogation waves is relatively long. In the study "Panel on household finances (PHF)" the time period between to waves is three years. The interviews are conducted face-to-face.
In this study, a series of measures are implemented between the waves to minimize panel attrition. As in every longitudinal study, this has two main objectives:
1. To keep the participants interested in the study and thus to ensure the participation of all panel households and their members for the following waves.
2. To assure the accessibility of all panel households for the following waves by tracking the panel households.
In order to reach these targets, the panel households in the PHF are contacted twice a year between the survey waves. One of the measures is a short survey (PAPI / CAWI), which explicitly serves the objectives of panel stability rather than data collection. Such a short survey was carried out for the first time about two years after the first wave of the PHF.
Before conducting the first short survey, the following questions arose:
- Is an additional survey between two waves at all a suitable means to ensure panel stability?
- Does an additional survey lead to increased withdrawals from the panel?
- How to design a survey that achieves both: securing the willingness to participate and collecting information on the households.
In the meantime, two short surveys were conducted in the PHF which show that short surveys between two waves can be a suitable means for the maintenance of a panel survey. High response rates indicate the interest of the panel households in the survey without causing any higher withdrawal than other measures between the waves.
In this paper we analyze the nature and extent of attrition of the two most relevant longitudinal surveys in Chile: the Socioeconomic Household Panel Survey (known in Spanish as PCASEN) and the Social Protection Panel Survey (known in Spanish as EPS). The PCASEN, which involves personal interviews to all adult members of the sampled households, was conducted yearly between 2006 and 2009 (four waves). It is nationwide representative for urban and rural areas and used a sub-sample of 8,079 households composed by 30,104 individuals from the CASEN survey 2006. The rate of response between the first and second wave was 73% and for the subsequent waves the response rate was 89% and 90% respectively. As for the EPS, this was implemented the first time in 2002. It is representative of the individuals affiliated to the pension system. As it doesn’t consider the non-affiliated, the first wave is not nationwide representative. Given the importance of knowing who are the persons that remain marginalized from the pension system, in the year 2004 a sample of the non-affiliated to the pension system was incorporated, and at the same time the sample of affiliated was updated with the new affiliated. Thus, the Social Protection Survey 2004 is nationally representative of the population over 15 years old and the sample counts 19,822 individuals. The rate of response between 2004 and 2006 was 77%, between 2006 and 2009 was 73%.
Several studies have used the PCASEN to study poverty dynamics and the EPS to study labour market transitions and its relation with the pension system. However, none of them have considered the attrition bias problem, which might distort crucial outcomes. Through the use of standard methods, firstly we analyze the determinants of response and secondly we construct inverse probability weights. We used a simulation model to evaluate the effectiveness of weights and how these affect low-income dynamics and labour transition results. We conclude that the attrition bias overestimates the entries and exits for both key variables analyzed and hence, have significant effects on policy recommendations. This study represents the first effort to demonstrate the critical relevance to control bias due to non-random survey attrition in two of the most relevant longitudinal surveys of a Latin American country.