Methodological Challenges in Longitudinal Surveys 1 |
|
| Session Organisers |
Ms Sara Möser (University of Bern, Department of Sociology of Education) Ms Almuth Lietz (DZHW) |
| Time | Tuesday 16th July, 16:00 - 17:00 |
| Room | D19 |
This session includes papers that address the challenges of longitudinal survey research, in particular, such as the problems of attrition and panel conditioning.
Keywords: panel surveys, nonresponse, attrition, incentives
Ms Almuth Lietz (DZHW) - Presenting Author
Dr Jens Ambrasat (DZHW)
Mr Uwe Ruß (DZHW)
To improve data quality of panel surveys – in the sense of higher response rates and higher willingness of participation in later waves – survey designers frequently focus on questionnaire length and incentives. Empirical studies have shown that both, the response rate and the willingness to participate in later waves can be improved by monetary incentives. Likewise, studies focusing on questionnaire length have shown that announcements of a shorter questionnaire increase response rates, but have no effect on willingness to participate in panel surveys. Such results are expected to be very sensitive to the precise population surveyed. For highly qualified people it is often doubted, whether small monetary incentives can have the same effect as for the broader population. Similarly, it can be expected that the tolerance of long questionnaires is higher if the announced topic is close to interests and identities of the participants.
We examine these issues on the very specialized population of doctoral candidates. We implemented an experimental design within a web survey, where the randomized controlled experiment combines both treatments - announcement of an incentive within email invitation in form of a lottery and announcement of questionnaire length. 10,458 doctoral students of 26 German universities were invited to the web survey in May 2018.
Results confirm causal evidence of the lottery and the announced questionnaire length on the response rate. In addition, there is a strong positive influence of the lottery on the willingness to participate again in panel surveys. The effect of the announced questionnaire length on the willingness to participate again is not confirmed.
Professor Rolf Becker (University of Bern, Department of Sociology of Education)
Ms Sara Möser (University of Bern, Department of Sociology of Education) - Presenting Author
Dr David Glauser (University of Bern, Department of Sociology of Education)
In this study, we evaluate short- and long-term effects of three different prepaid incentives: a ballpoint-pen (gift worth about 2 CHF), a voucher (cash card worth 10 CHF) and cash (10 CHF banknote) on young panellists’ cooperation and response rate in three waves of a mature panel study with sequential multi-mode design (DAB panel study). The survey experiment involved an alternative study design in order to analyse the effect of different types of prepaid incentives, taking selective attrition into account as well as considering problems related to causal inference (and the stable unit treatment value assumption behind such problems). The subjects were students from randomly selected school classes who had finished their compulsory school.
The findings are clear: even in a mature panel, there are positive effects of incentives on response rates that vary across the type of gift given. Unconditional prepaid money (“cash in the hand”) provides the strongest direct, positive effect on the overall response rate, as well as on the latency until response after first contact. The other incentives did not work as efficiently as cash. It seems to be that the subjectively evaluated value of an incentive accounts for the anticipated effect on survey participation, since the effect of a 10 CHF banknote was significantly stronger than the effect for a cash card (voucher) with the same objective value. Additionally, cash is most likely to minimise social selectivity in response, though in spite of the monetary incentives there is a social bias in response caused by the individuals’ education and achievement. Finally, cash provides the potential to convert refusals in previous waves into cooperation.
Mrs Nerdit Stein-Kapach (ICBS - Israel Central Bureau of Statistics ) - Presenting Author
The Israeli Longitudinal Survey is conducted annually by the Central Bureau of Statistics. The survey tracks a fixed sample of households and individuals.
So far six waves have been interviewed. When examining the extent of enumeration in the first four waves, a significant decrease in the number of households and individuals who responded to all the waves is apparent: Only 46% from individuals in wave 1 were responded in all 4 waves (from wave 3 to 4: decrease of 10%). Also, refusal percentage increase from 16% in wave 1 to 27% in wave 4.
After four waves, the response rates had declined significantly and the ability to track was impaired. The respondent population decreased compared with the original sample, which raises a difficulty about the degree of representation of respondents in the survey.
Therefore, In wave 5 (2017), a gift was given to each household who answered the interview.
Results Wave 5: The increase in the percentage of refusal has been halted (24%), and the decline in the number of individuals responding to all waves has moderated: 40% of individuals in the first wave, responded all 5 waves - only 6% less than the previous wave.
As a result of the refusal-percentages of wave 5, we decided to give to every person aged 18+ a gift at wave 6 in order to increase more the responded- percentages.
The paper will compare the characteristics of the population that refused to respond in previous waves to those who refused and those who answered in wave 5 and 6. The paper will examine whether there is a connection between the characteristics of the household and the decision of whether to refuse to respond or participate in the survey.