The agony of attrition - challenges in longitudinal studies 2 |
|
| Chair | Ms Joanne Corey (Australian Bureau of Statistics ) |
| Coordinator 1 | Dr Jutta von Maurice (Leibniz Institute for Educational Trajectories) |
The Growing Up in New Zealand study recruited 6,853 children from before birth in 2009 and 2010. Mothers were recruited in pregnancy and fathers were invited to participate from the outset. The child cohort has been demonstrated to be broadly generalizable to all current New Zealand births and importantly represents the ethnic and socioeconomic diversity of the new generation of "kiwis" growing up in New Zealand in the 21st century. In particular 1 in 4 of the children identify as Maori, 1 in 5 as Pacific and 1 in 6 as Asian. Almost half of the children identify with multiple ethnic groups and ethnic identity is measured over time together with multiple domains of influence on child wellbeing and development. The study was explicitly designed to provide population relevant evidence to inform cross-sectorial policy so as to find ways to improve the health and wellbeing status of all New Zealand children and reduce the inequalities seen in child and adult outcomes across sectors by ethnicity and socioeconomic status.
Face to face data collection has occurred in pregnancy with both parents, and when the children were 9 months, 2 years and 54 months old. Currently planning is underway for the 96 month face to face data collection in the field. In between these major data collections multiple strategies have been utilised to stay in contact with the families and children, to track and trace the families, and continue to engage them in longitudinal data gathering. Traditional methods of CATI and CAPI have been augmented by electronic and text contact as well as with incentives, competitions and the use of social media and linkage to routine datasets.
To date retention rates have remained very high with only 3% of the baseline cohort opting out of the study completely in the first 5 years of the study, and with over 90% compete data collection at the last 54 month data collection point (7% skips). This high retention rate has occurred in the face of great residential mobility of the families, 50% of whom have moved between each data collection wave.
This presentation will describe the various methods used to maintain this diverse cohort and to minimise attrition bias over the important early years of child development. We will examine the pros and cons of strategies applied and describe what has worked well and what has worked less well in the NZ context. We will consider the relevance of these findings in an international context by comparing retention strategies and retention rates in the NZ study with those in comparable international "growing up" cohort studies, with whom we collaborate closely. We will provide cost-benefit analyses to support the approaches utilised in order to deliver robust relevant population evidence that can inform strategies to improve child and family wellbeing and maximise the value of the longitudinal information collected directly from all the children and their families.
Buying a home and getting a mortgage are often considered to be the largest financial decisions that someone can make. Nevertheless, almost half of US homebuyers seek only one mortgage offer before selecting a mortgage (CFPB, 2016), most likely resulting unnecessary expenses. To study the potential benefits of comparison shopping in the mortgage market, we designed a longitudinal online experiment with prospective US homebuyers. Consumers were randomly assigned to receive shopping advice using 1) messaging, 2) online resources, or 3) neither (control). This paper explores attrition from that experiment.
In order to identify prospective homebuyers, we emailed 5.2 million individuals who had accounts on a large real estate website. Those who opened the email were asked whether they would be interested in participating in a research study on homebuying consisting of a baseline survey and up to six check-in surveys. All surveys were conducted online with invitations sent by email. We offered $5 for completing the baseline survey and an additional $20 for completing the full study. Over three waves of recruitment, approximately 71,600 people opened the email, 19,500 were eligible, and 14,000 finished the baseline survey.
The baseline survey collected demographic and psychological characteristics, expectations regarding home and mortgage outcomes, and other variables. The check-in surveys were delivered every two weeks for three months unless participants missed two surveys or bought a home before the end of the study. These surveys focused on recent home and mortgage search activities.
In an attempt to reduce attrition, we also randomly assigned participants to an experiment inspired by work on “implementation intentions,” which shows that people who make plans are more likely follow through on behaviors (Gollwitzer, 1999). This study followed a 4 (weekday) x 2 (scheduling: yes/no) design. Specifically, unbeknownst to them, all participants were assigned to a default survey invitation day between Monday and Thursday. Those in the scheduling group could pick a preferred day (and therefore override the default), while those in the non-scheduling group did not have that option.
Overall study retention was 60% (n = 8463), which includes 1145 participants who finished the study by buying a home and 7318 who finished the study by answering at least five of six check-in surveys. The majority (about 80%) of participants who dropped out skipped consecutive surveys rather than returning to the study after missing one. We analyze study attrition using demographic, psychological, and study characteristics. Using logistic regression models, we find lower attrition rates for participants with certain 1) demographics: married, white, younger, more educated, and higher self-reported credit score; 2) psychological characteristics: higher levels of patience, subjective numeracy, and financial literacy; and 3) study variables: those in the control group (versus messaging group), those who chose to enroll in text message reminders about survey completion, and those who answered surveys on a computer (versus mobile device). We find no benefit in retention from scheduled survey invitations once controlling for day-of-week effects.
Since its inception in 2005 the Growing Up in Scotland study (GUS) has tracked the lives of more than 10,000 children and families living in Scotland. Currently, GUS follows two cohorts of children: c.3,500 children born in 2004/05 and c.6,000 children born in 2010/11.
After the first six sweeps, face-to-face fieldwork on GUS changed to be undertaken on a biennial rather than annual basis. This change prompted considerations about potential cost effective means of collecting data and keeping in touch with respondents, and short web-CATI surveys were introduced in years where no face-to-face interviewing took place. To date, a web-CATI survey has been undertaken with both GUS cohorts and both cohorts have also subsequently taken part in face-to-face interviews.
The use of web-CATI surveys in the context of a long-running longitudinal study is by no means a tried and tested approach. This presentation will explore how the use of web-CATI surveys has worked in the context of GUS. Specifically, it will examine how web-CATI response compares with face-to-face response and whether web-CATI participation appears to be associated with response at subsequent face-to-face fieldwork (and, if so, what might drive this relationship). Finally, the presentation will consider the usefulness of web-CATI surveys in the context of longitudinal cohort studies – both as a means of data collection in its own right and as a way of keeping in touch with respondents between face-to-face interviews.
Growing Up in Scotland (GUS) is a longitudinal birth cohort study funded by the Scottish Government and undertaken by ScotCen Social Research. Study website: www.growingupinscotland.org.uk