Surveying Children and Young People 3 |
|
| Chair | Ms Kate Smith (Centre for Longitudinal Studies ) |
| Coordinator 1 | Dr Lisa Calderwood (Centre for Longitudinal Studies) |
The longitudinal study “Health Behavior and injuries in the school career” surveys roughly 11,000 pupils in about 150 schools in Germany. Accidents in the school environment and the health behavior of the respondents are the key variables, surveyed on an annual basis (from 5th to 10th class) using an offline survey app for gathering data. Currently, the project is in its third wave.
In our paper, we want to present some recommendations on how to survey young people at schools. At first, we want to discuss what is crucial in convincing schools to participate in longitudinal surveys constantly. In wave two, the project suffered from an only moderate dropout rate of 12 percent at the school level which we attribute to our efforts to reduce panel attrition. Moreover, we managed to re-identify more than 90 percent of all participating pupils in the first wave on the basis of a self-generated code. Therefore, our paper aims at presenting our self-generated code and discussing opportunities for re-identifying participants using record-linkage-procedures. Furthermore, we want to present our data-generating procedure using an offline survey app and tablet devices and discuss whether this is suitable for that age group.
Finally, we want to briefly address other issues concerning surveying pupils in schools, e. g. data protection issues and procedures that might help to organize large-scale surveys at schools.
The aim of this study is to understand the impact of student proxy reporting on the quality of socio-economic measures in a survey of schools in Qatar. Obtaining proxy reports from children is a frequently used practice in education studies to obtain information on household characteristics without the added burden and cost of fielding a separate survey to parents. The quality of the proxy reports has important implications for degree to which researchers can conclude that characteristics of the household shape important outcome variables of interest, such as aspirations for post-secondary education, enrollment and interest in advanced secondary coursework, and student achievement scores. Parents who have higher levels of education and have greater financial wherewithal have been found to provide additional academic resources for their children and foster an environment at home leading to higher probability of post-secondary educational attainment. The extent to which students’ reports align with their parents’ on socio-economic status then can affect our understanding of these outcomes.
In this paper, we evaluate two aspects of data quality using the 2015 Qatar Education Study: 1) measurement error in student proxy reports, and 2) nonresponse error as a result of missing parent surveys. The QES 2015 is a comprehensive school-level study in which all stakeholders (e.g., students, parents, teachers, and administrators) are surveyed. The extent to which measurement error is introduced using proxy reports from students has been long studied in the field of education and survey methodology, and more recent work has turned to examining how parental nonresponse biases estimates of socio-economic status. To our knowledge, our study is the first to evaluate measurement and nonresponse error in an education study fielded in the Middle East and North Africa. Our baseline models include several measures that have been shown to influence quality in previous studies, such as the child’s age and gender, but we also account for the extent to which the broader societal context influences student’s reporting of this information. Qatar is a highly segmented and heterogeneous society in which children of immigrants make up a majority of the school age population, and our models build on previous research by accounting for the child’s nationality and length of time in country. As such, these results will have important implications for the design of education research in not only Qatar but also the broader Middle East and North Africa region and societies in which immigrants comprise a significant proportion of the population.
To analyze nonresponse processes occurring within the cohort sample of grade 5 students of the National Educational Panel Study (NEPS), we present a bivariate probit model framework addressing explicitly the longitudinal, hierarchical and dyadic multi-actor character of the surveyed data implied by the design of the NEPS. The NEPS cohort of grade 5 students, also known as starting cohort 3 (SC3), is followed up throughout lower secondary education starting in school year 2010/2011. To allow for portraying their educational development, the NEPS surveys and tests students every year. Next to surveying students, one of the student's parents is asked to provide background information on the student in a computer assisted telephone interview. Such a dyadic multi-actor approach allows for validating information given by the student and for collecting further information on the family background. A dyadic multi-actor approach like this leads to four wave-specific nonresponse patterns, that is, both, the student and a corresponding parent participate, both do not participate, or only the student or a parent participates. We deal with the two distinct but possibly interdependent participation decisions of students and their parents by allowing for correlation between both decision processes. While the hierarchical structure is incorporated in terms of correlated student and parent specific random effects at the school level, the time dependence of participation decisions is modelled by consideration of the lagged observed decisions. Using simulated maximum likelihood estimation, the empirical results point at significance of explaining factors like school type and family background. Specification tests highlight the importance of considering correlation as well as clustering structures when modeling participation processes.
BACKGROUND. Parent-reported well-being for their children is often used where children are unable to respond for themselves. The inconsistent relationship between child- and parent-reports were reported. Understanding influential factors to agreement levels can help better interpret reported well-being by children and their parents. Using cognition as a demonstration example, this study aimed to evaluate effects of life-threatening condition (i.e., cancer versus the US general population) and type of rating scale (i.e., frequency versus intensity) on the level of concordance between child- and parent-reported cognition in children.
METHODS. Data from 1,409 children (mean age=12 yrs; 57% male; 83% White) drawn from the US general population and 515 children with cancer (53% brain tumor; mean age=14 yrs; 57% male; 70% White) and their parents were analyzed. For general population sample, 23% had a neurologic condition, 26% had attentional deficit/hyperactivity disorder and 16% had repeated a grade. For cancer sample, 34% received radiation therapy, 72% chemotherapy, 71% surgery and mean years since last treatment = 3.3. All completed two versions in a random order of a validated pediatric perceived cognitive function short-form (pedsPCF-SF). Both had the same 13 item stems but with different rating scales: 1) “frequency” (from “none of the time” to “all of the time” and 2) “intensity” (from “not at all” to “very much”). Weighted Kappa was used to evaluate agreement at individual item level between parent- and child-reported pedsPCF-SF. Intra-class correlation (ICC) and Pearson correlation were used to evaluate agreement at the scale level.
RESULTS. At item level, for cancer sample, weighted kappas between children and parents were 0.27-0.39 for both intensity and frequency. For general population, weighted kappas were 0.53-0.67 and 0.64-0.70 for intensity and frequency, respectively. At the scale level, higher agreement was found in general population sample (ICC=0.77 and 0.81 for intensity and frequency, respectively) than cancer sample (ICC=0.46 and 0.38 for intensity and frequency, respectively). Correlations between parent- and child-reported pedsPCF-SF were consistently higher in general population sample than cancer sample. For general population sample, parent-frequency had higher correlations with child-frequency and child-intensity (r=0.86) compared to parent-intensity (r=0.86 and 0.84 with child-intensity and child-frequency version, respectively). For cancer sample, child-intensity had higher correlations to parent-intensity and parent-frequency (r=0.52 and 0.50, respectively) when compared to correlations between child-frequency versus parent-intensity and parent-frequency (r=0.46). Yet the difference within sample groups (i.e., general population and cancer) were minimal.
CONCLUSIONS. This study showed higher agreement between children and their parents from the general population compared to those with cancer. These two samples also had different levels of agreement between rating scales. Future studies should be conducted to evaluate other potentially influential factors to agreement between parent- and child-reports.