Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Measurement in panel surveys: methodological issues 1 |
|
| Convenor | Ms Nicole Watson (University of Melbourne) |
| Coordinator 1 | Dr Noah Uhrig (University of Essex) |
All surveys are affected by measurement error to some degree. These errors may occur due to the interviewer, the respondent, the questions asked, the interview situation, data processing and other survey processes. Understanding measurement error is particularly important for panel surveys where the focus is on measuring change over time. Measurement error in this context can lead to a serious overstatement of change. Further, recall effects of events between two interviews may lead to serious understatements of change. Nevertheless, assessing the extent of measurement error is not straightforward and may involve unit record level comparison to external data sources, multiple measures within the same survey, multiple measures of the same individuals over time, or comparisons across similar cohorts who have had different survey experiences.
Session Details
This session seeks papers on the nature, causes and consequences of measurement error in panel data and methods to address it in either data collection or data analysis. This might include (but is not limited to):
- Assessments of the nature and causes of measurement error
- Evaluations of survey design features to minimise measurement error (such as dependent interviewing)
- Examinations of the consequences of measurement error
- Methods to address measurement error in analysis.
When people are asked to report their monthly income they tend to round their income to the nearest fifty, hundred or thousand, or they even completely misreport the value of income. The same problem applies when people are asked to recall durations such as unemployment spells. People tend to round off to the nearest half year or year. Misreporting data this way causes abnormal concentrations of reported values at certain "heaping points". Using such data to compute mean characteristics does usually not cause any bias. However, the situation differs if other distribution characteristics like sample quantiles or proportions should be computed. In order to allow to adequately modeling heaped data, we introduce a general method that explicitly specifies the heaping mechanisms present in a data set. We suggest to model heaping mechanisms and the true underlying model in combination. This way we are able to simultaneously estimate the parameters of the true distribution and to determine the heaping pattern present in the data. We validate our novel approach using simulation studies. To illustrate the capacity of the approach we conduct a case study using income data from the adult cohort of the German National Educational Panel Study.
The aim of life course surveys is the collection of live course data as completely and consistently as possible. One of the major challenges when updating life-course information in an ongoing panel survey is to prevent so-called 'seam effects'. Dependent interviewing techniques are the most effective means for avoiding this problem. Whereas by reactive dependent interviewing information from the prior interview is offered only in reaction to certain responses, proactive dependent interviewing uses the information from the previous interview (the preload) to stimulate the memory as part of the questioning process (Brown et al, 1998). Proactive depending interviewing is widely used in panel surveys because of its potential to lower respondent burden, to increase efficiency, and to reduce seam effects (Jäckle/Lynn 2004).
However, preloads will not in all cases provide a proper memory anchor. So what is really of interest here is what it is that turns a preload into a good memory anchor? We will answer this question in our presentation in three steps. First, we will collect theoretical arguments what features of preloaded data will improve its potential to serve as such an anchor. Second we will describe how we implemented preloads in the partial study "Adult Education and Lifelong Learning" conducted by the German National Educational Panel Study (NEPS). Thirdly, by analyzing disagreement with different types of preloads we will identify determinants of preloads which them qualifies to become a good memory anchor in life course panel surveys.
The collection of 'event histories' asking respondents to report changes in their lives which occur between interviews is of central importance to longitudinal studies. However, it is recognised that the accuracy with which respondents can provide this information is affected by recall error.
Previous evidence suggests the use of Event History Calendars can increase the quality of reporting of events, relative to standard questionnaire approaches, particularly when reference periods are relatively long. Event History Calendars can encourage parallel cueing, whereby the recollection of events in one domain of life can help to trigger the recollection of events in other domains.
This paper describes a visual calendar which was developed for the eighth follow-up the 1970 British Cohort Study, a multi-disciplinary study following those born in Britain in one week in 1970 which took place in 2012 when over 9,000 respondents were interviewed at age 42. The calendar was designed to aid collection of changes in relationships, housing situation and economic activity, which in prior follow-ups (which occur every four years), were collected using a standard questionnaire approach. The calendar seeks to aid recall by simultaneously displaying transitions across these domains.
By making comparisons with the data from prior follow-ups the paper evaluates the extent to which the new visual calendar has affected the quality of the data collected. We anticipate that missing data, seam effects and the reporting of illogical dates will be reduced and that consistency between transitions across domains will be increased.
To study urban dynamics, we suggest to start with a spatial unit: dwellings. We demonstrate the advantages of a dwelling panel by discussing the methodological issues, such as the implication of separating households from dwellings, and show how a dwelling panel can be constructed, how a bell-board technology can be applied , and finally how the panel can be maintained over several waves. Further, we discuss panel attrition in the case of a dwelling panel. Drawing upon our ongoing study on gentrification in two Cologne residential areas, we demonstrate the as well the complex data structure after two waves. I a concluding section we illustrate how the dwelling panel can overcome the problem of selection bias and illustrate its advantages to account for urban processes.