Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Household panel surveys: recent developments and new challenges |
|
| Convenor | Dr Emanuela Sala (Dipartimento di Sociologia e Ricerca Sociale, Università di Milano Bicocca) |
| Coordinator 1 | Dr Jonathan Burton (ISER, University of Essex) |
Traditionally, panel surveys have been considered powerful research resources to study how people's socio-economic circumstances change over time. Although this remains their main goal, panel surveys are facing additional challenges. These include the increasing demand to supplement survey data with administrative data; the requests from the funding agencies to collect direct measurement of health conditions; the need to implement methodological experiments to improve the quality of the data collected; the trade offs between granting certain quality standards (in terms, for example of response rates) and limited financial resources available; the increasing constraints posed by the ethical committees .
With regard to these challenges, panel surveys have to deal with methodological issues that are specific to this kind of surveys. The aim of this session is therefore to provide an overview of the methodological issues that survey practitioners have to deal with while facing these new challenges. The focus of the papers will be on household panel surveys. Examples of the topics of this session include (but are not restricted to):
- methodological issues in linking survey data to administrative data,
-experiments in questionnaire content or fieldwork procedures,
- methodological issues in the collection of biomarkers and cognitive measures,
-innovative ways of collecting data.
The Survey of Income and Program Participation (SIPP) is a nationally representative, longitudinal survey that collects monthly details over a four-year panel. The Census Bureau has been reengineering the SIPP to continue its mission, providing a nationally representative sample with accurate and comprehensive sub-annual information for evaluating the dynamics of income, movements into and out of government transfer programs, the family and social context of individuals and households, and interactions between these areas. A key component of the reengineering process is the development of a replacement that moves from three-times-a-year to annual interviewing. The development of an Event History Calendar (EHC) interview that allows conversational and dependent data collection, the development of a new processing system, and new training procedures were technological, methodological, and logistical challenges, and have required multiple field-test evaluations.
The decision to reengineer the SIPP was born through fiscal crisis. We discussed many options to maintain or improve the SIPP's ability to meet its mission while dramatically reducing costs. These included this dramatic survey redesign, as well as several administrative-records based options, and hybrid-survey-records options. During reengineering, we conducted detailed evaluations of the administrative-records data available and options to replace, supplement, or enhance collected survey data, undertook an extensive period of development and evaluation of the new survey design, initiated a complete redevelopment of our data processing system and interviewer training. This paper discusses the development, evaluations, challenges, successes and lessons in reengineering the SIPP.
Longitudinal studies are of paramount importance to study how people's circumstances change over time and have become increasingly popular in many different research fields. Recent years have seen however a surge in the challenges posed to longitudinal designs. In particular, while the rising pressures on government coffers have clearly limited the amount of resources available for scientific research, the demand for high-quality (panel) data rose considerably as well as the costs necessary to provide the desired quality standard. Against this background, increasing attention is devoted to the use of nonprobability samples for scientific research as cost-effective alternatives. To date, quite a few studies have compared the representativeness of probability and nonprobability samples as well as the quality of the collected measures, both in term of accuracy (i.e. the difference from trusted benchmarks) and validity (such as testing the correlation with theoretically related items or the predictive power of the answers). The evidence so far is however scanty. Furthermore, no study has carried such comparisons within a longitudinal setting. The present work contributes to this literature analyzing the validity of the answers given by a probability and a nonprobability sample in the household panel survey "Saving and Old-Age Provision in Germany" (SAVE), which consists of two subsamples differing by their sampling scheme. The concurrent and predictive validity of the answers as well as the degree of satisficing are taken to assess measurement quality and are compared over time.
In August 2010 a national regulation for statistics on income and living conditions came into effect in Austria, providing the legal basis for linking personal records from register data to survey data collected in the Austrian EU SILC survey. This link between register and survey data enables the use of register data for the computation of household income variables from EU-SILC 2012 onwards.
For EU-SILC 2010 incomes from both sources - register and survey - are available. The presentation describes the evaluation process undertaken to prove the suitability of the use of register data. This process included the assessment of the frames, the availability, the data quality of registers, the linkage with survey data, the calculation of comparable income variables and the comparison of income variables. The comparison focuses on two questions: First, how do these data sources match? And second, are the distributions of the received incomes comparable? For both questions the difference between data sources was of particular interest. Beside the description of the evaluation process the presentation provides an overview of the consequences of the use of register data for the EU-SILC operation in Austria.
The German Socio-Economic Panel Study (SOEP) is a wide-ranging representative longitudinal study of private households. Every year, nearly 11,000 households and more than 20,000 individuals are interviewed by the fieldwork organization TNS Infratest Sozialforschung. Cognitive competencies of respondents are measured with two ultra-short cognitive performance tasks that were developed by the SOEP to allow for reliable assessment of general intellectual ability and that distinguish between two components of intellectual functioning: cognitive mechanics and pragmatics (e.g., Lindenberger & Baltes, 1995). Cognitive mechanics are hard-wired, biologically based capacities for information processing and are measured with the Symbol Digit Test (SDT). Pragmatics are educational and experience-related competencies and are measured with the Animal Naming Task (ANT).
Furthermore, the SOEP group is currently creating a new Innovation Sample (SOEP-IS) while at the same time increasing the size of the core SOEP. The SOEP-IS will run from 2012 to 2017, with a cumulative target size of N=5,000 households. Starting with the 2012 survey, a CAPI version of the Day Reconstruction Method (DRM) has been given to respondents. The DRM is used to determine how people spend their time and how they experience their various activities and surroundings. The aim is to collect data on a person's experiences in a given day through a systematic reconstruction of his or her activities on the following day.
The presentation will give an overview of the measures of cognitive competencies and the CAPI adaptation of the DRM.
The wording of a survey question is one of the key determinants of the quality of responses. Measures of a particular construct may differ, depending on the question wording, since the wording is likely to influence the recall and judgment strategies used by respondents to compute an answer.
We focus on questions about dates of events and examine the quality of survey data obtained when respondents are asked about dates explicitly (e.g. "in what month and year...?") versus implicitly (e.g. "when did you...?" or "in which months did you...?").
We investigate whether question wording affects the quality of reported dates, by examining the extent of "heaping", that is, the tendency of respondents to report durations since an event that are multiples of, say, 6 or 12 months. We further investigate whether the question wording affects the response strategies used by respondents, how response strategies relate to the quality of responses, and whether certain groups of respondents are more sensitive to the question wording. We report findings from a set of experiments carried out in waves 2, 3 and 4 of the Innovation Panel of the UK Household Longitudinal Study.
Preliminary analysis shows that (i) heaping is more likely with implicit date requests, (ii) respondents use different retrieval strategies to recall or reconstruct dates of events, depending on the question wording, (iii) respondents' recall strategies are related to the quality of responses, and (iv) we found no differences between different types of respondents in their sensitivity to the question