Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Friday 17th July, 13:00 - 14:30 Room: O-202

Marrying survey methodology with survey management: Minimizing the Total Survey Error (TSE) with limited resources during fieldwork
Convenor	Dr Frederic Malter (Max-Planck-Society )

Session Details

The key goal of any survey is to deliver statistics with minimal errors that facilitate correct conclusions about the target population. It is easy to understand how applying the TSE can be a useful concept for minimizing the risk of flawed survey statistics at the “early life” of a survey, i.e. the design stage (e.g. designing survey items or scales with good psychometric properties) or the “late life” stage, i.e. the post-production phase such as applying post-stratification weighting to account for unit non-response. It is much less clear, however, how to allocate the limited resources at a survey’s “mid-life” stage, i.e. the fieldwork phase, to the various components of the total survey error. For example, how do surveys allocate resources to minimize measurement errors arising from non-standardized interviewing? A practical example is interviewers’ shortcutting question texts or introduction texts that will create non-standardized interviewing. Another example may be issues arising from sampling errors: how do surveys minimize the risk of unit nonresponse with their limited resources?
The goal of this session is to bring survey managers and survey methodologists together to discuss solutions to the problem of allocating limited resources (training, survey managers’ time, incentives for interviewers etc.) to the various components of the TSE while fieldwork is still ongoing. Ideally, the session will yield ideas on important “set screws” and how to get the biggest bang along the two major lines of intervening: training/managing interviewers and providing monetary or non-monetary incentives.
Any contribution applying the TSE ex-ante to fieldwork management or contributions of survey studies retro-fitting principles of TSE to their current fieldwork management are welcome. Ideally, submission will briefly lay out how the input side (e.g. incentives) is mapped onto the output.

Paper Details

1. How to effectively reduce the total survey error when resources are limited? A survey agency’s perspective on responsive design for two case studies from Germany
Dr Thorsten Heien (TNS Infratest Sozialforschung)
Mr Jochen Heckmann (TNS Infratest Sozialforschung)

The question of how to reduce measurement and nonresponse errors as important TSE components is discussed by two examples: The “Survey of Health, Ageing and Retirement in Europe”, an international longitudinal CAPI study, and the survey on “Old-Age Incomes in Germany”, a national cross-sectional multi-mode study. By referring to the fieldwork “set screws” of sample management, interviewer management and respondents’ incentives, the paper illustrates by which means, to what degree and at what cost measurement and nonresponse errors could be decreased in both surveys in terms of a “responsive” design to improve cost efficiency and quality.

2. Quality in Telephone Surveys: Market Research Companies Perspective
Dr Wojciech Jablonski (University of Lodz)

According to Groves (2011), there is a breach between the government and academic surveys on the one hand, and private sector on the other. Within this paper, we will outline the results of the study what was carried out between 2009 and 2010, as well as in 2013 among 12 major Polish commercial survey organizations. One of the main aims of this project was to investigate the methodological procedures that are implemented by these agencies and to assess to what extent they are in line with scientific rules and survey industry recommendations.

3. Improving the quality of paradata collection on interviewer-mediated surveys
Mrs Lucy Haselden (Centre)
Mr Andrew Cleary (Ipsos Mori)
Mr Stephan Tietz (Ipsos Mori)

Obtaining high quality and timely paradata is essential for effective fieldwork management on random probability surveys but challenging on face-to-face interviewer-mediated ones. This paper describes and evaluates the new real-time electronic system designed to improve paradata quality on wave 6 of the UK Millennium Cohort Study (MCS). The system combined fieldwork monitoring and mover tracing functionality with improved ability to ensure compliance with complex fieldwork protocols. The effectiveness of this is examined by comparing the paradata collected using the new system with the same paradata collected using more traditional contact sheet system on the previous wave.

4. Total Survey Error across a program of three national surveys: using a risk management approach to prioritise survey error mitigation strategies
Ms Sonia Whiteley (Australian Centre for Applied Social Research Methods)

The Australian Government recently announced the Quality Indicators for Learning and Teaching (QILT). QILT consists of three surveys of one million students throughout the student lifecycle into employment.

A Total Survey Error (TSE) approach is already in place, however, mitigating all identified errors during the initial survey cycle would be costly and put additional pressure on clients already stressed by the introduction of QILT. To address these complexities, a risk management approach was developed to assess each error and prioritise it for remediation. The integration of TSE and risk management frameworks for QILT will be discussed from a change management.