Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Friday 17th July, 11:00 - 12:30 Room: HT-105

Estimating effects of modes and mixed modes designs 1
Convenor	Mr Alexandru Cernat (Institute for Social and Economic Research, University of Essex )

Session Details

Traditional approaches to data collection in the social sciences (i.e., face to face and telephone surveys) are becoming more expensive. At the same time cheaper approaches, such as web surveys, lack traditional sampling frames. This has led to a surge in data collection designs that aim to combine the strength of each mode into a single survey. In this context, accumulating evidence that informs design decision in mixed modes surveys is essential.

This session will contribute to this debate by tackling some important topics such as:
- Is the effect of social desirability moderated by mode?
- How do self-administered strategies (e.g., paper and web) differ in data quality?
- Are traditional scales (like those measuring personality, depression, cognitive ability) equivalent across modes?
- How does selection/non-response bias differ across modes?
- Does the use of mixed mode data impact substantive results?
- How does research on mixed mode integrate in the Total Survey Error framework?
- How to prevent mode effects through design?

Paper Details

1. Comparing estimates across survey designs – are mode effects the greatest cause for concern?
Dr Michèle Ernst Stähli (FORS - Swiss Centre of Expertise in the Social Sciences)
Professor Caroline Roberts (University of Lausanne)
Ms Rosa Sanchez Tome (University of Lausanne)

A number of high-quality repeated cross-sectional and longitudinal surveys using ‘traditional’ single mode designs are considering switching to a mixed mode alternative. This has stimulated a large number of studies designed to assess the impact of mixing modes on the estimates produced, and in particular, their comparability with estimates obtained in the original survey design. In this paper we present the results of a mode experiment conducted in the context of the Swiss European Social Survey. We raise questions about the relative importance of mode effects alongside the multitudinous other variables that influence survey data quality.

2. Mixed mode designs and sensitive questions, an experimental design comparing CATI and CATI/Web
Professor Peter Kriwy (Sociology, University of Technology, Chemnitz, Germany)
Dr Gerhard Krug (Sociology, University Erlangen Nuremberg, Germany)
Mr Johann Carstensen (Sociology, University Erlangen Nuremberg, Germany)

This presentation compares a mixed mode telephone and online interview to a single mode telephone interview with regard to (1) response rate and (2) measurement error due to social desirability bias. Within a health survey, an experiment was conducted. Individuals were randomly assigned to a single mode telephone and a mixed mode telephone and online survey (respondents choose to participate via telephone or online survey by themselves). In the mixed mode group the response rate was significantly higher, and unit nonresponse error was smaller. Results of regression analysis indicate that social desirability bias was reduced as well.

3. Estimation of Mode Effects in the Health and Retirement Study using Measurement Models
Mr Alexandru Cernat (Institute for Social and Economic Research, University of Essex)
Professor Mick Couper (Joint Program in Survey Methodology, University of Michigan)
Dr Mary Beth Ofstedal (Population Studies Center, University of Michigan)

Using multiple modes to collect data is becoming a standard practice in data collection agencies. While this helps to save costs and may decrease non-response it may have detrimental effects on measurement quality. In this paper we use a quasi-experimental design from the Health and Retirement Study to compare the measurement quality of a number of scales between face to face, telephone and web. Some scales, depression and physical activities, show systematic differences between interviewer administered modes and the self-administered one while others, religiosity and cognitive ability, show inconsistent patterns.

4. A systematic review of mixed-mode research in the European Social Survey
Dr Ana Villar (City University London)
Mr Rory Fitzgerald (City University London)
Miss Yvette Prestage (City University London)

Like the European Social Survey, many survey research programmes are considering the possibility of introducing mixed-mode data collection, generally driven by a need to reduce costs while maintaining a similar standard of quality. Survey programmes also wish to use new technologies because of their flexibility and timeliness. The ESS has been conducting experiments to test the feasibility of implementing mixed-mode data collection designs for over 10 years. This paper summarises findings from 6 studies conducted in 10 countries, involving 4 modes and numerous experiments, highlighting consistent findings, contradictory findings and research gaps.