Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Is it Worth Mixing Modes? |
|
| Chair | Dr Teresio Poggio (University of Bozen-Bolzano) |
The Fishing, Hunting, and Wildlife-Associated Recreation Survey (FHWAR) team incorporated a mixed mode design for the 2011 survey out of necessity. In the past, FHWAR was an all computer-assisted personal interviews (CAPI) survey, but recent budget restrictions have significantly reduced, and will continue to reduce, the ability to conduct interviews through CAPI. In 2011, the survey design was changed to incorporate the less expensive interviewing method, computer-assisted telephone interviews (CATI). CAPI interviews usually result in a higher response rate; this was the case for 2011 FHWAR interviewing. Due to low contact rates in the initial CATI, emergency design changes had to be implemented. Because of these changes, CAPI, CATI, or a mixture of both were used to contact individual cases.
The FHWAR team studied the response rates and participation rates by data collection mode: CAPI, CATI, or a combination of the two. The complex survey design includes weights, strata, and clusters; all used to calculate response rates and the estimates of wildlife participation in the United States of America by collection mode. Results show CAPI produced better response rates but lower proportions of wildlife participation. Mixed mode data collection has the potential to improve future FHWAR sampling designs by increasing response rates and finding a higher proportion of wildlife participants.
As part of its electronic data collection programme, the Office for National Statistics is planning to introduce web collection in the UK Labour Force Survey. In this paper, we describe the design of the pilot, including the split sample design and the various multi-mode designs that we would like to evaluate. The set of designs we would like to consider includes a simple sequential design and a two-phase hybrid design. In the latter, the first phase involves selecting a large sample of addresses to invite them to register for the LFS on the web, and to obtain basic information about the household. In the second phase, a subsample of the full sample will be selected for a Face to Face (FtF) interview; the addresses in the sample that register for a web collection and are not selected for a FtF interview will be contacted for a web collection.
We are considering a modular structure to the LFS questionnaire, with a cross-topic module, for all addresses selected for interview, and a number of topic-specific modules, each module assigned to a subset of the addresses. We will discuss the implications this will have on the sampling design and sample size.
We also present proposals for measuring mode effects and adjusting for them in estimation for each of the multi-mode designs we consider, and discuss the potential practical problems we could encounter when undertaking the pilot, including field management.
This paper focuses on mode and incentive effects on substantive variables as well as associations between these variables. We use data from the 2011 Swiss Electoral Study (Selects), which included an additional web experiment conducted using the same questionnaire and probability sampling frame as the usual telephone survey. This allows us to overcome limitations of previous similar comparisons, which mostly used opt-in panels. As the web survey was separated into incentive and control groups, we can also study ways to improve the engagement of participants in internet surveys.
Theory and previous research lets us expect disparities in point estimates due to differences related to coverage, nonresponse, and measurement error in the three scenarios. It is possible to partially control for coverage error using sociodemographic information from the sampling frame and for selection effects by comparing the actual vote choice and turnout figures from the election to reported behaviour - a specific benefit of electoral studies. Results from univariate analyses show that the regular internet survey produces results very close to the telephone survey, while the incentivized internet survey differs more often from the two other samples. However, the latter yields figures that are closer to actual election results, even after controlling for sample composition. Relationships between variables are tested using regression models, which show few differences between the three conditions. In general, the results speak in favour of the incentivized internet survey. However, care must be taken when comparability of time series is an important factor.
Web surveys have several advantages: low data collection costs, simplified and fast field operations. They also allow to expose subjects to different stimuli.
However, coverage is a major reason of concern, with the exception of studies targeted to specialized - ICT literate - populations.
When considering general population surveys, is it possible to take advantage of web questionnaires while using a probability-based sample with limited problems of coverage? If so, what are the main methodological and operational implications?
Trying to provide some answers to these questions, the paper discusses the main evidences from a mixed-mode survey carried out on young households in Trento (Italy, Spring 2010) with a 90% response rate (816 interviews).
The main focus of this study was on childcare and nursery schools. The survey was also intended as a methodological experiment, in order to explore alternatives to the (sole) CATI surveys, because of coverage problems of the latter.
Sampling and contact phases (population registers as a frame and mail contacts respectively) were distinguished from data collection modes: CAWI (60% of the interviews) + CATI (27%) + PAPI (13%).
Immigrant parents were of special interest to the research. They were oversampled. Contact materials were delivered in 12 different languages. CATI and PAPI interviewers covered the same set of languages.
The survey reasonably succeeded in balancing the production of good quality data - in terms of good coverage and use of a probabilistic sample - with reasonable costs, thanks to the possibility to use CAWI and CATI wherever possible.