Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Mixed Mode Surveys - Reports from the Field Work 1 |
|
| Chair | Mr Patrick Schmich (Robert Koch-Institute) |
Most of the literature about mixing modes of administration focusses on surveys to individual citizens. Our paper analyses a survey to 400 municipalities implemented in two steps. The first one was an on-line survey with a limited response rate (36%). The second step was a follow-up telephone survey that increased the response rate significantly (70%). However, mixing the results is problematic since each mode of administration appears to have introduced its own biases. The paper describes the procedures and results of both survey modes and specially focusses on the reasons that may explain the differences between both modes
In recent years, web-based questionnaires have become a common method for collecting student based health behaviour surveys. Compared to traditional 'pen and paper' methods it provides an efficient approach to surveillance but may affect individual response. The aim of this study was to examine the optimum mode for collecting behavioural information among a third level student population, comparing web and lecture based sampling for the first time.
A total of 2,275 self-completed questionnaires were obtained by distribution at lecture theatres yielding a response rate of 51%. 333 undergraduate students responded to the web-based questionnaire yielding a response rate of 2.4%. Mantel-Haenszal tests, independent samples t-tests and multivariate logistic regression were utilised to investigate the impact of mode of collection on response.
The sample was predominantly female (60.8%), in first year (36.7%) and living in their parents' house (41.6%) or in a rented house/flat (42.7%). Those who completed the web-based survey were 60% less likely to report high levels of physical activity but twice as likely to have taken recreational drugs in the previous 12 months. Independent samples t-tests showed that alcohol consumption and mental health and well-being scores were lower among the lecture theatre sample.
Significant differences in student reporting of sensitive data and demographic representation have been uncovered between web and lecture based sampling methods. Thus, lecture based sampling as opposed to a web-based are advocated for behavioural surveys among third level students.
The most significant challenges in survey research nowadays is to tackle against an increasing rate of nonresponse and decreasing financial resources. New technology offers a splendid platform to attract respondents, especially urban young people. A web survey is available 24/7 and requires no other efforts. In a survey process sending and receiving traditional mail requires weeks altogether. Quality questions are also on the agenda. In web survey the option of using control keys avoiding wrong or invalid answers is an efficient method maintaining the quality issue. In paper, for example a respondent can give any answer he or she likes to and afterwards checking the data requires weeks of working hours. Still, one of the big issues there is to formulate questions in a way that suits for both of the methods. An interesting example of mixed-mode surveys in Finland is International Social Survey Programme (ISSP) that is an annual social survey, which is conducted by Statistics Finland. There are two options to participate: a respondent choose either web-option or traditional questionnaire. The majority of the respondents participate in the web, but there is still almost another half of the people, who does not feel comfortable with the new technology. Why not offer a mixed-mode option to be able to reach for every optional respondent?
At the end of 2005 Statistics Netherlands (SN) conducted a large-scale experiment (n = 10,995) in which incentives (stamps, value: 4 euros) were included in the advance letter of the Labour Force Survey. This increased response rates from 66% to 73%. Incentives were particularly successful in the big cities, but were not very effective for immigrants (Wetzels et al. 2008).
High response rates were also reached in the Dutch Parliamentary Election Studies in 2006 (stamps) and 2010 (stamps and gift vouchers - value: 5 euro - randomly assigned): 72% and 67% respectively. Stamps and vouchers were equally effective. Again however, little effect was found for immigrants.
In 2010 SN started using a sequential mixed-mode data-collection (web/cati/capi) for most of its surveys. In 2012 incentives were included in the advance letters of the Housing Demand Survey (n = 126,553) and the Survey on Social Cohesion (n = 7,500) at the end of the data collection period due to rather low response rates (below 55%). Results show that also in a mixed-mode design using incentives improved response by approximately 7 percentage points. By linking these surveys to register data, we will show how smaller subpopulations (e.g. non-western immigrants and urban/rural areas) and nonresponse bias are affected by the use of incentives in mixed-mode surveys.
Wetzels, W., et al. (2008). Impact of prepaid incentives in face-to-face surveys. A large-scale experiment with postage stamps. International Journal of Public Opinion Research, 20:507-16