Challenges and Opportunities of Switching to Web 2 |
|
| Session Organisers |
Miss Elise Braekman (Sciensano) Miss Hannah Carpenter (Kantar Public) |
| Time | Wednesday 17th July, 09:00 - 10:30 |
| Room | D25 |
This session includes papers that address the challenges and opportunities of switching survey designs to web-based data collection, particularly in push-to-web mixed mode designs.
Keywords: mixed mode, push-to-web, web first, mode effects
Miss Emmanuelle Duwez (CDSP - Sciences Po)
Mr Mathieu Olivier (CDSP - Sciences Po)
Mr Thomas Pilorin (CDSP - Sciences Po)
Mr Malick Nam (CDSP - Sciences Po)
Mr Simon Le Corgne (CDSP - Sciences Po) - Presenting Author
The European Social Survey (ESS) is an academically driven cross-national survey conducted every two years across Europe, to which France has participated since the first round in 2002. The ESS measures the attitudes, beliefs and behaviour patterns of various populations in more than thirty countries. One of its main goals aims to provide analysis elements on how Europe’s social, political and moral fabric is changing. In the ESS, data are collected via face-to-face interviews.
In France, the fieldwork of the 7th round led to a survey replication on the panel ELIPSS (Longitudinal Internet Studies for Social Sciences). Every month the panel members answer a 30 minutes self-administered questionnaire developed by researchers and selected by a scientific and technical committee. In order to answer the surveys, they are provided with a touch-screen tablet and a 3G Internet subscription.
In the ELIPSS panel, members are randomly selected by the French census bureau (INSEE). Traditionally, this is also the same for the ESS, but it was different for the 7th round. This time, the fieldwork institute in charge of data collection has proposed its own sampling methodology. One can find significant differences in methodologies between these two surveys, from the sampling procedure to the data collection collect method and analysis tools.
In 2017, we focused on the results comparison between the two devices; this year, we intend to focus on the different ways to set up a survey. In particular, we want to aim at the importance of the gender within the interviewer-respondent relationship. The replication of a face-to-face survey into an auto-administered panel makes it possible to study the effects resulting from being surveyed by a person of the same or the opposite gender. Consequently, it asks the question : to what extent auto-administered surveys could provide an access to objective measurement ?
Miss Hannah Carpenter (Kantar Public) - Presenting Author
Understanding Society (also known as the UK Household Longitudinal Study) adopted a sequential mixed mode approach in 2016. The study went from being conducted almost entirely face-to-face, to a large proportion of households being initially asked to take part online, and being followed up by a face-to-face interviewer if they did not.
This transition has, overall, been successful. Fieldwork costs have been reduced and high response rates have been maintained.
However, with higher proportions of households being issued web-first at each wave, the task for interviewers has become more challenging. ‘Keen’ respondents mostly take part online and interviewers are left with the more reluctant sample members. As the web survey remains available throughout face-to-face fieldwork interviewers are also required to manage web fieldwork: when sample members say they will complete online interviewers must monitor whether they do, and re-visit them if not.
In 2018 a new fee was introduced, so that interviewers received a payment for any web interviews completed after a household had been issued to them. Whilst this was not introduced experimentally, we can look at similar metrics for the fieldwork period before the fee was introduced and the period after it was introduced.
This paper will examine what happens when interviewers take on web-first cases. Do they put in as much effort as cases that go straight to CAPI? Do they continue to contact sample members who have said they will take part online? The paper will also look at what impact the fee paid to interviewers for web interviews has had on interviewer behaviour in both these respects.
Miss Elise Braekman (Sciensano) - Presenting Author
Dr Rana Charafeddine (Sciensano)
Mr Stefaan Demarest (Sciensano)
Mrs Sabine Drieskens (Sciensano)
Dr Johan Van der Heyden (Sciensano)
Dr Jean Tafforeau (Sciensano)
Professor Guido Van Hal (Antwerp University)
The European Health Interview Survey (EHIS) provides cross-national data on health status, health care and health determinants. Two waves have been launched and wave-3 is scheduled for 2019. Given its long and complex questionnaire, face-to-face data-collection is particularly suitable but expensive. Potential is seen in web-applications due to its cost-effectiveness and easiness of implementation. For EHIS wave-2, 10 of 30 member states integrated a web-based mode within a mixed-mode design but none used single-mode data-collection. This research tests web-only data-collection regarding response and cost-effectiveness.
A Belgian pilot study with a target sample size of 1000 (age: 15-85) was organized using a web-only survey containing EHIS wave-3 model-questions. A multistage, clustered sampling procedure with geographical stratification was applied. Field substitution was implemented: non-participants were substituted by individuals matched on statistical sector, sex and age. One reminder letter was sent and a €10 conditional incentive was offered. The response rate was assessed and the costs per completed web-based questionnaire were compared with these for a face-to-face questionnaire.
The total response rate was 16.0% which was lower than the EHIS wave-2 response rates of member states using mixed-mode web-based data-collection (26.2%-91.6%). In this pilot study, a lower participation was found for elderly, for singles, for people living in Brussels, for lower educated people and for people without the Belgian nationality. Web-based data-collection was three times cheaper than face-to-face data-collection due to no interviewer’s payment and less labor-extensive fieldwork logistics.
Developing an (adaptive) mixed-mode design for EHIS could be preferred over web-only data-collection to obtain a higher and more equal participation while keeping costs low.
Dr Arto Selkälä (University of Lapland) - Presenting Author
Professor Ulf-Dietrich Reips (University of Konstanz)
Dr Leena Viinamäki (Lapland University of Applied Sciences)
Professor Asko Suikkanen (University of Lapland)
The research on invitation mode for web surveys has revealed that a mailed letter or a mixed-method invitation works better than an e-mail invitation alone. Despite the progress in the research field of invitation procedures, little is yet known about self- selection of entry options on the postal invitation letter. To address this question, the present study used the data of a stratified sample of 1329 persons who received an invitation letter to participate in a web-based survey about the Finnish competence-based education system. We applied a multinomial logistic regression analysis to explain the selection of three entry options shown to all recipients on the postal invitation letter to the web survey:
1. The URL and password to the web survey;
2. A request to send e-mail to the researcher in order to receive a response link to the web survey;
3. A request to send a recipient’s e-mail address to the researcher by using SMS in order to receive a response link to the survey.
Results indicate that the odds of selecting Option 2 (“Response link by email”) instead of Option 1 (“Typing the URL”) is 4.1 times higher for those with an elementary school education and 3.2 times higher for those with a basic comprehensive school education compared with the participants with high school education. In addition the odds of selecting the “Response link by email” increase significantly by a factor of 1.05 (OR) for every year increase in age. We found no significant predictors of the comparison between Option 3 and Option 1. In conclusion, less educated and older recipients tend to select cognitively less challenging entry option to the web survey. An experimental study of a selection of entry options will be conducted and presented.