Push2web surveys: How to encourage people to go online when using a different contact mode? 2 |
|
| Chair | Ms Gerry Nicolaas (Ipsos MORI ) |
| Coordinator 1 | Dr Patten Smith (Ipsos MORI) |
The use of mixed-mode surveys has become common in recent years, especially in countries with high internet penetration. The purpose of these mixed designs has been threefold: to improve population coverage, improve response rate and reduce survey costs. Self-administered questionnaires, both postal and web questionnaires are accepted as valid modes of data collection by the International Social Survey Programme (ISSP) making it an ideal vehicle for methodological experiments. This paper describes results from four different modules of the ISSP in Iceland where a random sample of the general population was either offered a choice of answering the questionnaire on paper or online, pushed to answer on the web or randomly assigned to the different modes in order to disentangle selection and mode effects.
The ISSP module on Environment in 2011 was implemented in Iceland as a mixed-mode survey: a probability sample of the general population under age of 50 was first contacted by telephone and asked to complete a questionnaire on the web, while those over age of 50 were allowed to choose web or mail (a concurrent mixed-mode design). In the ISSP module on Citizenship in 2014 five countries participated in a mode experiment where a probability sample of the general population was randomly assigned to different modes, web or paper. These countries were Denmark, Finland, Iceland and Norway, all with internet penetration over 90% and Spain with a somewhat lower internet penetration rate, under 80%. In four out of five countries, response rate turned out to be higher in the web mode than in the paper mode. In the ISSP module on Work Orientations 2015 a probability sample of the general population was, in a letter, invited to answer the questionnaire online but asked to send an e-mail or call if they preferred to answer on paper. Finally, in the ISSP module on Role of Government in 2016 a probability sample of the general population was invited to answer the questionnaire online, but telephone calls and text messages were used to boost the response rate.
Results show that response rate in internet surveys in countries with high internet penetration is in many cases higher than with other modes – and that for most of the survey variables in many respects a web survey alone may be the optimal method of collecting data, when considering the quality of answers, costs, bias and mean squared error.
The presentation will focus on the results of a Push2web(+mail) survey carried out in 2014 in Spain parallel to the International Social Survey Program face-to-face survey (ISSP Citizenship). Both surveys were carried out by the Centro de Investigaciones Sociológicas (CIS, www.cis.es), a public institution that has been doing around 50 political and social attitudes surveys every year since the mid 1970s. This Push2web(+mail) ISSP study was conceived as an experimental one, parallel to the usual face-to-face survey, since it was the first time the CIS such methodology in one of its surveys (and, to our knowledge, the first time it was used in a social attitudes survey in Spain). At the time the rate of penetration of the Internet was around 75%, but still very unequal among different age groups.
The Push2web(+mail) ISSP Citizenship survey used a full probability sample of 3,000 individuals over 18 years old, selected by the Spanish National Statistics Institute from the Population Register. Letters were addressed to all individuals in the sample with a URL and an individual password, inviting them to participate in the survey. After two reminders, those sampled individuals who had not completed the questionnaire online received a third letter with a paper questionnaire and a reply-paid envelope.
Although the initial expectation was to have a low response rate, due to the lack of habit of the Spanish population with self-administered questionnaires, the survey obtained a very reasonable response rate (34% considering only Web and 44% including paper questionnaire follow-up). The presentation will analyze the response rate at the only-web phase and compare it with the mail mode (and comparing it altogether with the face-to-face mode), looking at the different socio-demographic profile of respondents. Several questions on Internet usage included in the questionnaire will make it possible to analyze the choice of mode Web/Mail according to this criterion.
The Web+Mail sample split in three to do an experiment with incentives (one third of the sample was offered no incentive, on third a non-conditional 5 euros voucher and the last third was offered the same incentive conditional upon completion of the survey. The impact of the incentive upon response rates will also be analyzed in the presentation.
Over the last four years Kantar Public (formerly TNS BMRB in the UK) have developed and refined a pioneering (to the UK) low-cost random sample survey method - Address-based Online Surveying.
The drive for change was initiated by the Cabinet Office [1] which wanted Kantar Public to invent and test a lower cost version of its in-person interview Community Life survey. This offered, at the time, a unique opportunity to explore the feasibility of moving an official statistic from an in-person to online/postal measurement.
The core UK ABOS design involves drawing a stratified random sample of addresses from the Royal Mail's postcode address file and sending an invitation letter to the occupant(s) at the sampled addresses inviting resident adult(s) to complete the survey online. Postal questionnaires are used to ensure inclusion of individuals who are offline.
Over the last couple of years Kantar Public have gathered response information across various ABOS studies and undertaken numerous experiments to help understand which design features work best in maximising response and/or improving sample representativeness.
This paper brings together findings and conclusions from various ABOS studies, presenting evidence where available. With a particular focus on:
• What response rate can be achieved?
• How does response vary across ABOS studies and why might this variation occur?
• How can response be maximised?
• Does response vary between subpopulations and what can be done about it?
[1] The Office for Civil Society is now situated within the Department for Culture, Media and Sport. This includes the responsibility for the Community Life Survey.
We use as a case study the Hellenic National Election Voter Study of 2015, which was conducted as a mixed mode combining telephone (CATI) and web (CAWI). In a list-based probability sample of a general population survey, web cannot be the only mode used, due the severe coverage issues that can arise. In Greece there is not a reliable official source which includes all the mail addresses of the population of the country. Hence, telephone mode using RDD is the most suitable offline mode which was preferred instead of the most expensive face-to face interviews.
In this study, we have tried to encourage people who were contacted via telephone to respond to a web survey. We have contacted selected respondents via telephone, and we have asked them to provide us with their email addresses in order to send them an email invitation to complete the online questionnaire. We were still giving the option to older people who usually do not have email addresses to answer the questionnaire through a telephone interview, in order to enable people who are less familiar with new technologies and Internet to participate in our survey.
In this paper we present data related to whether people were reluctant to provide email addresses on the phone and how many of them who have provided their email addresses, have answered the online questionnaire. In addition, we focus on the impact of reminders to the response rate of the survey. Maximum 6 follow-up reminders were sent through email to the respondents who had not completed the questionnaire, in order to increase the response rate of the survey. Almost 2245 invitations were sent and 60 respondents were interviewed via telephone. After sending all reminders we have collected 940 fully completed questionnaires, 336 partially completed questionnaires and 25 refusals (people who informed us that they have changed their mind and they are not interested in participating).
By taking advantage of new technology tools we have managed to keep expenses down pushing people to web instead of the more expensive telephone interviews. In a period, when research funds for the social sciences are decreasing - not only in Greece but also in a global level - we present this innovative low cost survey design as a reliable alternative. Especially if we take into account that in the future the internet penetration will be higher, restricting the coverage issues that exist, the need to push to web becomes more and more urgent.
The aim of this session is to provide the audience with the most up-to-date thinking and evidence on how to encourage people to complete an online survey when using offline contact modes.
In this final presentation, the session organisers will summarise the main highlights from the papers and draw overall conclusions from the session as a whole. They will also present a practical framework for maximising response to a push2web survey and identify areas for further research. The session will conclude with a discussion with the audience.