Mobile-First Survey Design: Going Beyond Mobile Optimization in Web Surveys 1 |
|
| Session Organiser |
Ms Laura Wronski (SurveyMonkey) |
| Time | Wednesday 17th July, 14:00 - 15:00 |
| Room | D02 |
People use their smart phones for everything: email, banking, dating, ordering a coffee, hailing a cab… and taking surveys. While the prevalence of smart phones presents many opportunities for web survey creators—who can now reach respondents on their phones 24 hours a day 7 days a week through several different means—it also raises the bar for respondents’ expectations when taking surveys on their phones. Accustomed to being delighted by the apps and mobile sites with which they regularly interact, survey takers are now looking to participate in mobile web surveys with the same ease—and even enjoyment—that they do everything else on their smart phones.
High quality web surveys, therefore, must be designed with an eye toward not just mobile optimization but with a mobile-first mindset. This session will gather research on innovative techniques and tools for web survey creators, particularly as it relates to survey design.
Topics may include:
- New recruitment methods, e.g. QR codes, geolocation-based surveys
- New question types, e.g. left/right swiping, sliding scales, voice response, photo upload
- New techniques to improve response rates, e.g. gamification, alerts/notifications
Keywords: mobile optimization, web surveys, innovation, survey design
Mr Oriol J. Bosch (Research and Expertise Centre for Survey Methodology (RECSM)-Universitat Pompeu Fabra) - Presenting Author
Dr Melanie Revilla (Research and Expertise Centre for Survey Methodology (RECSM)-Universitat Pompeu Fabra)
Mr Ezequiel Paura (Netquest)
Most mobile devices nowadays have a camera. Besides, posting and sharing images have been found as one of the most frequent and engaging Internet activities. However, to our knowledge, no research has explored the feasibility of asking respondents of online surveys to upload images to answer survey questions. The main goal of this article is to investigate the viability of asking respondents of an online opt-in panel to upload during a mobile web survey: First, a photo taken in the moment, and second, an image already saved on their smartphone. In addition, we want to test to what extent the Google Vision application programming interface (API), which can label images into categories, produces similar tags than a human coder. Overall, results from a survey conducted among millennials in Spain and Mexico (N = 1,614) show that more than half of the respondents uploaded an image. Of those, 77.3% and 83.4%, respectively, complied with what the question asked. Moreover, respectively, 52.4% and 65.0% of the images were similarly codified by the Google Vision API and the human coder. In addition, the API codified 1,818 images in less than 5 min, whereas the human coder spent nearly 35 hours to complete the same task.
Dr Melanie Revilla (RECSm-Universitat Pompeu fabra) - Presenting Author
Professor Mick P. Couper (University of Michigan)
More and more respondents are answering web surveys using mobile devices. Mobile respondents tend to provide shorter responses to open questions than PC respondents. However, voice input options are often used on smartphones: every day, more than 200 million voice messages are sent through WhatsApp. Using voice recording to answer open-ended questions could increase data quality, with longer/richer responses. It could also provide information regarding the tone of voice, and could help engage groups usually underrepresented in web surveys (e.g. older respondents).
We conducted an initial experiment exploring the feasibility of using voice-recording to answer open questions in 2018 in Spain. It showed that the use of voice-recording still presents a number of challenges, especially on Android devices. Voice-recording led to substantially higher nonresponse and significantly lower survey evaluations, relative to text input. However, completion time was significantly reduced using voice-recording. Among those who provided an answer, longer and more elaborated answers were obtained for voice-recording.
This presentation will report results from a follow-up experiment implemented within a smartphone web survey in Spain where we tried to improve the way the voice recording task was introduced to the respondents to reduce nonresponse, in particular due to lack of understanding. In this follow-up experiment, we focus on Android devices. Participants are randomly assigned to a control group (similar to the voice-recording group of the first experiment), or one of the following treatment groups: 1) new formulation of the instructions on how to use the voice-recording button, 2) shorter formulation of the instructions, 3) shorter formulation plus feedback, or 4) shorter formulation plus feedback plus use a test question. The experiment will be conducted in late 2018. We expect that treatment 4 will lead to the lowest nonresponse.
Mr Zewei Zong (SurveyMonkey) - Presenting Author
Mr Jack Chen (SurveyMonkey)
For more than a decade, millions of survey creators have used SurveyMonkey to send web surveys to their customers, employees, target markets, students etc. With access to an incredible amount of survey paradata, SurveyMonkey is in an unique position to provide a “State of Surveys” report, which gives a snapshot of surveys conducted online. In particular, the large number of respondents taking surveys on SurveyMonkey using mobile devices gives us a great opportunity to investigate what role mobile devices play in the survey business. Has the percent of respondents taking surveys using a mobile device been growing for recent years worldwide? Are there any significant differences among different countries? What is the distribution of screen size for the mobile devices that respondents are using to take surveys? We will answer important questions like these for the benefit of researchers developing new techniques in an effort to optimize the mobile survey-taking experience.