Online and smartphone survey innovation |
|
| Session Organiser | Dr Carina Cornesse (University of Mannheim) |
| Time | Friday 2 July, 13:15 - 14:45 |
A lot of innovation is happening online, in particular since the COVID-19 pandemic. Examples include novel designs for smartphone surveys, innovative mobile app studies, elaborate web probing designs, and the use of online health services. This session tackles a variety of such online and smartphone survey innovations.
Keywords: Smartphones, apps, web probing, online health services
Ms Xin (Rosalynn) Yang (University of Maryland) - Presenting Author
Ms Ai Rene Ong (University of Michigan)
Ms Yanzhi Shen (University of Maryland)
Mr Christopher Antoun (University of Maryland)
Mr Brady West (University of Michigan)
Mr James Wagner (University of Michigan)
Ms Jennifer Sinibaldi (National Science Foundation)
Mr John Finamore (National Science Foundation)
Participatory design (PD) methodologies engage end-users throughout the product development process (Grudin and Pruitt, 2002). PD practices often involve hands-on design activities, allowing designers to observe users’ design ideas and understand user needs (Sanders et al, 2010). Researchers have used PD in the domains of software development and public health (Halskov and Hansen, 2015). However, the use of PD methodologies has yet to be explored in survey research, directly engaging potential survey respondents.
In this paper, we will discuss results from three 90-minute online PD meetings conducted for developing a smartphone app to administer the Survey of Doctorate Recipients (SDR) in small parts (or “modules”). Study participants will be a diverse set of recent doctorate recipients directly recruited from respondents to the Early Career Doctorates Survey (ECDS). In these meetings, participants will first discuss their survey taking experience and thoughts about survey data collection. They will then be divided into small groups to work on an implementation approach for the survey app using Google Jamboard. Each Jamboard link includes basic information about the survey, as well as interactive tasks to be completed by participants, such as determining the ideal length and timing of survey modules, and identifying desired features for the smartphone app. Participants will also be instructed to use the sketching tools available in Jamboard to design their own mockups of the data collection app. Finally, participants will present their ideas and mockups and have them evaluated by the other participants.
We will report both substantive and methodological findings from these meetings, covering major themes related to survey data collection that emerged throughout the discussion, app features desired by participants, as well as participants’ receptivity to online PD meetings. These results will be of interest to researchers considering using PD methods for data collection app development and survey experience design.
Mr Max Felder (FORS) - Presenting Author
Mr Nicolas Pekari (FORS)
Dr Oliver Lipps (FORS)
Ms Nathalie Eggenberg (FORS)
Ms Gisana Riedo (FORS)
Ms Coline Kaufmann (FORS)
Mr Victor Legler (FORS)
Ms Michelle Cohen (FORS)
Aiming to broaden the use of discrete chose experiments (DCE) to applied research fields such as the acceptability of environmental policies, a pilot study was designed to analyse the experience and behaviour of participants when confronted with a forced choice task. Considering the growing use of DCEs in online questionnaires, combined with the fact that participants increasingly respond via mobile devices, our experiment aimed at testing how the choice task was perceived, as well as how individuals responded and behaved during the task, depending on the presentation of the scenarios and the type of device. To this aim, an online survey with adaptive design was conducted among a representative sample of 18 to 65-year-old individuals in French-speaking Switzerland recruited through an online panel. The discrete choice experiment consisted of five choices between two sets of seven pro-environmental behaviours (i.e. dimensions) each. The level of restrictiveness for each behaviour varied randomly (e.g. how often one could eat meat). It is common to ask respondents to rotate their devices horizontally to fit the entire width of the table on the screen. However, this is often viewed by researchers as a source of uncertainty and potential errors. We thus implemented a “vertical” variant to see whether this would yield different substantive results and better behaviour and experience (e.g. time spent, satisfaction, reported ease of the task). The tables of choices were presented either side by side or one under the other (vertical). These two layouts were randomly assigned to respondents.
We measured compliance with the rotation instructions in the side-by-side version throughout the five consecutive sets of scenarios. Familiarity with the use of mobile phones, usage frequency, and sociodemographics were used to predict how well different types of respondents complied with the instructions, as well as how they perceived the task and behaved during it. Results show that the mode of presentation, as well as the type of device used, do not significantly impact the way respondents report experiencing the choice experiment (i.e. how easy it was to find the information, how easy it was to make the choice, etc.) or the time needed to respond. In turn, familiarity with the use of mobile devices and age are found to be linked with behaviour regarding rotation of the screen. We find that even if a significant proportion of participants do not comply with the rotation instructions, it isn’t due to lack of ability. On the contrary, their ease with mobile devices allows them to complete the tasks without having to rotate the device. Finally, we find that data from both ways of displaying the scenarios yield the same substantive results. We therefore conclude that discrete choice experiments are well suited for use in online surveys, despite the large number of respondents using devices with small screens.
Dr Jose-Luis Padilla (University of Granada) - Presenting Author
Ms Dörte Naber (GESIS-Leibniz Institute for the Social Sciences Manheim)
Ms Maria del Carmen Navarro-Gonzalez (University of Granada)
Dr Katharina Meitinger (Utrecht University)
The growing use of web surveys is a challenge for the traditional methods of pretesting survey questions. Qualitative methods like “web probing” allow researchers to obtain evidence of the response processes “in vivo”, while answering the web survey questions. It also provides the opportunity to integrate such qualitative evidence with quantitative data: survey question responses, demographics and paradata, etc. So far, the dominant approach in web probing is to assess each question separately. In our presentation, however, we will present preliminary results of an experiment aimed to compare two web probes: a “single” category selection probe versus an “integrated” category selection probe. “Integrated” means participants are asked to explain why they have selected two particular response options for two content-related and consecutive survey questions by responding to one open-ended question. By comparing evidence from a single but “integrated” category selection probe with those from two probes, we expect to find out whether participants respond to the two content-related and consecutive survey questions with the same interpretation pattern of the key concepts in both survey questions, and by applying the same judgmental process. 1114 web panel participants, 559 from Germany and 555 from Spain, were randomly assigned to the two experimental conditions: “integrated” category selection probe versus “single” category selection probes. Both subsamples responded to two target survey questions on climate change extracted from the 8th European Social Survey Round (D22: Do you think that climate change is caused by natural processes, human activities, or both?; D23: To what extent do you feel a personal responsibility to try to reduce climate change?). In this presentation, we will focus on comparing data quality for both kinds of probes and indicators of respondent’s motivation and productivity by following the analytic procedures proposed by Meitinger, Braun, and Behr (2018). In addition, we will outline the potential of web probing to an “in vivo” validation of survey responses. Finally, we will also discuss on how to integrate qualitative evidence from web probing with quantitative data within a mixed methods framework.
Professor Aviad Tur-Sinai (The Max Stern Yezreel Valley College) - Presenting Author
Professor Perla Werner (University of Haifa)
COVID-19, caused by the novel coronavirus, is a respiratory disease with clinical symptoms including fever, dry cough, fatigue, muscle pain, and shortness of breath. On March 11, 2020, the World Health Organization declared it a global pandemic, in which Israel is included. Shortly afterward, Israel imposed restrictions on movement and economic activity. Israel’s lockdown lasted nearly two months, and at its peak people were confined to their homes and violators were fined.
The study aimed to assess the prevalence and correlates of forgoing recourse to a variety of healthcare services during a two-month COVID-19 lockdown, using Andersen's Behavioral Model of Healthcare Utilization.
By means of a survey among more than 300 Israeli Jews aged 40+ countrywide, it is found that, on average, one of two people who needed to visit a family doctor, a specialist, a paramedic, or a hospital outpatient clinic during the research period did not go ahead with the visit. Nearly one-third of survey respondents who needed to visit a family doctor at this time used virtual services offered by their HMOs (e.g., telephone or video conference with doctors). A smaller share of respondents reported using virtual services as a substitute for visiting a specialist, visiting a paramedical professional, or consulting a doctor at a hospital outpatient clinic. Fear of COVID-19 infection raised the probability of forgoing healthcare services, with adults with diabetes more inclined than others to forgo visits to doctors and specialists during the coronavirus’s lockdown period. The likelihood of consuming traditional healthcare services was lower among those aged 60+ and higher among those with higher income; in contrast, the probability of consuming virtual healthcare services was higher among those with hypertension but lower among those aged 70+, traditional and religious respondents, and those with diabetes.