New Communication Channels in Web-based Surveys |
|
| Session Organisers |
Dr Jan Karem Höhne (University of Duisburg-Essen) Mr Kostantin Gavras (University of Mannheim) Professor Melanie Revilla (RECSM-Universitat Pompeu Fabra) |
| Time | Friday 2 July, 15:00 - 16:30 |
Recent advancements in communication technology and digital data collection, coupled with a continuous increase of mobile device use, particularly smartphones, facilitate new communication channels in web surveys. For instance, smartphones support question reading through pre-recorded videos and answer recording through the microphone, potentially simulating interviewer-based surveys. They also support taking photos in the moment or uploading photos from the gallery to answer questions in a more objective way. Furthermore, telecommunication applications providing video and voice chat support interviewer-administered web surveys. Growing and powerful opportunities in data transformation, manipulation, and analysis, such as Automatic Speech Recognition and Text-as-Data Methods, facilitate a proper data handling, even from large-scale web surveys. Applying new communication channels in web surveys has not only the potential to extend the existing methodological toolkit, but to collect new data that provide more objective and in-depth information about respondents’ attitudes and behaviors.
Keywords: communication channels, digital data collection, mobile devices, new data sources, web-based surveys
Mrs Patricia Iglesias (RECSM-Universitat Pompeu Fabra) - Presenting Author
Professor Melanie Revilla (RECSM-Universitat Pompeu Fabra)
A significant share of survey questions asks people to recall events or daily life-related details, such as income, the amount of a bill, or what they ate in their most recent restaurant meal. This way, researchers are relying exclusively on people’s ability to recall events and provide proper answers, although previous evidence has shown that human memory has limitations that can eventually compromise the quality of the responses. Furthermore, respondents can be asked to answer questions on topics that they are not aware of (such as recognizing specific insects or plants, the specific items that compose their income, the building materials of their dwelling, etc.), leading to inaccurate answers or item nonresponse.
The increasing use of smartphones to answer web surveys presents several opportunities given the specific sensors included in this type of devices that could help overcome the limitations of respondents’ memory and awareness, by replacing or complementing survey questions. In this presentation, we will focus on the camera and gallery of images included in mobile devices. Particularly, the center will be placed in the sending of images by the respondents, as a tool to assist the collection of participants' behaviors, events and characteristics. Asking for visual data such as images could enrich data and help obtaining more accurate information, while also reducing respondents’ burden. Given the possibility that images can be used, the preciseness of the data provided could increase. However, there are also some possible issues related to the unwillingness to share images, the lack of awareness on how to do it, or the unavailability of pictures that could be detrimental for the implementation of this collecting tool.
In this presentation, we will provide insights about when it makes sense to ask for images to (mobile) web survey respondents, based on information such as the type of images that they have in their mobile devices and cloud storage, their willingness to share such pictures, the topics that they are more prone to share, and their abilities to share and upload images.
The data will be collected in 2021 through an online opt-in panel in Spain, targeting adults who have access to the internet, use it, and speak the country’s main language. The methods will include mainly descriptive analysis to characterize and compare the prevalence of the variables in the sample. We expect significant differences across respondents in their abilities and willingness to share images within the frame of web surveys. Also, we expect the availability of images to vary largely depending on the topic of interest. Thus, for some topics, availability could be one of the main causes of nonresponse.
Overall, the results will provide hints on how and when it seems worth it to ask for images within the frame of (mobile) web surveys. This information can be used by researchers to decide in their studies whether asking for images in some questions could improve their data quality and, consequently, help getting more accurate conclusions.
Professor Katharina Meitinger (Utrecht University) - Presenting Author
Professor Matthias Schonlau (University of Waterloo)
Technological advancements now allow exploring the potential of voice recordings for open-ended questions in smartphone surveys (e.g., Revilla & Couper 2019). Voice-recordings may also be useful for web surveys covering the general population. It is unclear whether and which respondents show a preference to provide voice-recordings and which respondents prefer to type responses to open-ended questions.
In this presentation, we report on an experiment that was implemented in the LISS panel in December 2020. Respondents in this experiment were randomly assigned to a voice-recording only, a text-recording only, or a group in which they could select between voice and text recording. In this presentation, we will report who shows preferences for voice-recordings and which factors influence these preferences (e.g., perception of anonymity of data, potential by-standers during data collection).
Revilla, M., & Couper, M. P. (2019). Improving the Use of Voice Recording in a Smartphone Survey. Social Science Computer Review, 0894439319888708.
Dr Christoph Kern (University of Mannheim) - Presenting Author
Dr Jan Karem Höhne (University of Duisburg-Essen)
Mr Konstantin Gavras (University of Mannheim)
Mr Stephan Schlosser (University of Göttingen)
Technological developments for surveys on mobile devices, particularly smartphones, offer a variety of new forms and formats for collecting data for social science research. A recent example is the collection of voice answers via the built-in microphone of smartphones instead of text answers via the keyboard when asking open questions. Voice answer formats, compared to text answers, may decrease respondent burden by being less time-consuming and may provide richer data. Furthermore, advances in natural language processing and voice recognition do not only facilitate to process and analyze voice data, but also allow to utilize tonal cues, such as voice pitch, that can be used to study new research questions. Specifically, we explore the usage of pre-trained emotion recognition models for predicting the emotional states of respondents based on voice answers.
In this study, we use data from a smartphone survey (N = 1,200) that was collected in Germany in December 2019 and January 2020. To collect respondents’ voice answers, we developed the JavaScript- and PHP-based “SurveyVoice (SVoice)” tool that records voice answers via the microphone of smartphones. We make use of the openEAR toolkit (Eyben et al. 2009) which allows us to predict respondents’ emotional states based on voice data using pre-trained NLP models for emotion recognition. On this basis, we analyze the effects of (predicted) emotions on survey responses with respect to both content (e.g., sentiment of response) and quality (e.g., number of words, lexical complexity). This study exemplifies how to utilize new meta-information that can be extracted from voice data and adds to the research on the merits and limits of collecting voice answers in smartphone surveys.
Eyben, F., Wöllmer, M., and Schuller, B. (2009). OpenEAR – Introducing the Munich Open-Source Emotion and Affect Recognition Toolkit. Paper presented at the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, 2009, 1-6.
Professor Frederick Conrad (University of Michigan) - Presenting Author
Professor Michael Schober (The New School)
Mr Andrew Hupp (University of Michigan)
Professor Brady West (University of Michigan)
Mrs Kallan Larsen (University of Michigan)
Mrs Ai Rene Ong (University of Michigan)
Mr Tianheao Wang (University of Michigan)
Recorded video that users watch on televisions, computers, smartphones, and other devices has become a fixture of the environment, from television screens in public places to online instructional videos to personal videos recorded and posted by smartphone users. The ubiquity of this technology makes its potential benefits and costs for collecting survey data a natural topic of inquiry for methodologists. It has some of the properties of a live interview, e.g., a moving, talking interviewer, but is self-administered in that the recorded interviewer is clearly inanimate and insensate. If a video recorded interviewer asks a question, is this free of the social presence that might be observed if a live interviewer asked the same question? For example, does a video recorded interviewer inhibit disclosure of sensitive information compared to a conventional, textual web survey, as an in-person interviewer might? Or does a video recorded interviewer create a sense of rapport with respondents, despite the absence of responsivity by the recorded interviewer, potentially improving data quality relative to web surveys? We report a study that compares data quality (number of rounded numerical responses, straightlining, and disclosing sensitive information) and respondent experience (satisfaction with the interview) in online surveys in which prerecorded videos of interviewers (n = 9) ask questions and respondents (n=385) type or click to enter answers, and conventional text-based web surveys in which respondents (n=403) read questions and type or click to enter answers. We included an additional mode, live video interviews (n=283), to help disentangle some patterns of responses. Prerecorded video produced higher quality data for two of three measures (less rounding and more disclosure) than in the textual web survey. There was no difference in the amount of straightlining in the two modes and respondents were relatively and equally satisfied in both (4.1 and 4.2 on a 5-point scale, respectively). The reduced rounding may be the result of differences in our design and implementation of the prerecorded video and textual web survey modes: in the former, respondents were able to answer only after each video-recorded question had been fully played, potentially giving respondents time to recall and count instances of the behaviors about which they were questioned, a strategy that would lead to more unrounded answers. The increased disclosure in prerecorded video “interviews'' may reflect a greater sense of accountability by respondents and perhaps some “rapport” with the recorded interviewer. However, respondents’ experience with prerecorded video interviewers clearly differed from that with live video interviewers: respondents reported being less connected to the interviewer in prerecorded than live video interviews (2.93 versus 3.57 on a 5-point scale, respectively) and less comfortable with the recorded than live interviewer (3.9 vs. 4.2 on a 5-point scale). The hybrid character of prerecorded video interviews produced hybrid results; for some survey topics and some sample sources this mode has the potential to lead to an optimal set of outcomes.
Professor Barry Schouten (Statistics Netherlands) - Presenting Author
Dr Annemieke Luiten (Statistics Netherlands)
Professor Peter Lugtig (Utrecht University)
Professor Vera Toepoel (Utrecht University)
Sensor-assisted surveys face an important question in designing and analyzing the combination of survey questions and sensor measurements: To what extent should respondents actively be involved in sensor data collection? One motive for employing sensor data in surveys may be to reduce burden for the respondent. Another motive may be to improve measurement quality of data. However, sensor data are subject to inaccuracy as well. They may have gaps and they may have both systematic and random measurement error. The sensor data quality may itself depend on the motivation and skills of respondents, if they are to initiate the measurements. This leads to a trade-off between data quality and respondent burden.
In the paper, three case studies are discussed where the extent of active respondent involvement plays a prominent role: Travel mode and travel purpose prediction in travel surveys, receipt scan classification in household budget surveys and physical activity tracking in health surveys.