The Power and Pitfalls of Combining Survey and Sensor Data 2 |
|
| Session Organisers | Miss Anne Elevelt (Utrecht University) Dr Peter Lugtig (Utrecht University) Dr Vera Toepoel (Utrecht University) |
| Time | Wednesday 17th July, 16:30 - 17:30 |
| Room | D19 |
Sensor data offer great potential for social scientists interested in studying attitudes and behaviors. These kind of data are particularly interesting when they can be linked to and compared with other data sources. With more and more possibilities to collect additional data through smartphones (for example through smartphone apps or activity trackers) large-scale population surveys could rather easily be enriched. Participants carry their smartphone everywhere, enabling scientists to ask respondents to make pictures, or to collect GPS and accelerometer data and track for example how much participants move around and where they go. Opportunities abound.
However, there are still many unsolved and unique methodological questions and issues to collecting and using sensor data. This session invites presentations that investigate the potentials and challenges when combining survey and sensor data. We especially welcome papers that used and collected these kind of data, and address;
The power of sensor data
o Higher data quality?
o Lower respondent burden.
The pitfalls of sensor data
o Implementation issues; nonresponse, willingness, device use.
o Technical problems.
o Issues in collecting and accessing these data across the general population.
o Data storage.
Keywords: Big data; sensor data
Dr Nejc Berzelak (University of Ljubljana, Faculty of Social Sciences) - Presenting Author
Mr Uroš Podkrižnik (University of Ljubljana, Faculty of Social Sciences)
Ms Jasna Urbančič (Artificial Intelligence Laboratory, Jožef Stefan Institute)
Mr Matej Senožetnik (Artificial Intelligence Laboratory, Jožef Stefan Institute)
Professor Vasja Vehovar (University of Ljubljana, Faculty of Social Sciences)
Modern smartphones incorporate sensors for collecting data about position, orientation, motion and environment. Previous studies have demonstrated how passively collected location and motion data can be effectively used in social science research. However, there is a lack of a detailed elaboration of such data collection from the perspective of social science methodology and its integrative placement among survey data collection methods. This extends to appropriate consideration of errors arising from sensor data in the context of their integration with survey data. While error sources have been comprehensively elaborated for survey research, particularly by the Total Survey Error framework, systematic efforts to accomplish a consistent conceptualisation for complementary sensor data remain limited.
This paper contributes a critical elaboration of specific error sources in data collection using smartphone sensors to complement survey data. It focuses predominantly on technical aspects that may have important methodological implications for social science research and addresses three main research questions:
1) How can technical characteristics of smartphones and behaviour of research participants in interacting with their devices affect the quality of data relevant for social science research?
2) How can these error sources be placed into the conceptual framework of the Total Survey Error?
3) How can device paradata contribute to better understanding of potential influences of these factors on the data quality?
The elaboration applies the findings from studies in various fields onto the context of survey research and further highlights potential error sources by evaluating a prototype mobile application for integrated collection of survey and sensor data. On this basis, the paper identifies technological and behavioural factors that can affect the data collection performance, discusses the placement of potential biasing effects into the Total Survey Error framework and underlines the importance of implementing appropriate measures for better understanding and monitoring of the technical environment during the data collection.
Miss Anne Elevelt (Utrecht University) - Presenting Author
Dr Jan Karem Höhne (University of Mannheim; RECSM-Universitat Pompeu Fabra)
Professor Annelies Blom (University of Mannheim)
Smartphones are becoming increasingly important and widely-used in survey completion. Smartphones also offer many new possibilities for survey research, such as extending data collection by using sensor data (e.g., acceleration). Sensor data, for instance, can be used as a more objective supplement to health and physical fitness measures in mobile web surveys. In this study, we therefore investigate respondents’ willingness to participate in fitness tasks during mobile web survey completion. In addition, we investigate the appropriateness of acceleration data to draw conclusions about respondents’ health and fitness level. For this purpose, we use “SurveyMotion (SM),” a JavaScript-based tool for smartphones to gather the acceleration of smartphones during survey completion and additionally employ traditional health and physical fitness measures. We ask respondents if they would generally be willing to take part in a fitness task during mobile web survey completion and employ a subsequent fitness task in which we ask respondents to do squats (knee bends) for one minute. Thus, we investigate respondents’ hypothetical as well as actual willingness and the general comparability of acceleration data with established health and physical fitness measures. We conduct an observational study by using a German nonprobability-based web panel with n = 1,500 respondents; the data collection takes place in September 2018. This study contributes to the development of more objective measures of respondents’ health and fitness in mobile web surveys and could be extended by further physical activity tasks in future research.
Dr Christoph Kern (University of Mannheim) - Presenting Author
Mr Stephan Schlosser (University of Göttingen)
Dr Jan Karem Höhne (University of Mannheim)
Dr Melanie Audrey Revilla (Universitat Pompeu Fabra)
Participation in web surveys via smartphones increased continuously in recent years due to a skyrocketing proportion of smartphone owners and an increase in mobile Internet access. However, previous research has shown that smartphone respondents are frequently distracted and/or multitasking, which might affect response behavior in a negative way. In this study, we therefore predict respondents’ completion conditions (e.g., standing or walking) and study their effects on data quality in mobile web surveys. For this purpose, we train machine learning models based on acceleration data of smartphone respondents – measured by means of the JavaScript-based tool “SurveyMotion” – that were collected in a lab experiment (N = 89) and in a field experiment (N = 521) that systematically varied the completion conditions. The lab experiment data were collected at the Center of Methods in Social Sciences at the University of Göttingen (Germany) in 2017 and the field experiment data were collected by the online fieldwork company Netquest (Spain) in 2018. We extract features from the acceleration data by aggregating over the repeated acceleration measurements that were collected for each respondent-page. Regularized regression and tree-based models were trained and tested using grouped cross-validation, reflecting the hierarchical structure of the data with pages nested in respondents. When building the prediction models, both a multiclass (sitting, standing, walking, climbing stairs) and a binary (moving, not moving) version of the outcome variable were considered. It becomes evident that the acceleration features can be used to build sparse prediction models that almost perfectly discriminate between completion conditions on hold-out sets, with cross-validated ROC-AUCs between 0.981 and 0.998. The evaluation results indicate that the trained prediction models can be used to precisely predict completion conditions in (new) mobile web surveys that collect acceleration data.