New measurement capabilities of mobile devices |
|
| Chair | Dr Peter Lugtig (Utrecht University ) |
| Coordinator 1 | Dr Vera Toepoel (Utrecht University) |
| Coordinator 2 | Miss Anne Elevelt (Utrecht University) |
| Coordinator 3 | Mr Randall Thomas (GfK Custom Research) |
| Coordinator 4 | Dr Frances Barlas (GfK Custom Research) |
Grid questions are used routinely in online surveys when researchers want to assess multiple items using the same response format. Grids typically present elements to be evaluated in rows with responses arrayed in the columns. Grids can be problematic with smaller screen sizes such as those of smartphones. Horizontal (left-right) scrolling is often required to view all response options, making it impossible to view both the item text and all response options at the same time and raising the possibility that respondents will be less likely to select response options not visible on the screen. In the accordion grid format, respondents see the item text listed vertically, then can click on each item to reveal the response scale in a standard single response format. Using this approach, respondents can see the entire response scale for each item, along with the response items on a single screen. However, vertical response presentation has been associated with an increased likelihood of response order effects. Is the accordion grid an effective alternative to the traditional grid format? We conducted two experiments each using over 1,000 cases from KnowledgePanel®, GfK’s U.S. probability-based online panel. With each study, respondents were randomly assigned to complete either an accordion or traditional grid. The time it took for respondents to complete each grid type was comparable, and we found few differences in results across the two types of grids for both dichotomous grids and unipolar scales with three or more response options. We found similar concurrent validity for both grid types when correlating the results with behavioral data and little evidence of response order effects. Respondents provided similar ratings of the ease and accuracy with which they could respond to the traditional and accordion grid scales. Taken together, the results suggest that the accordion grid is a promising alternative to the traditional grid design, especially given the increase in the proportion of online survey respondents who complete surveys using smartphones and other mobile devices.
Mobile devices have smaller screens which make survey response selection more difficult, especially on smartphones. In self-administered surveys, grids have evolved to efficiently measure multiple concepts using the same graded response format across a number of different targets, with responses typically in columns and the rating targets to evaluate in rows. Generally, grids allow us to more efficiently assess multiple targets compared to presenting each target is presented on separate screens. The response format that is chosen can vary in terms of semantic labeling - every response can have an associated semantic label (fully labeled) or only the end categories have a semantic label (end labeled). The response format could represent a bipolar scale (ranging from a concept and its antithesis, e.g. like – dislike) or a bipolar scale (with a concept and its absence, e.g. like – do not like). Finally, the response format can vary in terms of the number of response categories used. We summarize 4 studies comparing the use of end-anchored and fully-anchored categories with both unipolar and bipolar scales, holding the number of categories constant. The unipolar fully labeled variant generally appeared to be better as a measure across topics, with better differentiation and validity. Since online surveys are increasingly being taken on mobile devices, the utility of a grid is becoming affected by screen size – grids with more responses are much harder to implement on smartphones. In a series of 2 additional studies we examined how reducing the number of grid response categories affects the measurement of a variety of concepts. We found that response formats with fewer response categories take less time to complete, are easier to complete on mobile devices, and shorter scales show as much validity as formats with more response categories, especially when using the unipolar variant. We examine the ability of the scales to detect smaller differences between rating targets and found some improved differentiation with scales with fewer responses rather than those with more responses.