Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Tuesday 14th July, 14:00 - 15:30 Room: HT-104

Direction of Response Scales
Convenor	Dr Florian Keusch (University of Michigan )
Coordinator 1	Professor Ting Yan (University of Michigan)

Session Details

The measurement of many constructs in social and marketing research, such as attitudes, opinions, behaviors, personality traits, and personal states, heavily relies on the use of response scales. Survey literature has demonstrated that many design features of response scales (e.g., number of scale points, numeric and verbal labels, spacing of response options, alignment) affect how survey respondents process the scale and use these features to construct their responses. A response scale could run from the positive to the negative pole (e.g., “strongly agree” to “strongly disagree”) or the highest to the lowest point (e.g., “all of the time” to “never”). The same scale could also run from the negative to the positive pole (e.g., “strongly disagree” to “strongly agree”) or the lowest to the highest point (e.g., “never” to “all of the time”). An important question is then whether or not the direction of a response scale affects survey answers, holding constant the other features of the scale.
This session invites presentations that investigate the influence of scale direction on survey responses. We particularly invite presentations that analyze the influence of scale direction (1) under different modes of data collection, especially emerging modes, such as mobile Web and SMS/texting, (2) considering moderating effects of scale- and question-level characteristics, such as number of scale points, scale alignment, and question content, and (3) in a cross-cultural context.

Paper Details

1. Response scales: Effects of scale length and direction on reported political attitudes
Ms Marta Kolczynska (The Ohio State University, Polish Academy of Sciences)

The diversity in types of response scales in survey items on political attitudes raises questions about the inter-survey comparability. In this paper I present results of a survey experiment with response scales applied in question-items on trust in political institutions: the national parliament, political parties, and justice system, focusing on two features: scale length (number of scale points) and direction (negative to positive and vice versa). Exploration of these effects and errors contributes to the development of survey theory, as well as facilitates analysis based on combined (harmonized) survey data.

2. Does Satisficing Drive Scale Direction Effects?
Dr Florian Keusch (University of Mannheim)
Dr Ting Yan (Westat)

Previous research found that different scale directions do affect response distributions. This effect is usually attributed to scale direction as another case of response order effect caused by satisficing. Drawing on data from two individual experiments with members of a nonprobability online panel and students, we found that different scale direction has a significant and extreme impact on the mean ratings. However, respondents who were more prone to satisficing did not necessarily show stronger scale direction effects. Scale direction effects are observed across the board among those who are prone to satisficing and those who are not.

3. Ordering Your Attention: Response Order Effects in Web-based Surveys
Dr Frances Barlas (GfK Custom Research)
Mr Randall Thomas (GfK Custom Research)

Our study examined response order effects across a number of online surveys with over 500,000 total respondents, examining the impact how the number of responses, dimension assessed, and response orientation. We generally found that vertically displayed scales showed higher levels of primacy effects, and those scales measuring evaluative aspects were more likely to show the primacy effect than intensity aspects.

4. Impact of response scale direction on survey responses in web and mobile web surveys
Dr Ting Yan (Westat)
Dr Florian Keusch (University of Manheim)

Scale direction effect is understudied. An important gap in the literature is the lack of attention to the moderating effects of scale- and question-level characteristics on scale direction effects. To fill this gap, we conducted an experiment varying question type, scale direction, scale length, and scale labeling. This study is the first to experimentally manipulate scale features. It will allow us to tease apart the moderating effects of scale and question feature on scale direction effects and to provide guidance on direction of scales.

5. The importance of scale direction between different modes
Ms Vilma Agalioti-sgompou (ISER, University of Essex, UK)

This study aims at examining whether the effect of the direction of the response scale (from negative to positive or from positive to negative) varies between modes (face-to-face and internet). Using data from the American National Election Study, we answer this research question focusing on attitudinal items with a 5-point scale. We find that depending on the mode the direction of the scale impacts on response. Moreover, we report findings that take into account respondent's background and we conclude with a discussion on the implications of the findings on survey design.