Methodological and Practical Challenges to Measuring Political Ideology and Preferences |
|
| Session Organiser |
Ms Kelsey Starr (Pew Research Center) |
| Time | Thursday 18th July, 16:00 - 17:30 |
| Room | D17 |
Recent trends in Europe, such as terrorist threats, economic uncertainty and large-scale immigration, have drastically shifted the overall political landscape. For many, the traditional understanding of left- and right-wing ideologies do not hold as much merit as “populist” or “mainstream” ideologies – terms whose definitions are new at best and inconsistent at worst. While some of the most prominent examples of European populism have been connected to conservative policies, such as the Brexit vote and the success of the German AfD party, populism is not only connected to conservatism. For example, Podemos in Spain (a left-wing party) received 21% of the national vote in 2016.
Public opinion research consistently uses political affiliation as both an important analytical variable and as a statistical control. But with rising dissatisfaction with the status quo on both the political left and right, currently used measures of political affiliation and ideology run the risk of holding less statistical power overall. How can researchers best ask about political preferences while also tapping into populist ideologies that can transcend political ones?
The purpose of this panel is to explore recent methodological challenges, experiments and innovations about how to best ask respondents about their political ideologies. The panel incorporates ideas about political parties, the rise in populist ideologies, voting behavior and opinions about the current political climate.
The panel accepts papers that focus on questionnaire development for survey-based research, including survey experiments, cognitive interviews and large-scale projects. Paper topics include, but are not limited to:
• Best practices for understanding political party preference and political ideology
• Cross sections between mainstream political ideology and populist sentiment
• Decreasing item nonresponse
The focus of this panel is not the substantive findings but rather the methodological practices used to achieve them.
Keywords: Politics, questionnaire development, measurement issues
Mr Jakob Horneber (Bonn University)
Dr Alice Barth (Bonn University) - Presenting Author
The localization of political preferences on an axis between “left“ and “right“ positions is a popular tool to measure respondents’ political ideology in Western European countries. The differentiation between left and right is seen as a precise, yet easily comprehensible and widely applicable instrument to describe political affiliations (e.g. Potter 2001). Despite the prevalence of the left-right-scale, a collectively shared meaning of “left” and “right” for all respondents is usually rather assumed than tested.
Research has shown that the meaning of “left” and “right” varies both cross-temporally (Bauer-Kaase 2001; Franzmann 2009) and cross-culturally (Fuchs/Klingemann 1989; Tavits/Letki 2009; Piurku et al. 2011), compromising the comparability of scale values. Moreover, associations with “left” and “right” are different according to one’s self-placement on the scale (Bauer et al. 2017) and the meaning of the axis’ poles depends on respondents’ political priorities (Weber 2012).
In the German General Social Survey (ALLBUS) in 2008, respondents were asked about their associations with the terms “left” and “right” in two open-ended questions. Using a modified version of a comprehensive coding scheme for these data (Scholz & Züll 2012), we assess the interrelations between different associations with “left” and “right” using correspondence analysis. This technique allows for a visual representation of the most important dimensions in categorical data. Thus, structural differences in the meaning attached to “left” and “right” can be demonstrated. Further, we analyze how different ideological schemes are related to socio-structural characteristics.
We conclude with methodological implications for the use of left-right-scales in survey research and discuss possible alternatives for a comprehensive understanding and measurement of political ideology.
Ms Jule Adriaans (German Institute for Economic Research - DIW Berlin) - Presenting Author
Professor Stefan Liebig (German Institute for Economic Research - DIW Berlin)
Professor Guillermina Jasso (New York University)
Professor Clara Sabbagh (University of Haifa)
The rise of social inequality is considered one of the most pressing societal issues in Europe. Over the past few decades widening gaps in income and wealth have been observed. With these widening inequalities, the question of how goods and burdens should be allocated within a society has become an important aspect of political ideology that needs to be addressed by empirical research. More specifically, public opinion research needs to address the issue of social justice. Despite its long-standing tradition of asking respondents about their sense of justice, empirical justice research suffers from remarkable inconsistencies in the terminology used to query the justice or injustice of allocation procedures and their outcomes. Up to date, the terms “just”, “fair” and “appropriate” have been used interchangeably in querying justice attitudes, despite the fact that theoretical considerations suggest that different justice terms are associated with different aspects of justice. We provide a critically assessment of the current practice in empirical justice research, and use evidence from a wording experiment conducted in the German GESIS-Panel to investigate whether different justice stimuli lead to different evaluations. We further investigate, if and how the effect of justice evaluations on known consequences of experienced injustice depends on the justice term used in the question wording. Preliminary results reveal that the choice of justice stimulus in the question wording does indeed influence response behavior. We therefore contribute to the empirical investigation of political ideologies by critically assessing the common practice in the measurement of justice attitudes and sharpening the understanding of the way people think and talk about justice matters.
Mr Matous Pilnacek (Institute of Sociology of the Czech Academy of Sciences) - Presenting Author
Mrs Paulina Tabery (Institute of Sociology of the Czech Academy of Sciences)
Research problem: Past vote weighting is a widespread practice among polling companies in many European countries and also in the Czech Republic. Weighting samples on the basis of past votes is used to improve the estimates of voting intentions and in likely voter models it affects the results for individual political parties. However, the variable used for data weighting is not entirely reliable and, as noted in the literature, there are several effects that may come into play, such as forgetting, misremembering, or correcting past behaviour to match the current preference and social desirability (e.g. Durand, Deslauriers & Valois 2015). In our research on the elections to the Chamber of Deputies of the Parliament of the Czech Republic in October 2017, we show the impact of past vote weighting in the context of other adjustments, most of which are usually made in the likely voter models.
Data and Methodology: The data comes from the October pre-election survey that is part of the ‘Our Society’ research project implemented by the Czech Public Opinion Research Centre (CVVM). This is a long-term omnibus survey carried out once a month on a sample of approximately 1,000 respondents. Each sample, selected using the quota procedure, is considered representative of the population of the Czech Republic aged 15 years and over. Data collection is performed in face-to-face interviews with a standardised omnibus-type questionnaire.
Objectives and results: This paper will examine if past vote weighting makes any significant difference to the estimates of political parties' shares in a likely voter model and how important this difference is in the context of other adjustments, such as adding weights to sociodemographic variables and the reallocation of undecided voters. We will also discuss how to deal with the aforementioned effects that may occur in the process of recalling past voting behaviour.
Dr Sebastian Stier (GESIS Leibniz-Institute for Social Sciences)
Dr Johannes Breuer (GESIS Leibniz-Institute for Social Sciences) - Presenting Author
Dr Pascal Siegers (GESIS Leibniz-Institute for Social Sciences)
Dr Tobias Gummer (GESIS Leibniz-Institute for Social Sciences)
Dr Arnim Bleier (GESIS Leibniz-Institute for Social Sciences)
Internet use has become deeply ingrained in political life. People use the internet to get political information, directly follow political actors, discuss politics with friends, advocate for political causes or mobilize for offline campaigns. On the other hand, the spread of online news consumption has recently been blamed for strengthening the growth of populist movements (Müller and Schwarz 2018) because information on news websites and social media networks are often not verified, thus, favoring the proliferation of misinformation and extremist content. Adding to this, populist movements accuse mainstream media of supporting political elites against the presumed interests of the population.
Our paper addresses the question of whether online news exposure of individuals supporting right-wing populist parties (RWP) differs from supporters of non-populist parties. We assume that RWP supporters prefer social media networks and alternative news websites to established media outlets.
We use a dataset that links data from a one-year web tracking (June 2018 to May 2019) from 1,300 German study participants recruited from a non-probabilistic online access panel to an online survey administered in August 2018. In the survey we asked about media use, political attitudes, and basic sociodemographics.
The web tracking provides an innovative way to measure actual online news consumption, avoiding the recall and social desirability biases that have been found for self-reports of media use (Guess 2015, Prior 2009, Scharkow 2016). From a methodological perspective, the paper demonstrates how linking digital trace data to survey data can improve research on online news exposure.
Results show that a preference for a RWP is associated with a lower intensity of online news consumption for established media outlets (so-called hard news, Baum 2002, Bakshy et al. 2015), even when controlling for political interest, ideology, sociodemographics, and overall internet use.