All time references are in CEST
Survey Nonresponse Trends and Trust in Surveys: A global perspective on the current survey climate 1 |
|
| Session Organisers | Professor Bettina Langfeldt (University of Kassel) Mrs Ulrike Schwabe (DZHW) Dr Henning Silber (GESIS) |
| Time | Tuesday 18 July, 11:00 - 12:30 |
| Room | U6-06 |
In our complex and interconnected world, there is a strong need for databased scientific approaches to solving diverse local as well as global problems. However, to transform scientific recommendations successfully into policy measures, societal trust in scientific methods and results is required. Yet, mistrust toward scientific results seems to be on the rise in recent years.
In a democracy, surveys can be an important tool for measuring public opinion and informing political decision-makers about the views of their constituents. Yet, decreasing survey participation, attempts to manipulate polls, and misleading accusations of “fake polls” as well as polls carried out not in accordance with established scientific standards, put the validity of the gathered data in jeopardy. If the survey climate continues to be on the decline, this will have drastic consequences for survey-based research since both policymakers as well as the recipients of political interventions have to believe in the accuracy of the data.
Against this background, this session aims to bring together current research on the following topics:
Survey Climate: What is the current state of the survey climate? What caused changes? What can be done to foster a positive survey climate, both in terms of increasing survey participation and increasing the quality of survey data?
Trust in Surveys: What can be done to increase trust in surveys and their results? Which are determinants of participation and giving truthful answers that can be used to strengthen the quality of survey data? How is declining trust in surveys related to the declining trust in science?
Surveys and Society: If and how is political participation linked to trust in science and the generalized attitudes towards surveys? What is the role of surveys in the democratic process? Can process-based data (e.g., digital trace data) help to mitigate the survey
Ms Imke Herold (MEA-SHARE and SHARE Berlin Institute) - Presenting Author
Dr Arne Bethmann (MEA-SHARE and SHARE Berlin Institute)
Dr Michael Bergmann (MEA-SHARE and SHARE Berlin Institute)
Like most surveys, the Survey of Health, Ageing and Retirement in Europe (SHARE) suffers from a considerable amount of item non-response, particularly regarding income questions as well as respondents’ reluctance to consent to record linkage. Both aspects have a potentially severe impact on data quality leading to less precise as well as biased estimates and thus might hamper substantive analyses.
While SHARE tries to alleviate problems due to item non-response, e.g. through income imputations, it seems especially important to understand the underlying reasons that keep respondents from reporting their income or consenting to record linkage. In order to add to the available body of research from the SHARE perspective with its specific sample of respondents 50+ and its study focus, we used the opportunity to include an additional paper-and-pencil drop-off questionnaire with the German sub-study, focusing on issues like trust in surveys and organizations, data privacy concerns as well as attitudes towards income questions.
Using data from the German SHARE Wave 8 and Wave 9 drop-off questionnaire, we want to contribute to the question whether several aspects of trust are interrelated with income non-response on the one hand and denial of linkage consent on the other hand. By choosing income questions – as one of the classics of item non-response – and consent to record linkage – as a more innovative survey tool – we try to cover a broad range of aspects that can be relevant for respondents’ trust. Analyses based on these specific SHARE waves allow us to compare the survey climate before the onset of the pandemic with the time after. Furthermore, we are able to investigate consequences of income item non-response on potential panel dropout in subsequent waves.
Mr Jon Cohen (Primera Capital)
Ms Laura Wronski (SurveyMonkey) - Presenting Author
Early results indicate that the polls conducted prior to the 2022 midterm elections might be the most accurate in the past decade, and yet the American public continues to be skeptical of their accuracy and the accuracy of the polling field in general. This paper will use the results of three sets of polls conducted in the US, the UK, and Canada. In the US, the first poll was fielded the month prior to the 2022 midterm election, and the other in early 2023—both fielded using SurveyMonkey’s proprietary river sample. In the UK and Canada, both surveys were fielded in early 2023. These surveys ask about the role of polling in society today, probing respondents on how much they read/hear about polls in the news, what characteristics of a poll make it more trustworthy, and how confident they are in their ability to distinguish a high quality from a low quality poll. We also compare polling with other means of collecting data from individuals (e.g. focus groups, town halls). We will analyze the results by various subgroups to understand how pollsters can build trust in their work among those groups who are the most skeptical, and will create an international profile of what characteristics those poll-hesitant individuals share. We will also compare the trend over time, from October to December, to see how engagement with and trust of polls is affected by a high-stakes election.
Dr Eli Feiman (United States Department of State)
Ms Dina Smeltz (Chicago Council on Global Affairs) - Presenting Author
A crackdown on protests and free speech in Russia since February 2022 have undermined trust in the reliability of public opinion data about the Ukraine war and other key attitudes. Pollsters and journalists can only describe the current conflict in Ukraine as Russia’s “special military operations,” prompting skeptics to question the practical value of survey results when a climate of fear could be suppressing participation in public opinion surveys among critics of the war. At the same time, public opinion research by both Russian and international organizations continues apace, showing trends consistent across sources and resonant with changes following Russia's 2014 annexation of Crimea. This paper addresses key challenges to collecting and understanding public opinion data from Russia at this critical moment. It outlines the changes as well of continuities in the field of public opinion research in Russia before and after February 2022, assesses Russian polling organizations’ current tests for reliability, and offers some new approaches for future consideration. Recognizing both the challenge and importance of measuring attitudes in the current climate, this paper situates today’s Russia in broader literatures related to public opinion in authoritarian contexts and wartime. Leveraging multiple sources of data as well as the authors' practical experience in the field, this paper lays out a framework for researchers and policymakers for interpreting and building trust in research on Russian public opinion.
Professor jibum kim (Sungkyunkwan University) - Presenting Author
Professor Jae-Mahn Shim (Korea University)
Dr Sori Kim (Gallup Korea)
Dr Tom W. Smith (NORC at the University of Chicago)
Confidence in leaders in polling institutions is vital for the proper function of polling. Using the 2021 Korean General Social Survey (KGSS), we examined the relationship between survey attitudes, such as survey enjoyment, survey value, and survey burden, and confidence in leaders in polling organizations. Of the 19 leaders in major institutions, confidence (a great deal + only some) in leaders in polling institutions is ranked 7th (67%), preceded by medicine (85%), financial institution (83%), education (81%), major companies (81%), national statistical agency (78%), and scientific community (73%). Using the logistic regression, we found that survey enjoyment is positively related to confidence in leaders in polling organizations, and survey value and survey burden are not associated with confidence in leaders in polling institutions. Of control variables, trust in society and news is positively associated with confidence in leaders in polling organizations. Compared with liberals, independents and conservatives are less likely to have confidence in leaders in polling institutions. The survey organizations need to improve the ways to make the survey enjoyable to respondents.