ESRA 2025 sessions by theme

Back to Overview of Sessions

Session Details

The pretesting of survey questionnaires is crucial for ensuring high quality data in social research. In order to establish the validity of survey responses, it is essential to assess how respondents understand the questions and whether their understanding is consistent with the researcher's intended meaning. Over the years, Willis' (2005, 2015) original approach of cognitive pretesting, rooted in cognitive psychology, has been further developed and merged with other survey methodological and qualitative research approaches into new forms of open pretesting. These allow us to focus on respondents' understanding of survey questions by listening, to varying degrees, to what respondents actually think about the questions.

However, there is a trade-off between the richness of the data and the effectiveness of data collection and analysis. At one end of the spectrum is web-probing, which is highly efficient at collecting data on a large scale, but with a limited amount of information. At the other end are qualitative-interpretive approaches, such as grounded theory, which provide in-depth understanding at a high cost in time and resources. Classical cognitive pretesting, using e.g. standardised and emergent probes, falls between the two extremes, as do qualitative pretesting methods using more focused methodologies, such as problem-centred interviews.

In this session we bring together researchers who are developing and using different approaches across the spectrum of open pretesting. We are particularly interested in exploring the added complexity of different cultures and languages across countries. In addition, we will discuss how data collected using these methods can be used for further substantive research beyond the original purpose of pretesting survey questionnaires.