Back to Overview of Sessions

Session Details

According to the framework of the total survey error, validity refers to the degree to which survey results can be interpreted with respect to the concepts under investigation, i.e. certain opinions, behaviors, abilities, and competencies. Validity also regards interpretations about the differences or changes in the searched concepts, such as comparisons between different respondent groups, across time or cross-culturally. Survey researchers conduct studies in various languages and cultures within one country or in various countries and gather demographic, administrative and social data in these multi-cultural contexts, constantly trying to improve the accuracy of these measurements. The comparative aspects of validity have been referred to as measurement equivalence issues. Many researchers address measurement equivalence during data analysis, after data collection, often finding that there is a lack of measurement invariance. However, the sources of measurement invariance are more likely to be associated with questionnaire design and data collection processing.
Difficult questions, overloaded instructions or visual design elements can affect validity and measurement equivalence. This session aims to discuss methods of developing measurement instruments and their effect on validity and measurement equivalence. The goal is to better understand the corresponding sources of measurement error and to present methods which help to increase validity in comparative research. In particular, generic, multi-method approaches are of interest. Such approaches can include expert reviews by subject matter experts, cognitive interviews and pilot interviews with respondents who represent the main demographic groups of the target countries. In addition, quantitative analyses of findings, e.g. from experiments related to the use of different versions of questionnaires can help to evaluate the sources of decreased validity and deficient measurement equivalence.