Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Tuesday 14th July, 14:00 - 15:30 Room: L-103

Assuring Measurement Quality in the Social Sciences – new standards for quality documentation 2
Convenor	Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences )
Coordinator 1	Dr Natalja Menold (GESIS - Leibniz Institute for the Social Sciences)
Coordinator 2	Dr Constanze Beierlein (GESIS - Leibniz Institute for the Social Sciences)

Session Details

Conclusions drawn from survey data can only be reliable if the survey instruments (e.g. survey questions and questionnaires) are of sufficient quality. Based on the Total Survey Error approach accessing and documentation of measurement quality in surveys will be addressed. The aim of the session is to evaluate existing standards and current survey practices with respect to the measurement error in surveys and to discuss advantages and limitations as well future developments. Researchers are invited to submit papers dealing with standards and practices in assessment and documentation of measurement quality in surveys, such as assuring the validity of a measure as well diminishing non-systematic, systematic, and processing errors.

Paper Details

1. Best Practices in Short Scale Development: Comparing State-of-the-Art Methods using Simulated and Empirical Data
Dr Constanze Beierlein (GESIS - Leibniz Institute for the Social Sciences, Germany)
Dr Peter Kruyen (Radboud Universiteit Nijmegen, The Netherlands)
Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences, Germany)

Psychological constructs have attracted increasing attention as predictors of social phenomena. However, many psychological measures include too many items to be useful in large-scale surveys. In order to tackle this problem, researchers often remove items from these measures, but psychometricians have shown that well-known item-selection procedures impair the quality of these scales. Recently, psychometricians have developed new and advanced short-scale construction methods. In our talk we will compare different old and new “top-down” and “bottom-up”-item-selection techniques using simulated and empirical data. We will give hands-on recommendations for practical applications in surveys.

2. How well do short personality measures work? Comparison of the BFI-10 with the BFI-15 in the Swiss Household Panel
Dr Valérie-anne Ryser (FORS)

Using samples of the Swiss household panel (SHP), this study investigates the quality and validity of two different short scales of the Big Five personality traits: the 10-item Big Five Inventory Ten introduced in the SHP in 2009, and the 15-item Big-Five-Inventory-Short version introduced in a subsample of the SHP in 2014.
The measurement models postulated by the authors of both scales have been evaluated using confirmatory factor analyses. Results demonstrate that the SHP data do not support the five factors structure postulated by the theory of both scales.

3. Evaluating the quality of education variables in cross-national surveys
Ms Verena Ortmanns (GESIS - Leibniz Institute for the Social Sciences)

Measuring educational attainment in cross-national surveys correctly is challenging. We know that substantial inconsistencies in the distribution of education coded in the International Standard Classification of Education (ISCED) exist. They are mainly caused through inconsistent coding of country-specific education categories into ISCED. In the paper the processing of education variables of four cross-national surveys is analyzed. Based on these analyses, general recommendations on data harmonization and processing for quality assurance will be proposed. In the project “Computer-Assisted Measurement and Coding of Educational Qualifications in Surveys” (CAMCES) these recommendations are taken up systematically.