Assuring Measurement Quality in the Social Sciences – new standards for quality documentation 1 |
|
| Convenor | Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences ) |
| Coordinator 1 | Dr Natalja Menold (GESIS - Leibniz Institute for the Social Sciences) |
| Coordinator 2 | Dr Constanze Beierlein (GESIS - Leibniz Institute for the Social Sciences) |
Conclusions drawn from survey data can only be reliable if measures used are of sufficient quality. In 2012, the German Data Forum established an expert group to work on common quality standards for measurement instruments. Based on the Total Survey Error approach (Groves et al., 2004), six quality standards were derived. These quality standards address issues such as assuring the validity of a measure as well diminishing non-systematic, systematic, and processing errors. We introduce these quality standards for survey instruments and contrast it with existing alternative standards, present examples of instrument documentations and discuss advantages, limitations and future developments.
The reliability coefficient describes the systematic variance proportion of a measured variable. The reliability is important because descriptive statistics or association between variables can only be estimated accurately if observed variables are reliable. There are several methods for estimating the reliability such as the test-retest correlation, Cronbach’s alpha, the maximal split-half coefficient, or structural equation models. Usually, different methods reveal different reliability estimates because they make different assumptions about the underlying measurement models. I will discuss different estimation methods, their underlying assumptions, and which method is most appropriate for which data.
Quality standards in survey research are in theory established: a good survey shall minimize the Total Survey Error. However, it is difficult to quantify this because it is linked to many decisions taken when a survey is developed. In this presentation we describe the approach behind the software “Survey Quality Predictor “(SQP 2.0) which aims to fill the gap in methodological research by being a tool that quantifies the quality of a question in a survey. SQP 2.0 is a tool to improve survey quality at the stage of questionnaire design and after data collection.
Croatian Burreau of Statistics accepted Total Quality Management approach as the general model for quality management, quality assessment and quality improvement.
Further, CBS established database of quality information which is planned to become a key tool for quality assessment, quality documentation and quality reporting for CBS surveys. The database contains the exhaustive list of quality information, which is based on two two widely accepted ESS structures, ESMS and ESQRS. There is approximately 100 items included in the database which can be divided in two parts: numerical information (quality indicators) and descriptive (textual) information.