Methods for Measurement Error Adjustment |
|
| Session Organiser |
Dr Stéphane Legleye (INSEE ; Inserm) |
| Time | Friday 19th July, 13:30 - 14:30 |
| Room | D21 |
This session includes papers that showcase different methods for adjusting for measurement error.
Keywords: survey error, correction methods, mixed mode data
Dr Stéphane Legleye (INSEE ; Inserm) - Presenting Author
Mr Gaël de Peretti (INSEE)
Mr Tiaray Razafindranovona (INSEE)
While the data collection mode effect is generally perceived as a nuisance, it can be beneficial and desired and help dictate the choice of a mixed-mode protocol. Situations must be assessed on a case-by-case basis, according to the purpose of the surveys, their place in observation systems, their public and research uses, and finally by the existence of series of measurements over time. After having defined the selection effect and the measurement effect, and recalled the main techniques to separate them, I will present some approaches recently defined to try to contain or even neutralize the measurement effect.
Finally, I will expose a pragmatic and parsimonious approach developed at the French National Institute for statistics and economic studies (INSEE), aiming to reduce measurement bias to the maximum when necessary. The method can be applied to all kind of mixed-mode designs with a reference and an alternate data collection mode (defined by their measurement quality). It is based on the estimation of the measurement effect by classical means then on the imputation of only a subset of the observations of the alternative mode. The subset is defined in two steps: first, a matching technique (or a balance random sampling) selects a subset of the observations (imputation support); second, a random selection of the observations whose values are the farthest from their counterfactual value estimated in the imputation support (the proportion being defined to achieve equality of the outcomes in the imputed imputation support). Then the imputation (either deterministic or stochastic by the mean of multiple imputations) is performed. Efficiency, limits and comparison with calibrations are discussed.
Mr Christoph Giehl (TU Kaiserslautern) - Presenting Author
Systematic measurement errors like question order effects are not only biasing the response behaviour towards a specific item, but also measures of latent attitudes and data quality itself. If, for example, an assimilation effect of question order leads to enhanced covariations be-tween subsequent items of an attitude scale, the latent mean of such an attitude, the factor loadings of the items, measures of model fit, and reliability measures will be systematically skewed due to systematic error correlations.
Analysing studies, determinants like the mode of information processing and the attitude ac-cessibility (Mayerl/Giehl 2018) as well as the general attitude towards surveys (Stocké 2004) were identified in order to control systematic measurement errors. Those variables were then used within a latent moderation structural equation model (LMS) to control for interaction effects between subsequent error terms within a confirmatory factor analysis. Factor loadings, latent means and measures of data quality were thereby adjusted, leading to more unbiased measures.
In this presentation we introduce a structural equation model using multiple interactions on error term level (multiple interactions on error terms structural equation model, MIETSEM) based on data of a 2017 conducted experiment with students of the technical university Kai-serslautern, Germany. In addition, we present findings from a simulation study showing the benefits and limitations of MIETSEM in order to provide a tool for identifying and control-ling for systematic measurement errors.
Mayerl, J.; Giehl, C. (2018): A Closer Look at Attitude Scales with Positive and Negative Items. Response Latency Perspectives on Measurement Quality. Survey Research Methods 12 (3), 9999 - 10016.
Stocké, V. (2004): Entstehungsbedingungen von Antwortverzerrungen durch soziale Er-wünschtheit. Ein Vergleich der Prognosen der Rational-Choice Theorie und des Modells der Frame-Selektion. Zeitschrift für Soziologie 33, 303–320.