Dr Kristen
Miller (National Center for Health Statistics )
Coordinator 1
Kristen Miller
Coordinator 2
Gordon Willis (National Cancer Institute)
Session Details
While researchers have analyzed cognitive interviews in a variety of ways, there has been little discussion regarding the process of analysis in cognitive interview literature. That is, there has been little explanation as to how cognitive interviews should be examined and studied to produce reputable findings. In the past year, however, this void has begun to be addressed by publications presenting various analytic techniques for producing cognitive interview findings. Additionally, it has just recently been recognized that cognitive interviewing studies can serve multiple functions toward understanding the performance of a survey question. As traditionally understood, cognitive interviewing studies can identify various difficulties that respondents may experience when attempting to answer a survey question. Cognitive interviewing studies may also examine construct validity in that they can identify the content or experiences that respondents consider and ultimately include in their answer. Finally, cognitive interviewing studies can examine issues of comparability, for example, the accuracy of translations or equivalence across socio-cultural groups. The type of analytic processes employed within a cognitive interviewing study guides the types of conclusions that can be made. This session will focus specifically on issues related to the analysis of cognitive interviews. Topics include analytic techniques, specific methods for addressing study goals (e.g. accuracy of translations and cross-cultural comparability), practices to support study transparency and believability, and ways of assessing and addressing varying levels of data quality,
Paper Details
1. Cognitive Interviewing for new survey of U.S. Microbusinesses
Ms Audrey Kindlon
(National Science Foundation)
Dr Jennifer Crafts (Westat) Dr Brad Chaney (Westat)
As part of a new survey of small independent U.S. microbusinesses (those with fewer than five employees), cognitive interviewing was employed to assist with questionnaire development. Six rounds of cognitive interviews were conducted with microbusiness owners using an iterative approach. The use of multiple rounds allowed a thorough examination of respondent’s interpretation of key terms, and the exploration of alternative approaches to more precisely measure businesses’ Research and Development and innovation activities.
2. Analytic Techniques to Examine Construct Validity in Cognitive Interviewing Studies
Dr Kristen Miller
(National Center for Health Statistics)
Cognitive interviewing studies are traditionally used to discover survey question problems. However, because the method can determine respondents’ interpretations, they also identify the phenomena (or construct) measured by a variable. Using analytic techniques described in Cognitive Interviewing Methodology (Miller, et al, 2014), this paper illustrates how cognitive interviews can be analyzed to examine construct validity. A key theme will be data quality of cognitive interviews and the assessment of that quality when performing analysis. The presentation will draw upon a 60 cognitive interview case study to evaluate a set of disability questions intended for international use.
