Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Open-ended Questions: Methodological Aspects, Use and Analysis 1 |
|
| Convenor | Mrs Cornelia Zuell (GESIS) |
| Coordinator 1 | Dr Evi Scholz (GESIS) |
| Coordinator 2 | Professor Matthias Schonlau (University of Waterloo) |
Open-ended questions in surveys serve to look into respondents' understanding of ideas, issues, etc. The efforts to prepare, code and analyse data of open-ended questions in contrast to closed questions are considerable. Thus, open-ended questions are not nearly as popular as closed questions. However, the growing number of web surveys might offer the chance of investigating various aspects of open-ended questions.
While for closed survey questions much methodological research has been conducted, open-ended questions are, in terms of methodological research, rarely investigated. Recent research on open-ended questions investigates, e.g., mode effects or the length of answers as quality indicator for responses. Other research covers reasons for (non-)response. The quality of answers to open-ended questions is one source of survey error that, if based on factors other than randomness, will result in biased answers and put the validity of the data into question - often disregarded in substantive analyses.
The proposed session aims to help filling that gap. We welcome papers on open-ended questions referring to
a. A comparison of software for textual data analysis,
b. The use of open-ended questions,
c. Analyses techniques,
d. Typology of open-ended questions,
e. Mode effects,
f. Design effects, e.g., question order or position in a questionnaire,
g. Effects of response or non-response,
h. Bias analyses, or
i. Any other topic that addresses quality or assesses the value of open-ended answers.
We also welcome papers that investigate other methodological aspects, e.g., a comparison of response behaviour to open-ended questions in general population surveys vs. in special sample surveys; a comparison of response behaviour to open-ended vs. closed questions for the same topic; or investigation of cross-cultural differences in response behaviour to open-ended questions.
Internet surveys are a useful method of collecting data from children. However, lack of direct contact between respondent and researcher means that very sensitive questions should be avoided. Therefore, the online Kids' Life and Times survey of 10-11 year olds in Northern Ireland consists mainly of closed questions. However, children are given an open-ended question to type in suggestions for future year's surveys. While most responses are innocuous, a small percentage is on sensitive issues that raise concerns about children's health and wellbeing. This paper reports on the methodological and ethical challenges of addressing these issues.
The left-right dimension plays an important role in all sorts of political science studies and analyses are mostly based on the left-right self-placement. In some studies that are interested in deeper understanding, this self-identification is probed by open-ended questions about the meaning of left and right. Due to a considerable non-response in these open-ended questions in the German General Social Survey 2008, quality assessment is recommended before substantive analyses on Germans' ideological ideas start. This paper investigates what affects response behavior to the open-ended questions on the meaning of left and right: we test respondents' personality traits, cognitive abilities and socialization, motivation, and left-right self-placement. Our findings indicate that not only substantial variables matter in response behavior but also some methodological issues.
Answers to open-ended questions, which are provided in continuous text, contain a variety of content-related types of information that go beyond the quantitative capturing of terminology. Accordingly, their coding and analysis pose some specific challenges. The more complex texts become, the more important it is to capture information that derives from text logic or from semantic relations within the text. The GABEK methodology, which was developed for the analysis of complex open texts, takes these requirements into account. Its underlying concept of encoding bases on the semantic structure of the analyzed text seems also to be suitable for analyzing open-ended questions in standardized interviews. The potential of this approach is shown using two open questions from the German General Social Survey 2008.