Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Thursday 16th July, 11:00 - 12:30 Room: HT-104

The impact of questionnaire design on measurements in surveys 5
Convenor	Dr Natalja Menold (GESIS )
Coordinator 1	Ms Kathrin Bogner (GESIS)

Session Details

Questionnaire design is crucial for obtaining high-quality survey data. Still, there is a great need for research that helps to better understand how and under which conditions different design aspects of questionnaires impact measurement process and survey data quality. Therefore, researchers are invited to submit papers dealing with questionnaire design features such as question wording, visual design and answer formats, instructions, introductions and other relevant design aspects of questionnaires. Also, different means of measurement such as questions with nominal answer categories, rankings, ratings, sematic differentials or vignettes can be addressed or can be matter of comparison. Of interest is the impact of questionnaire design on response behavior, on systematic as well as non-systematic error or on validity. In addition, respondents’ cognition or motivation can be in focus of the studies.

Paper Details

1. Rounding in household financial surveys: the roles of survey design and individual characteristics
Dr Joanne Hsu (Federal Reserve Board)
Dr Michael Gideon (NORC)
Dr Brooke Mcfall (University of Michigan)

Household financial survey questions often offer ranges to collect some useful data even when respondents are unwilling or unable to provide an exact value. Yet, because self-reported “exact values” are often rounded, bracketed measures may sometimes be more precise than exact numbers. This paper explores the prevalence of rounding, exact values, and item nonresponse on income questions on three surveys with different range alternatives (range cards, user-provided ranges, unfolding brackets) and in different modes. We analyze the determinants of rounding, particularly the role of education, wealth categories, cognition/financial literacy, time to respond (using paradata) and survey mode.

2. The Influence of an Up-Front Experiment on Respondents’ Recording Behaviour in Payment Diaries: Evidence from Germany
Dr Tobias Schmidt (Deutsche Bundesbank)
Ms Susann Kuehn (Deutsche Bundesbank)

We analyse the effect of an experiment, eliciting respondents’ risk preferences, on the recording behaviour of German consumers in a one week diary on their point-of-sales expenditures. In the experiment, run shortly before the consumers start to fill in the diary, the respondents either win 20 euro or nothing. Our results indicate that the outcome of the game has an impact on the quantity of transactions recorded, but does not affect the quality of information recorded and measures like the cash share.

3. Numeric Codes in Questionnaires - The Influence on Unit Nonresponse, Item Nonresponse, and Misreporting
Mr Johannes Bauer (Ludwig-Maximilians University of Munich )
Mr Felix Bader (Ludwig-Maximilians University of Munich )
Mr Patrick Riordan (Ludwig-Maximilians University of Munich )

Numeric codes printed on the questionnaire are often used for a fast and efficient entering of the data. However this procedure can provoke concerns about anonymity that may lead to unit nonresponse, item nonresponse and misreporting. We conducted an experiment in a mail survey on group-focused enmity. Our results show no deviation in case of unit nonresponse. We found an increase in nonresponse to sensitive items between questionnaires with and without codes. There is also a misreporting bias towards socially desirable answers to sensitive questions for questionnaires with a statement to the numeric code in the cover letter.

4. How can we solve problem of “telescoping effect” with using landmark events in questionnaire design: case of dog bites survey
Mr Eugen Bolshov (Kiev center of political and conflict studies)

It is well-known that self-reports about events that occurred long before the interview may not be entirely valid and reliable. Such difficulties we faced during CATI-study on the problem of stray dogs. Data about incidence of dog bites from survey and from medical statistics didn’t match (survey results showed higher incidence). To explain the differences between official statistics and survey results we used hypothesis about respondent’s memory processing and impact of questionnaire from the victimization studies. Three experiments where carried to test hypothesis about ‘telescoping effect’, ‘context effect’ and impact of landmark events in questionnaire