All time references are in CEST
"Other, please specify": Analyzing Open-ended Responses for Sociodemographic Questions |
|
| Session Organiser | Ms Christina Pao (Princeton University) |
| Time | Tuesday 18 July, 09:00 - 10:30 |
| Room | |
For many key sociodemographic characteristics of interest (such as gender, sexual orientation, race, and ethnicity), survey designers have taken on a mixed format or fully open-ended response design to allow respondents to write in their identities in a free-text box. Often times, this space encourages respondents to "please specify" or provide "detailed" information. Though these free-text boxes are often included to promote inclusivity, many researchers exclude the content of these free-text responses or censor those selecting "other" from their samples due to small cell sizes. In this session, we will discuss papers that utilize these free-text responses and describe their challenges and opportunities.
Keywords: open-ended, free text, gender, sexual orientation, race, ethnicity
Dr Peter Lugtig (Utrecht University) - Presenting Author
The isolation of the causal effect of survey mode on measurement is difficult due to the fact that selection and measurement effects are (potentially) correlated. Some studies have tried to eliminate selection effects by for example re-interviewing face-to-face respondents in a self-interviewing mode shortly after the original interview. Or they have randomized respondents into a survey mode only after successfully recruiting a respondent into the survey.
In this presentation I will use data from the European Social Survey rounds 9 and 10 to investigate mode effects. In round 9, all countries used face-to-face interviewing. In round 10, nine countries used a self-interviewing instrument (web and paper) with the twentytwo other countries using face-to-face interviewing. The change in survey modes was mostly due to the effects of Covid-19 had on the ability to conduct in-person interviews in some countries. The quasi-experimental design however does allow us to compare countries that switched to self-interviewing with countries that kept using face-to-face interviewing.
Results show that we find no large effects of changing interviewing modes on means, variances and covariances across 111 variables that were measured in both rounds 9 and 10 of the ESS. There are approximately 25 variables where we find effect sizes in the change in means associated with the mode switch is .20 (hedges g) or larger, indicating that there are some variables for which we find mode effects. Additional analyses on experimental data from Great Britain and Finland give further evidence that these effects are probably due to mode measurement. We will in the conference presentation show for what particular variables we find mode effects, and also zoom in differences across countries