All time references are in CEST
Experiments in asking for informed consent to data linkage in general population studies 2 |
|
| Session Organisers | Dr Jonathan Burton (Institute for Social and Economic Research, University of Essex) Professor Annette Jackle (Institute for Social and Economic Research, University of Essex) |
| Time | Wednesday 19 July, 16:00 - 17:30 |
| Room | U6-06 |
Linking survey and administrative data offers the possibility of combining the strengths, and mitigating the weaknesses, of both. Such linkage is therefore an extremely promising basis for future empirical research in social science. For ethical and legal reasons, linking administrative data to survey responses will usually require obtaining explicit consent. It is well known that not all respondents give consent. Past research on consent has generated many null and inconsistent findings. A weakness of the existing literature is that little effort has been made to understand the cognitive processes of how respondents make the decision whether or not to consent. The overall aim of this session is to improve our understanding about how to pursue the twin goals of maximizing consent and ensuring that consent is genuinely informed.
We welcome papers which employ an experimental design to:
1. Understand how respondents process requests for data linkage: which factors influence their understanding of data linkage, which factors influence their decision to consent, and to open the black box of consent decisions to begin to understand how respondents make the decision.
2. Develop and test methods of maximising consent in web surveys, by understanding why web respondents are less likely to give consent than face-to-face respondents.
3. Develop and test methods of maximising consent with requests for linkage to multiple data sets, by understanding how respondents process multiple requests.
4. Test the effects of different approaches to wording consent questions on informed consent.
Keywords: consent, administrative data, experiments
Miss Kamila Izimova (HSE University) - Presenting Author
Mr Daniil Lebedev (HSE University)
With introduction of data protection regulations (e.g., GDPR) each survey collecting paradata must get a consent from respondents, but it is not clear how such consent and its format may influence the resulting survey data quality. The aim of the study is to assess how informed consent to the collection of paradata and its format can affect the quality of the resulting web survey data.
The research is based on a full factorial vignette experiment with a between subject design, in which 461 web survey respondents were randomly assigned to 9 groups depending on the format of informed consent. We varied the following elements of the survey: presence of the consent, location of the consent in a survey (at the beginning or at the end), size of information regarding paradata (short or long), format of the answer to consent (Yes/No or one option confirmation). Based on these elements we compared data quality between the groups using the following indicators: paradata (mouse movements, browser focus, number of warnings, length of answers to open questions, completion time), test-retest reliability, straightlining, item nonresponse rate, dropout rate, consent rate and validity.
Results show that completion time was lower among respondents with short information on paradata than with long and with one-option consent question format than two-options. Mouse movement distance was higher when short information about paradata and two-option consent question format were presented while mean number of browser focus changes was lower in case of respondents who received longer paradata description and higher for those who received a two-option consent question format. We can see that not only did the format and presence of the consent influence the data quality, but also led to changes in attentiveness, multitasking behavior, cognitive effort and engagement.
Ms Hannah Soiné (Mannheim Centre for European Social Research (MZES)) - Presenting Author
Dr Jörg Dollmann (Mannheim Centre for European Social Research (MZES) & German Centre for Integration and Migration Research (DeZIM))
Enriching survey data with existing administrative data requires obtaining respondents’ consent. Comparatively little research has addressed this challenge: How should we design record linkage requests in to maximize consent? In this context, we must consider that not all respondents might react in the same way to a specific framing, but consent rates may vary between groups.
We took the ninth wave of the German continuation of the Children of Immigrants Longitudinal Survey in Four European Countries (CILS4EU-DE) as an opportunity to test whether wording the consent question for record linkage as a prospective gain (the information you will give will be more valuable) versus a sunk costs loss framing (the information you have given is less valuable) influences consent rates (see Sakshaug et al., 2015). We also investigate whether sociodemographic characteristics of respondents are related to consent rates and whether the consent rates differ between early and late panel entrants. To do so, we exploit the setup of the CILS4EU-DE data collection: There is considerate variation in respondents’ ethnic and socioeconomic background and the panel that started in 2010 was topped up with a refreshment sample in 2016, which means two points of panel entry.
The survey experiment testing the different wordings of consent was implemented in the web survey (N = 3189). Data collection is almost finalized, and preliminary results suggest that the sunk costs loss framing leads to slightly higher consent rates, although the difference is not statistically significant. We find no effect of sociodemographic characteristics on consent. Surprisingly, respondents from the refreshment sample are more likely to consent than those we retained from the original panel sample. Further analyses are needed to uncover the mechanism behind this pattern, but this finding may have important implications for the timing of record linkage questions in panel surveys.
Mr Sebastian Hülle (Institute for Employment Resereach (IAB)) - Presenting Author
Maximizing panel consent is most important for panels that do not allow for refreshment samples. Additionally, many studies need to maximize linkage consent which can minimize linkage consent bias. These challenges are addressed by the innovative design of a new panel study that uses repeated requests with varying framings to maximize linkage and panel consent.
It´s an evaluation study of two employment programs (§16e and §16i SGB-II) for long-term unemployed in Germany. The two independent samples of wave 1 comprise 16,792 telephone interviews. Panel consent was asked up to three times and linkage consent was asked up to four times within wave 1 fieldwork. The first linkage request was at the very beginning before the questionnaire starts. The 16e-panel has two experimental groups differing randomly in the framing of requests. The design also comprises conversion procedures after the interview via postcard and phone.
The amount of linkage non-consent identified with the first linkage request was reduced with the repeated requests by 53% for the 16e-panel. The total linkage consent rate for the 16e-panel is 98%, for the 16i-panel with its mandatory linkage it is 100% for completed interviews and 97% for partial interviews. The amount of panel non-consent was reduced by about 57%. The overall panel consent rate is 97%. The article also addresses the predictors of why people do not consent or change their mind to consent after repeated requests.
Hence, repeated requests can reduce refusals regarding linkage and panel consent substantially within one wave. This encompasses asking for the (mandatory) linkage at the very beginning of an interview. This article contributes to the state of research by providing applications and evidence on how linkage and panel consent can be maximized to improve data quality in linked panel surveys.
Professor Johann Bacher (University of Linz) - Presenting Author
In surveys, attempts are increasingly being made to link survey data with register, geo-spatial and/or social media data on an individual level. This requires an informed consent. In contrast to other countries, there are only few experiences with an informed consent to data linkage in Austria. Therefore, the willingness of respondents to agree to an informed consent was surveyed in a pilot study.
The pilot study demands for a multiple informed consent to data linkage (Walzenbach, Burton, Couper, Crossley, & Jäckle, 2022), including less sensitive requests like education and occupational status and more sensitive requests like income and health. The respondents are asked whether they agree, disagree or don´t know. A relative long introduction was given in order to cover all aspects that are important in an informed consent (Sakshaug, Schmucker, Kreuter Frauke, Couper, & Holtmann, 2021).
The presentation will provide results of the pilot study concentrating on the following questions: (1.) How high is the average consent rate of persons to data linkage? (2.) How large are the differences in the rates of agreement to data linkage for the different requests? (3.) Are there differences by respondents´ characteristics?
References
Sakshaug, J., Schmucker, A., Kreuter Frauke, Couper, M. P., & Holtmann, L. (2021). Respondent Understanding of Data Linkage Consent. Survey Methods: Insights from the Field. Advance online publication. https://doi.org/10.13094/SMIF-2021-00008
Walzenbach, S., Burton, J., Couper, M. P., Crossley, T. F., & Jäckle, A. (2022). Experiments On Multiple Requests For Consent to Data Linkage in Surveys. Journal of Survey Statistics and Methodology. Advance online publication. https://doi.org/10.1093/jssam/smab053
Dr Sandra Walzenbach (Universität Konstanz) - Presenting Author
Dr Jonathan Burton (Institute for Social and Economic Research, University of Essex)
Professor Mick P. Couper (Survey Research Center, University of Michigan)
Professor Annette Jäckle (Institute for Social and Economic Research, University of Essex)
When asking respondents for consent to link their survey data to administrative records, one of the few consistent findings is that consent rates are much higher in personal interviews than in web surveys: Previous studies have found differences of around 30 percentage points. This difference is somewhat surprising - even more so as studies that have recorded and behaviour-coded respondent-interviewer interactions have found that interviewers do not actively do much to encourage consent, apart from reading out details of the linkage and the actual consent request.
So far, previous research could not disentangle the effects of social desirability bias and mode-specific decision processes: Are the observable differences in consent a consequence of socially desirable responding or does the process of interviewers reading out the relevant information nudge respondents into processing the request more thoroughly, which in turn leads to higher consent rates?
To empirically assess these questions, we ran experiments in the Understanding Society Innovation Panel 2022. The survey used a mixed mode design whereby sample members were randomly allocated to web-first or CAPI-first. For the consent question (asking for permission to link to government tax records), CAPI respondents were assigned to one of three groups:
-CAPI
-Partial-self-interview (CASI) module, in which the interviewer reads out the consent question but hands the computer over for the respondent to answer the question privately
-CASI
Web respondents were assigned to one of two groups:
-Control group asked the standard consent question
-Photo of the study director and personal message
The treatments vary factors that are expected to affect social desirability bias (degree of interviewer involvement, personal message) while holding the decision process constant.
Fieldwork is completed, with around 730 CAPI and 1850 web respondents, however we do not yet have the data to provide early results.