ESRA 2025 Preliminary Program
All time references are in CEST
Recruiting Web Surveys via Postal-Mail: Best-Practice, Experiments, and Innovation |
| Session Organisers |
Dr Jean Philippe Décieux (University of Bonn & Federal Institute for Population Research (BiB))
Dr Carina Cornesse (GESIS)
|
| Time | Thursday 17 July, 09:00 - 10:30 |
| Room |
Ruppert paars - 0.44 |
Since e-mail addresses are usually unavailable on standard sampling frames of broader population surveys (e.g., population registers), recruiting high-quality web surveys is challenging. When conducting such large-scale and large-scope web surveys, recruitment and surveying is, therefore, typically conducted in two separate steps: First, a (probability-)sample of the study population is drawn and contacted offline, often during a brief face-to-face or telephone recruitment-interview. Second, members of the sample are asked to switch to the online mode for the actual survey.
Compared to interviewer-administered contact and recruitment, postal-mail strategies are becoming increasingly popular and a large number of cross-sectional as well as longitudinal web survey projects are currently being initiated using postal-mail recruitment in combination with online survey methodology. There are several reasons for this. For example, recruiting web surveys via postal-mail is usually both more time- and cost-efficient than the available alternatives. In addition, this strategy avoids undesirable interviewer effects and allows respondents to read through study and recruitment material at their own speed, time, and convenience.
Currently, the methodology for successful postal-mail recruitment of web surveys is advancing fast. Therefore, this session aims to provide a broad exchange forum for researchers and projects working on and with postal-recruited web surveys. In addition to sharing experiences and best-practices, we are particularly interested in experimental approaches that might include, topics such as:
• Strategies for enabling the transition from offline contact to web data collection mode
• Comparing the success of postal-mail recruitment to other web survey recruitment strategies
• Optimizing initial response, panel consent, and panel registration for postal-mail recruited longitudinal studies
• Push-to-web and other mixed-mode recruitment approaches
• Cost-benefit analyses of different incentive and reminder strategies
• Design and layout
Keywords: Web Survey, Recruitment, Mixed-Mode, Survey Costs, Postal Recruitment, Experimental survey research
Papers
Sustainability of an offline-recruited online-panel of refugees: The experiences of the “Refugees from Ukraine in Germany Survey”
Dr Andreas Ette (Federal Institute for Population Research) - Presenting Author
Dr Jean Philippe Décieux (Federal Institute for Population Research)
Following Russia’s invasion of Ukraine in early 2022, more than one million refugees arrived in Germany. These Ukrainian refugees differ in many aspects from Germany’s past experiences with forced migration, creating an urgent need for reliable data to inform policymakers, practitioners, and academics. In response, the “Refugees from Ukraine in Germany (IAB-BiB/FReDA-BAMF-SOEP Survey)” was established to provide high-quality longitudinal data.
This presentation offers insights into the six waves of this new refugee panel, providing a unique case study on the challenges and strategies involved in maintaining an offline-recruited online-panel of refugees. Initiated during the early months of the war in Ukraine, the survey employed a register-based sampling frame to recruit participants offline, who were then invited to participate in subsequent survey waves online. This hybrid recruitment approach aimed to balance the need for representative sampling with the flexibility of digital data collection, particularly for the highly mobile Ukrainian refugee population.
The presentation evaluates the sustainability of the offline-recruited online-panel in two steps. First, the determinants of nonresponse in the recruitment wave are analyzed using auxiliary personal information from Germany’s population register, alongside detailed regional and survey characteristics. This highlights the key factors influencing participation, attrition, and engagement. Second, the determinants of retention—stayers, gradual attriters, and attriters—are examined across six panel waves, demonstrating how survey design decisions in the recruitment wave have lasting impacts on panel participation.
We explore how initial offline recruitment shaped panel composition and how digital communication strategies, such as email reminders and adaptive fieldwork techniques, supported retention. Special attention is given to subgroup-specific dynamics to identify disparities in participation patterns. The findings underscore the importance of integrating adaptive strategies into survey design to sustain engagement in online panels over time.
Think inside your box? Formal, friendly, and honest advance letter designs at the Hungarian Central Statistical Office
Dr Ferenc Mújdricza (Hungarian Central Statistical Office)
Mrs Mária Zanatyné Fodor (Hungarian Central Statistical Office)
Mr Mátyás Gerencsér (Hungarian Central Statistical Office) - Presenting Author
Mrs Linda Mohay (Hungarian Central Statistical Office)
The first impression on most household survey respondents, advance letter design, is paramount in gaining cooperation – especially in online self-completion. The main approaches, the ‘old’ formal, mandatory approach and the ‘new’ respondent-centred approach with behavioural science techniques were combined at the Hungarian Central Statistical Office. This ‘new-old’ official design employs authoritative tone, purposefully complicated text, legal references, pressing nudges, austere graphics, etc. in a tweaked behavioural science framework so as to harness the potential of the mandatory approach in voluntary participation surveys. Initial experiments proved that the ‘new-old’ formal design performs drastically better than semi-formal alternatives. Priority mailing added further significant amplification to the effect of the formal design.
The two split-sample experiments in the present study with further improved push-to-web letter designs and additional postage modes of formal letters reinforced previous results in a CAWI setting. The authoritative/formal design yielded significantly better response rate and response quality than a semi-formal letter, tailored to target young respondents, as well as a novel ‘honest’, trust-inducing design. The latter was to inform in a candid, open way, avoid manipulative features, and express trust by sharing facts (e.g. out-of-date databases necessitate conducting the survey). Priority mailing of formal letters increased cooperation significantly in comparison with the ordinary mail condition, with the additional benefit of more prompt responding. Registered postage performed similarly, but with much higher unit costs.
In sum, thinking more ‘inside our box’ is useful in designing effective push-to-web advance letters: in contrast with semi-formal, ‘respondent centred’ letters, formal advance letter designs harvest core, official characteristics of NSOs. This authenticity might be essential in gaining cooperation as it probably aligns better with general respondent expectations towards (statistical) offices, superseding respondent ‘preferences’ or ‘needs’.
Optimizing Contact Strategies in the Self-Administered Panel FReDA
Dr Detlev Lück (Federal Institute for Population Research (BiB)) - Presenting Author
Dr Pablo Christmann (GESIS - Leibniz Institute for the Social Sciences)
Dr Tanja Kunz (GESIS - Leibniz Institute for the Social Sciences)
Professor Tobias Gummer (GESIS - Leibniz Institute for the Social Sciences, Germany)
Ms Angelika Steinwede (infas - Institute for Applied Social Science, Germany)
“FReDA - The German Family Demography Panel Study” is a large German panel study that fielded its recruitment wave in Spring 2021. Interviews are conducted in self-administered modes: mainly by web interviews (CAWI), partly by mail-sent paper questionnaires. The initial plan to switch to self-administered modes only after a personal recruitment interview was disrupted by the COVID-19 pandemic. So, also the recruitment took place by postal invitation letters and self-administered interviews. This was accomplished with satisfying rates of participation and panel consent. One presumably helpful design feature was a short recruitment questionnaire with low respondent burden. A second feature is an unconditional prepaid incentive of 5 Euro in cash that is added to each invitation, in the recruitment as well as in all subsequent waves.
In late 2024, a first refreshment sample was drawn. Although there are no longer any contact restrictions, FReDA still does not use interviewers and continues with its recruitment strategy described above. Nevertheless, the recruitment of the refreshment came with a number of experiments which are supposed to help optimising this strategy. In a first experiment, the gross sample (n=50,270) was split up into one experimental group interviewed in web mode only and a second group interviewed in a sequential “web first” mixed mode design. Further experiments tested the effects of various enclosures to the invitation letter: a project information flyer, a greeting card that plays a sound file when opened, and a card with a QR code inviting to visit an online motivation video. One experiment tested the effect of sound files in the web questionnaire, reading out the questions. Finally, the effect of an additional 10 Euro postpaid inventive in case of a panel consent was tested. We compare outcome rates for participation, panel consent and sample composition.
