ESRA 2025 Preliminary Program

Monday 14 July    Tuesday 15 July    Wednesday 16 July    Thursday 17 July    Friday 18 July   

All time references are in CEST

Recruiting Web Surveys via Postal-Mail: Best-Practice, Experiments, and Innovation 2
Session Organisers	Dr Jean Philippe Décieux (University of Bonn & Federal Institute for Population Research (BiB)) Dr Carina Cornesse (GESIS)
Time	Thursday 17 July, 13:30 - 15:00
Room	Ruppert paars - 0.44

Since e-mail addresses are usually unavailable on standard sampling frames of broader population surveys (e.g., population registers), recruiting high-quality web surveys is challenging. When conducting such large-scale and large-scope web surveys, recruitment and surveying is, therefore, typically conducted in two separate steps: First, a (probability-)sample of the study population is drawn and contacted offline, often during a brief face-to-face or telephone recruitment-interview. Second, members of the sample are asked to switch to the online mode for the actual survey.
Compared to interviewer-administered contact and recruitment, postal-mail strategies are becoming increasingly popular and a large number of cross-sectional as well as longitudinal web survey projects are currently being initiated using postal-mail recruitment in combination with online survey methodology. There are several reasons for this. For example, recruiting web surveys via postal-mail is usually both more time- and cost-efficient than the available alternatives. In addition, this strategy avoids undesirable interviewer effects and allows respondents to read through study and recruitment material at their own speed, time, and convenience.
Currently, the methodology for successful postal-mail recruitment of web surveys is advancing fast. Therefore, this session aims to provide a broad exchange forum for researchers and projects working on and with postal-recruited web surveys. In addition to sharing experiences and best-practices, we are particularly interested in experimental approaches that might include, topics such as:
• Strategies for enabling the transition from offline contact to web data collection mode
• Comparing the success of postal-mail recruitment to other web survey recruitment strategies
• Optimizing initial response, panel consent, and panel registration for postal-mail recruited longitudinal studies
• Push-to-web and other mixed-mode recruitment approaches
• Cost-benefit analyses of different incentive and reminder strategies
• Design and layout

Keywords: Web Survey, Recruitment, Mixed-Mode, Survey Costs, Postal Recruitment, Experimental survey research

Papers

It's a bird! It's a plane! It's an ABS Approach to Explore Knowledge of Local Potentially Harmful Health Effects

Ms Tania Gutsche (University of Southern California) - Presenting Author
Mr Bas Weerman (University of Southern California)
Ms Jenna Blyler (University of Southern California)
Dr Arie Kapteyn (University of Southern California)

Approached by colleagues at USC, the Center for Economic and Social Research (CESR) identified members of our probability based national internet panel, The Understanding America Study (UAS), who were living adjacent to an aerospace lab that has been in operation under numerous owners since 1955, in the Los Angeles region. We then complemented our known sample with an ABS push-to-web approach. Though our center has numerous experiences with specific population recruiting, we had always included a letter of institutional support, e.g. from the Mayor (Compton Well-Being Study) or the Los Angeles Department of Public Health (DPSS Tayportunity Study). This survey was solely researcher-driven. The 10-minute survey offered at $20 Amazon gift code, and landed with a population keen to tell their story. Our colleagues had done a great deal of research in the field to make sure the survey was relevant through in- person interviews and community meetings. The survey explored topics such as knowledge of environmental effects, correct assignment of blame/clean up responsibility, duration of time living near the site, intent to move, and self-reported health outcomes such as cancers for themselves and their pets. At the end the survey offered an opportunity to further share their story with a qualitative researcher by leaving their contact information. We achieved a response rate of 28%, which is higher than our prior California ABS recruitment rates. Contributing factors outside of familiarity or trust in our local academic institution are likely in part due to the immediacy of the topic. Through this experiment, we are encouraged to find a way to make our ABS recruitment campaigns as personally engaging as this one.

When push comes to shove: Assessing the impact of soft-push-to-web versus hard-push-to-web strategies in mailed online questionnaires

Mr Felix Cassel (University of Gothenburg) - Presenting Author
Dr Sebastian Lundmark (University of Gothenburg)

Rising costs of traditional survey modes and declining response rates are well-documented challenges for contemporary survey research. These challenges may have implications for the representativeness of survey results. Push-to-web data collection methods are increasingly used by survey practitioners to decrease the costs of printing and postage when mailing back physical questionnaires. This study examined whether two different push-to-web strategies produced dissimilar survey response rates, nonresponse bias, and subsequent willingness to join an online access panel.

In 2024, the SOM Institute at the University of Gothenburg conducted a push-to-web mixed-mode survey of a random probability sample of the Swedish population (N = 4,046). The sample persons received four invitations by mail and two text message reminders. Prior to inviting the potential respondents, all sample persons were randomly assigned into two experiment groups: 1) a “soft-push-to-web” group (n = 2,027) in which the participants received a paper questionnaire by mail from the first reminder, and 2) a “hard-push-to-web” group (n = 2,019) in which a paper questionnaire was distributed by mail only in the third and final reminder. At the end of the survey, participants were asked to sign up for the Swedish Citizen Panel (SCP), a non-commercial online access panel administered by the SOM Institute.

Preliminary findings from the experiment showed that the softer-push-to-web recruitment strategy increased response rates by approximately 4 percentage points. Further analyses on the recruitment strategy’s impact on nonresponse bias as well as the likelihood of joining the SCP will be conducted in early 2025 and presented at the conference.

Designing Passwords for Web Survey Access: The Effect of Password Length and Complexity on Survey and Panel Recruitment

Mr Georg-Christoph Haas (Institute for Employment Research) - Presenting Author
Mrs Marieke Volkert (Institute for Employment Research)
Mr Stefan Zins (Institute for Employment Research)

For online probability surveys that recruit participants via postal invitation letters, passwords are used to manage access to the survey. These passwords serve several purposes, such as, blocking uninvited individuals, and preventing multiple submissions from the same individual. Research on web survey passwords has primarily focused on whether providing a password for survey access affects response rates. However, the chosen password strength, that is, length and complexity, may also affect response propensities. Password length refers to the number of characters in a password. Password complexity involves the set of characters from which the password can be derived (e.g., lowercase letters and numbers). Our research evaluates the effect of different password lengths and complexities on survey access, response rates, panel registration and linkage consent. We implemented a survey experiment by varying password length and complexity during the first wave of the OPAL study. For recruitment, every individual received a postal invitation letter with a web-link and QR-code directing to the survey, along with an individualized password. We conducted a 2×2 experiment that manipulated password length (five and eleven characters) and complexity (uppercase letters and uppercase + lowercase letters + numbers). Additionally, we included a group that used the default length and complexity settings of the service hosting the survey (eight uppercase letters). Invited individuals were randomly assigned to one of these five groups across two different probability samples: employees (N=77.173) and welfare recipients (N=99.176). Results show that short as well as long passwords increase the access rate compared to the control group (16.7%, 19.2% vs. 14.9%). The positive effects of the password designs remain for response and panel registration rates. We also find that long passwords have a positive effect on the propensity to consent to linking survey with administrative data.

Evaluating the First Year of a Push-to-Web Recruitment Strategy in the German Job Vacancy Survey

Professor Nicole Gürtzgen (IAB)
Dr Alex Kübis (IAB)
Dr Andre Pirralha (IAB) - Presenting Author

As web-based data collection continues to replace traditional interviewer-led modes, effective offline-to-online recruitment has become central to large-scale surveys. This presentation evaluates the first year of a push-to-web mail recruitment strategy in the German Job Vacancy Survey (IAB-JVS), a repeated cross-sectional establishment survey that draws a large, stratified random sample of about 110,000 German establishments each year.

Traditionally, the IAB-JVS included a comprehensive questionnaire in the fourth quarter, complemented by shorter telephone surveys in subsequent quarters. Recently, the design shifted toward inviting establishments to complete the survey online, relying on postal mailings as the primary contact. If no response is received, a follow-up mail reminder includes an option to participate via a paper-and-pencil questionnaire.

This push-to-web approach offers an interesting setting to examine how postal mail can facilitate transitions from offline contact to web-based participation, especially in a survey that must balance cost, large sample sizes, and diverse establishment types. Drawing on data from the first year of implementation, we assess key metrics such as response rates, fieldwork timing, and nonresponse patterns. The findings contribute to the discussion on postal-mail recruiting strategies by illustrating both strengths and limits in reaching certain subgroups and maintaining data quality. By sharing these insights, we aim to inform best practices for recruiting web surveys via postal-mail, particularly in the context of establishment surveys.

Recruitment incentive experiment of the probability-based panel Health in Germany: results on outcome rates, non-response bias and panel case costs

Mr Stefan Damerow (Robert Koch-Institut) - Presenting Author
Mr Johannes Lemcke (Robert Koch-Institut)
Mr Ilter Öztürk (Robert Koch-Institut)
Mrs Jennifer Allen (Robert Koch-Institut)

Background
The Robert Koch Institute (RKI) set up a probability-based panel infrastructure focused on public health research (‘Health in Germany’) (registered active panelists about 47.000). Due to declining response rates in recent decades, incentives have become increasingly important. Therefore, for the first recruitment study of the panel Health in Germany the RKI conducted an incentive experiment with a random sub-sample in order to test the effectiveness of different incentive schemes. The central questions of the incentive experiment are: Differentiated by incentive group (a) the response rate for panel registration, (b) the possible distortion due to non-response bias and (c) the panel case costs.

Methodology
The study population comprises all persons aged 16 and over living in Germany. Around 170,000 addresses of the residents' registration offices were used as a random sample. The field period ran from January to May 2024. Some of the target persons were randomly selected for an incentive experiment with four groups of 1440 individuals each. The incentives were either paid unconditionally beforehand (‘before’) or were linked to registration for the panel (‘after’). The incentive schemes of the groups were: (1) €5 before, €10 after, in cash; (2) €10 after, in cash; (3) €5 before, €10 after, as a voucher; (4) no incentive at all (control group).

Results
The incentive experiment replicates existing incentive experiment studies. The use of cash instead of vouchers and the unconditional payment (‘in advance’) of €5 substantially increases the willingness to participate (about 12 percentage points difference). Particularly in the hard-to-reach population group of people with a low level of education, a less biased sample composition can be observed in comparison to all other incentive groups. The results of the incentive experiment provide insights into how high data quality can be achieved with fewer resources.