What do we tell them, and how? Reviewing current practices for communicating about surveys with respondents 2 |
|
| Chair | Mr Alfred Tuttle (US Census Bureau ) |
There is a relational dependency between survey research and the wider public. On the one hand, rigorous research provides empirical grounding for public debates and political decisions. On the other hand, funding for large-scale, high quality survey research depends on the fact that decision-makers and politicians actually acknowledge this importance. In addition, it will be easier to persuade potential respondents to take part in a survey if they are convinced that they participate in something worthwhile that has actual, real world impact. For these reasons, large-scale research infrastructures such as the European Social Survey (ESS) need to think carefully about how they present themselves to the world. This paper evaluates the communication efforts targeting respondents and the general public in the German sample of the ESS. In addition to standard practice, such as advance letters for respondents and press releases for the wider public, three innovations were introduced for round eight of the study: (1) a short, web based film explaining the main features of the survey, (2) targeted press releases to local newspapers in the local area of primary sampling units, and (3) the use of social media profiles on Twitter and Facebook. First, a short video was produced to explain the importance and advantages of the ESS to the general public, but also to support fieldwork efforts. The video was shown at interviewer briefings to boost interviewer motivation and is placed on the website specifically dedicated to potential respondents. Getting possible respondents to see such a video in a face-to-face survey such as the ESS is however a main challenge because the mode of first contact is in writing via the advance letter sent by the survey agency. Respondents could only be pointed to the video by a link in this letter. We investigate the success of this enterprise using page views of the website showing the film. In addition, we present the results of an experiment regarding the placement of the link in the advance letter. Second, we evaluate the effect of press-releases sent out to local newspapers in the specific area of primary sampling units. Before such a measure may impact on actual response behavior, newspapers must actually decide to write something about the survey. We evaluate success at this step using findings from a screening of the targeted newspapers after the press releases were sent. Third, Twitter and Facebook accounts were set up, complementing a study-specific website, in order raise overall awareness of the study. While Twitter is more suitable for targeting media and communication experts, Facebook is potentially more valuable in reaching out to “ordinary citizens”. We present our considerations and challenges in finding and attracting followers, discussing our experience of experimenting with types and topics of content, based on (non-)reactions of friends and followers. We discuss advantages and challenges related to each measure presented here in terms of impact and cost-effectiveness as parts of an overall communication strategy for repeated, cross-sectional population surveys.
All types of surveys are experiencing declining response rates, but longitudinal surveys also have to combat attrition between cycles. Effectively communicating the importance of the survey and the value it provides can help attract new respondents and keep old respondents invested. Additionally, as technologies change and respondents are busier and more difficult to reach, survey contacts must adapt as well. The research presented here begins with a description of the evolution of the contact materials and mailout strategy for a national longitudinal survey, including the content of the letters, the envelopes, and the number of mailings, and culminates with an experiment that tests an updated set of materials and mailout strategy aimed at increasing survey appeal while reducing burden and cost. The experiment reduces the total number of mailings, while increasing the variety of mailing pieces (in look, format, and content) and making the message more motivational. Using feedback from focus groups and cognitive interviews, the experimental messages give specific examples of how the data are used and who uses them, including an infographic based on survey data from prior cycles designed to show potential respondents the value of the survey and to provide feedback to prior respondents. By comparing the results from the existing materials to the updated materials, we will provide insight into the impact that a package of mailing materials with a variety of envelopes with specific, visually appealing messaging has on response rates, response timing, respondent burden, and cost.
There is an increased interest among survey researchers in linking administrative records to survey data. This is valuable because data from administrative records may sometimes be more accurate, or more detailed than it is possible to obtain in surveys.
Linking administrative records to survey data, however, requires the informed consent of the participant and thus makes the communication of the proposed request an important aspect of the study and crucial to obtained high consent rates.
How to best communicate to participants’ the consent request, while appropriately addressing their major concerns and ensuring they have sufficient information for their consent to be fully informed, thus forms a central part of the communication to participants about data linkage.
Asking for consent becomes more challenging in mixed mode studies, and in particular in web surveys, in which there is no immediate support from an interviewer for clarification of questions and reassurance of one’s privacy at the time of the interview. The web, however, provides other opportunities such as better content visualisation, likely to help participants absorb key messages quickly, or quick access to other useful sources of information.
Administrative data holders and ethical committees form another major determinant of the consent process and protocol development setting further legal and ethical requirements.
This paper will discuss how we addressed the challenges in asking for administrative data linkage consent in the Next Steps Age 25 Survey. Formerly known as LSYPE 1, Next Steps is a longitudinal study of over 16,000 young people in England born in 1989/1990. The most recent sweep of data collection took place in 2015-2016 when respondents were aged 25/26, using a sequential mixed mode design involving web, telephone and face-to-face interviews. Seeking data linkage consent was a major component of the study. Nine linkages were sought from 4 different domains (education, health, economic, crime). A 16-page leaflet about data linkage was produced and sent to all participants, and all participants received a letter confirming which consents they had given after the interview.
As data linkage consent was such a major part of the study, the communication of the consent request, as well as the broader process and protocol were a key area for testing prior to the main data collection stage. Exploratory qualitative work tested its general acceptability to participants and contextual issues; all participant materials and operational procedures were tested in the study pilot and approved by data holders and ethical committees.
We will summarise the main findings of this developmental work, particularly in relation to the design of the participant communication materials. We will also cover how we utilised the opportunities of the web mode, for example to embed video content and hyperlinks to help screens and external websites, to enhance participant communications. Finally, we will also report the experience we had with its main stage data collection implementation, and the consent outcomes we achieved and use the web interview paradata to better understand the experience of providing consent online.
The Crime Survey of England and Wales (CSEW) is a continuous face-to-face cross sectional study conducted by Kantar Public on behalf of the Office for National Statistics. Advanced letters are used on the CSEW to pre-notify sampled households that they have been selected for participation and that an interviewer will be calling round to conduct an interview. These letters aim to increase co-operation and (hopefully) participation by ‘selling’ the study to the residents of each sampled household.
Improvements to the advance mailings therefore have the potential to increase participation in the study, and even relatively small improvements in the mailings would have two clear benefits. Firstly, the potential to make estimates from the study more robust (a higher level of response reduces the risk of non-response bias) and, secondly, to reduce costs (maximising response to the original fieldwork phase reduces the need for costly reissues).
Around 53,000 addresses are issued into field each year for the CSEW; this large number of issued addresses provides us with sufficient power to experimentally test the impact of relatively minor changes to the advance mailings. We have recently run experiments on the survey to test the impact of:
(1) Adding the following motivational statement to the advance letter: “Most people we ask to take part in the survey agree to help us”. This statement was developed using principles from the nudge theory, on the basis that by normalising the act of participating in the survey, sampled individuals would be more willing to take part, improving the original issue response rate.
(2) Getting rid of the survey leaflet which traditionally accompanies the letter (which includes additional background on the study and assurances relating to privacy and information security). We tested removing the leaflet, but including a summary of the information on the back of the letter. The hypothesis tested in this experiment was that removing the leaflet (which would reduce costs) would not have a negative impact on the study.
This paper examines the impact which these experiments had on response rates, the number of contact attempts required for interviews to be achieved, and the proportion of respondents that reported having read the advanced mailings.