Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Web data collection for probability-based general population surveys 1 |
|
| Convenor | Professor Peter Lynn (University of Essex) |
| Coordinator 1 | Ms Lisa Calderwood (Institute of Education, University of London) |
| Coordinator 2 | Ms Gerry Nicolaas (NatCen Social Research) |
Web survey methodology is well-established for non-probability online panels and for specialist populations where web access can be assumed to be universal and where an available sampling frame includes email addresses. However, for probability-based general population surveys experience of web data collection remains limited. Many such surveys are now considering the inclusion of web within a mixed-mode design, though few have yet incorporated a web element and there is no consensus on the best way to do this. Meanwhile there is a very small but growing number of single-mode probability-based online panels, using different methodologies.
Survey researchers anticipate several potential benefits from the use of web data collection for general population surveys, such as reduced data collection costs and faster data collection. However, there are significant challenges to be overcome. Major themes include:
Sampling and coverage: How can we design surveys, incorporating web, so that they meet the representativity requirements of general population surveys?
Participation and engagement: What must we learn and do to engage participants in web surveys so that we get high, unbiased response and good quality data? In particular, how do we engage with sub-groups crucial to the success of social surveys - those with poorer access to technology and lower skills, those from disadvantaged and minority groups, etc?
Measurement challenges: How can we best capture complex data using the web and what new opportunities (and related research implications) are there for us to capture new kinds of data?
We welcome submissions to this session that address any of the issues faced by probability-based general population surveys with respect to the inclusion of a web-based data collection element. We particularly welcome reports of findings from experimental or developmental work. We also welcome case studies of general population surveys that have added a web element.
3rd co-ordniator Dr. Caroline Roberts, University of Lausanne, caroline.roberts@unil.ch
Authors: Annelies Blom, Dayana Bossert, Annette Holthausen, Ulrich Krieger
The German Internet Panel (GIP) is based on a true probability sample of individuals living within households and it is therefore the first of its kind in Germany. In 2012 the recruitment of the GIP was conducted offline through face-to-face interviews. Subsequently, all household members were invited to participate in the online panel. After online registration interviewing takes place bimonthly on topics of political and economic behavior and attitudes.
To accomplish representativeness of the general population the GIP has undertaken great efforts to minimize coverage- and nonresponse errors. Households without access to the internet ("offliners") were equipped with the necessary hardware and/or a broadband internet connection and thus were able to participate in the panel.
The analyses that we present span two general questions. First, to what extent does the inclusion of offliners in the GIP reduce coverage error? We investigate this by comparing the characteristics of online households to those of and offline households in the recruitment interview. Second, which characteristics - demographic and attitudinal - of offliners are associated with participation in the GIP? I.e. are households that accept equipment and/or internet access from us and register for the panel different from those who do not?
These analyses clarify the extent to which the inclusion of offliners in probability-based online samples reduces undercoverage and thus contributes to the quality of the achieved sample in terms of its representativeness.
Inspired by American and Dutch experiments, especially the LISS panel, ELIPSS (Etude longitudinale par internet pour les sciences sociales) is a probability-based Internet panel representative of the French population dedicated to social sciences. The participation of panel members consists in filling out web questionnaires for about 30 minutes every month. This paper describes and discusses the recruitment procedure of ELIPSS pilot study.
In France, Internet access at home has tripled over the last ten years but in 2012, 22% of households remain not connected to the Internet. To deal with the non-coverage issue, each panel member receives a computer tablet and a corresponding plan in exchange for his participation. Furthermore, providing mobile terminals and paid plans ought to reduce the non-response and attrition rates.
A pilot study is conducted from 2012 to 2014 and consists of 1500 individuals aged 18-75.
The French National Statistical Institute (INSEE) drew a stratified two-stage probability sample (first municipalities then dwellings) from the 2011 Population Census. A sequential design was implemented to randomly select and recruit individuals within drawn dwellings.
Households were first contacted by mail. A 10 euro unconditional incentive was sent to half of the sample units with the first letter. Non-respondents and some refusals were then reissued by phone (if any) and finally by face-to-face. The paper will study the effects of the different contact modes and will examine the representativeness of the panel by comparing the sample to Census data.
Between 2007 and 2012, we conducted five United States household survey experiments for which a mailing address was the only means of contact. The purpose of these experiments was to systematically test procedures for pushing respondents to answer by web. Each experiment used residential postal addresses from the U.S. Postal Service Residential Delivery Sequence File, which appears to be best household sample frame now available in the U.S. for household coverage. Each test also included multiple experimental panels used to evaluate the effectiveness of a variety of factors on survey response rates and quality. Among the factors tested in these experiments were: mode choice, token cash incentives with the initial response request, additional token cash incentives sent with a subsequent contact, withholding a mail option until late in the contact process, offering mail-only as a response mode (control group), and sponsorship from the same vs. a distant state. The five survey experiments were developed sequentially. An explicit effort was made to learn from the results of early tests, and "add" other treatments onto factors found effective in the early tests, while removing those that were not effective. Response and data quality were examined in terms of: mode effects on response rates, item-nonresponse rates, and representativeness of response. The overarching goal of this cumulative experimentation was to determine whether household samples could be effectively surveyed by web-only, when email contacts could not be used. Implications for other studies will be articulated.
Cultural and technological change has made the web a possible and even desirable mode for complex social surveys. Yet while progress in carrying out large-scale complex social surveys on the web has been made, there is still no consensus about how this can best be achieved while maintaining population representativeness and preserving data quality.
To address this problem, the UK National Centre for Research Methods has funded a network to synthesise and share existing knowledge and catalyse discussion about future possibilities. The network has hosted two events which have drawn on the expertise and knowledge of a wide range of institutions and experts from across the national and international survey research community, including representatives from academia as well as organisations with serious social survey delivery capabilities from government, the private and not-for-profit sectors. The events were organised around three themes: (1) coverage and sampling, (2) participation and engagement, and (3) measurement challenges. In addition to these events, the network has engaged more widely with the international social research community through its online blogs and synthesis papers.