Keynotes

Edith de Leeuw: Mixed Mode: Past, Present, and Future

Abstract

One of the earliest mentions of mixed-mode in a methodological monograph was by Dillman and Tarnai in 1988. Mixing survey modes appears almost inevitable today, especially for academic research and official statistics. Only now online surveys have almost replaced telephone surveys in the mix. Furthermore, in international comparative surveys, mixed-mode is almost inevitable, as countries differ in survey technology and customs.

Just as in 1988, today there are three important reasons to use a mixed-mode survey design: improving coverage, increasing response rates and reducing costs. However, there are also potential drawbacks, such as, increased administrative and logistic burden, and potential for mode specific measurement error. From a Total Survey Error perspective one wants the best of all worlds and reduce overall error. The steps involved in a high quality mixed mode study are: design, diagnosis, and adjustment. Early methodological publications mainly focused on design (De Leeuw, 2005; Dillman, 2000). During diagnosis it is important to discern between desired differential selection effects, which help reduce coverage and nonresponse error, and unwanted differential measurement effects. Only then one can estimate the unwanted mode measurement effect while controlling for the wanted selection effects, and if necessary adjust for unwanted differential measurement error (e.g., Tourangeau, 2017; Hox et al., 2017).

A new technological challenge facing survey researchers are mobile devices, such as smart phones and tablets, which are increasingly being used to access the Internet. Web surveys are now morphing from a computer-oriented into a multi-device oriented concept, and although a mixed-device survey is not a mixed-mode survey in the traditional sense of the word (all devices are self-administered), the devices used vary widely in screen sizes, data entry interface, and social customs in use. Therefore, the question arises whether or not answers obtained via smartphone and tablets are comparable to answers obtained from pc or laptop, and how to design high quality mixed-device surveys.

In this presentation, I will summarize the state of the art in traditional mixed-mode surveys and discuss implications for mixed device surveys.

Bio

Edith Desiree de Leeuw is MOA-professor in survey quality at the Department of Methodology and Statistics at Utrecht University. She was a Fulbright scholar with Don Dillman at Washington State University and visiting scholar with Jan de Leeuw (no relative) at the Program on Social Statistics, UCLA. She is a fellow of the Interuniversities Joint Institute for Psychometrics and Sociometrics (IOPS) in the Netherlands, and was awarded the Visiting International Fellowship at the Institute of Social Research, University of Surrey. Edith has over 140 scholarly publications and is co-editor of 4 internationally renowned books on survey methodology: The International Handbook of Survey Methodology, Total Survey Error in Practice, Advances in Telephone Methodology, and Survey Measurement and Process Quality. Her recent publications focus on design and analysis of mixed-mode studies, online surveys, nonresponse, and total survey error. Edith is currently associate editor for the Journal of Official Statistics (JOS) and editor of MDA (Methods, Data, Analyses); she is also on the editorial and scientific boards of international journals in the field of survey methodology, such as IJPOR, Field Methods, SMR, & BMS.

References

De Leeuw, E. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2): 233-255.

Dillman, D. A. & Tarnai, J. (1988). Administrative issues in mixed mode surveys. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, & J. Waksberg (Eds.), Telephone Survey Methodology (pp. 509-528). New York: John Wiley & Sons.

Dillman, D. A. (2000). Mail and Internet surveys: The Tailored Design Method. New York: John Wiley & Sons.

Hox, J., de Leeuw, E., & Klausch, T. (2017). Mixed-Mode Research: Issues in Design and Analysis. In P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, N. C. Tucker, & B. T. West (Eds), Total Survey Error in Practice (pp. 511-530). New York: John Wiley & Sons.

Tourangeau, R. (2017). Mixing Modes: Tradeoffs among Coverage, Nonresponse, and Measurement Error. In P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, N. C. Tucker, & B. T. West (Eds), Total Survey Error in Practice (pp.115-132). New York: John Wiley & Sons.

Beth-Ellen Pennell: Trends and Developments in Multinational, Multiregional, and Multicultural (3MC) Surveys

Abstract

This presentation will discuss trends and new developments in 3MC survey research while highlighting the unique and challenging dimensions of this research through a Total Survey Error (TSE) model adapted for 3MC studies. The 3MC TSE framework has also been expanded to incorporate other aspects of the survey lifecycle (see Pennell et al., 2017). The presentation will also discuss the opportunities brought about by the diffusion of affordable technology to low resource settings. Data collection technologies that have been widely used in high income countries are increasingly being adapted and used in new contexts. This transfer of technology is facilitating new approaches to quality control, including the collection of rich paradata, as well as other innovative applications. In addition to immediate access to the survey and process data (including call records), self-administered modes such as ACASI are being used in very diverse contexts, as are the use of digital recordings, global positioning systems (GPS), areal photography, digital photography and fingerprinting, among other examples. However, these innovations do not come without challenges. These developments and challenges will be illustrated using examples from large scale surveys 3MC surveys. The presentation will also draw upon examples from newly published resources (see e.g. Johnson et al. forthcoming; CCSG 2016).

Bio

Beth-Ellen Pennell is the Director of International Survey Operations of the Survey Research Center of the Institute for Social Research at the University of Michigan. Ms. Pennell has 40 years of experience in survey research operations and methods research. Pennell also serves as the Director of the Data Collection Coordinating Centre for the World Mental Health Survey Initiative, a joint project of the World Health Organization, Harvard University and the University of Michigan. In the position, Ms. Pennell coordinates the technical support and oversees the implementation of the data collection activities for these general population epidemiological studies in more than 30 countries.

Pennell is an elected member of the International Statistical Institute, led the development of the Cross-cultural Survey Guidelines (see CCSG 2016) and was one of the editors of Survey Methods in Multinational, Multiregional and Multicultural Contexts (see Harkness et al. 2010), which won the 2013 American Association for Public Opinion Research (AAPOR) annual book award for outstanding contributions to survey research methodology. Ms. Pennell is also chair of the Comparative Survey Design and Implementation Workshop (see CSDI 2012) and co-editor of the monograph “Advances in Survey Methods in Multinational, Multiregional and Multicultural Contexts” (see Johnson et al. forthcoming).

References

CCSG (2016). Cross-Cultural Survey Guidelines. Available from http://ccsg.isr.umich.edu

CSDI (2012). Comparative Survey Design & Implementation. Available from https://www.csdiworkshop.org/

Harkness, J. A., Braun, M. Edwards, B., Johnson, T. P., Lyberg, L. E., Mohler, P. Ph., Pennell, B.-E., & Smith, T. W. (2010). Survey Methods in Multicultural, Multinational, and Multiregional Contexts. Hoboken, New Jersey: John Wiley & Sons.

Johnson, T., Pennell, B.-E., Stoop, I., & Dorer B. (forthcoming). Advances in Survey Methods in Multinational, Multiregional and Multicultural Contexts. Hoboken, New Jersey: John Wiley & Sons.

Pennell, B.-E., Cibelli Hibben, K., Lyberg, L. E., Mohler, P. Ph., & Worku, F. (2017). A Total Survey Error Perspective on Surveys in Multinational, Multiregional, and Multicultural Contexts. In P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, N. C. Tucker, & B. T. West (Eds.), Total Survey Error in Practice (pp. 179-202). Hoboken, New Jersey: John Wiley & Sons.