Adaptation and Administration of Questionnaires in Surveys on Migrants: Challenges and Solutions |
|
| Session Organisers |
Professor Patrick Brzoska (Martin Luther University Halle-Wittenberg, Faculty of Medicine) Dr Dorothée Behr (GESIS – Leibniz-Institute for the Social Sciences) Dr Steffen Pötzschke (GESIS – Leibniz-Institute for the Social Sciences) |
| Time | Thursday 18th July, 16:00 - 17:30 |
| Room | D11 |
Migrants constitute increasingly large proportions in many European and non-European countries. While some have lived in the host countries for several decades, others are recent immigrants, the latter of which also include refugees and asylum seekers. In order to devise and evaluate appropriate policies and services, survey data on this population is of high relevance. Given migrants’ wide ethnic, cultural, and linguistic diversity, survey research among migrants is associated with different challenges that encompass the entire research process and that go beyond the problems encountered in cross-national and cross-cultural survey research in general. Migrants are not only difficult to define and hard to reach, which makes the application of sophisticated sampling techniques necessary to collect high-quality data. One particular challenge concerns the application of quantitative questionnaires itself: As many migrants have limited proficiency of the host country’s language, it is usually necessary to also provide questionnaires in their ‘mother tongue’. Frequently used questionnaires or scales are often available in several language versions. However, because of differences in language development over time, migrants may have problems understanding these questionnaires in case they have only been adapted for the population of their countries of origin. As a consequence, research instruments developed for source populations may perform differently when administered to the respective migrant communities and therefore need to be re-adapted to the language style, level of language proficiency, and the cultural background of migrants without compromising their overall properties and their cross-group comparability. Substantial cultural and linguistic adaptations may also be necessary if questionnaires are to be administered in the language of the host country. These challenges need to be addressed by appropriate strategies to account for different forms of bias and to ensure high-quality survey data. This is not only relevant for researchers specialized in research on migrants. In an increasingly diverse society, also general surveys need to increasingly employ adequate approaches in order to include groups of individuals who are a significant part of a country’s society.
For this session, we welcome contributions illustrating novel methodological approaches that address (language-related) problems involved in research on migrants in terms of the adaptation (including translation) and administration of questionnaires. We invite researchers to share their experiences in these regards. Topics may cover different phases of the research process, such as instrument design and translation/adaptation/pretesting, field work, and data analysis.
Keywords: migrants, questionnaires, adaptation, translation, language
Dr Anna Sandoval Giron (United States Census Bureau) - Presenting Author
Dr Patricia Goerman (United States Census Bureau)
The US Census Bureau conducts research to improve data collection with non-English speakers. This paper is part of a larger study that tested and produced recommendations to improve doorstep messaging for hard-to-count non-English speakers in the United States. The framework for this study is based on prior literature that examined census participation mindsets. A mindset refers to a group’s shared attitudes, knowledge and perceptions. One of these mindsets describes respondents with a language barrier, those with limited or low English-language proficiency. In the U.S. about 21% of the population speaks a language other than English (U.S. Census Bureau, 2012-2016 American Community Survey 5-Year Estimates). One of the tools that the Census Bureau developed to aid in the communication with non-English speaking respondents is the Language Identification Card. The card is designed to help enumerators identify the language a respondent speaks in order to provide appropriate follow-up.
Our team conducted 42 focus groups in seven languages: Arabic, Chinese, English, Korean, Russian, Spanish, and Vietnamese. We explored respondents’ reactions to videos that mimicked common scenarios, one of the scenarios portrays an English speaking enumerator interacting with a non-English speaking respondent using the Language Identification Card. Afterwards focus groups discussed the Language Identification Card. Findings show that some respondents found the card overwhelming, had difficulty finding their language, and expressed concern about those with low literacy across languages. However, overall respondents welcomed the use of the card, praised the card for inclusion of multiple languages, and interpreted the card as a sign that the Census Bureau cares about ethnic and linguistic minorities. As a result of these findings, changes were implemented that are intended reduce overall language barriers during the 2020 Census. Other organizations may be able to learn from our methods and our findings to overcome similar language barriers.
Mr John De Maio (Australian Institute of Family Studies)
Dr Pilar Rioseco (Australian Institute of Family Studies)
Dr Galina Daraganova (Australian Institute of Family Studies) - Presenting Author
Building a New Life in Australia (BNLA) is a ground-breaking longitudinal study that aims to better understand how humanitarian migrants settle into life in Australia. More than 1,500 individuals and their families (totalling close to 2,400 respondents) who had been granted a permanent humanitarian visa to live in Australia participated in the first interview in 2013. Participants were residing in urban and regional communities across Australia. Survey and participant materials have been translated into 14 languages. Five annual waves of data collection have taken place between 2013 and 2018, with alternating waves of home visits (Waves 1,3 and 5) and telephone interviews (Waves 2 and 4). In Waves 1,3 and 5, the survey was administered via Computer Assisted Self Interview (CASI) on a computer tablet. Participants who preferred to complete the survey with an interviewer were also offered a Computer Assisted Personal Interview (CAPI) option. Participants could also be assisted by an accredited interpreter over the phone or in person.
This paper describes in more detail the methodological challenges of collecting data from a study population that is both extremely diverse and highly vulnerable. Participating families have arrived in Australia from 35 countries and spoke close to 50 different languages. Some participants had low literacy and education levels. This created complex challenges for data collection and required innovative solutions to address these challenges. Strategies including the use of Community Engagement Officers (respected members of local migrant communities) who were recruited to promote and advocate for the study will be discussed. The challenges associated with the translation of survey and participant materials will also be explored in the paper.
Mrs Laura Treskow (Criminological Research Institute of Lower Saxony) - Presenting Author
Since 1970s migration studies are an important topic in social science. The focus of the scientific investigations is mainly on labor immigrants and the second generation of immigrants. Especially since the years 2014 and 2015, refugees have been increasingly finding a new place to stay in Germany.
In 2016, the Criminological Research Institute of Lower Saxony (KFN) started the research project “Everyday experiences and settings of refugees in Lower Saxony (ALFiN)”. The study aims to compile a picture of the situation among refugees recently arrived in Lower Saxony. Important topics of the study are: motives for the immigration, the journey to the west, and the consequences of flight for refugees. Refugees were asked about their expectations for the future and about their values. A further major focus is the everyday experience of refugees in Germany. A key point of attention in this regard is refugees’ social setting.
This presentation is based on a quantitative survey about refugees in Lower Saxony (Germany). It is especially focused the motivation of interviewers who are asylum seeker, hence not yet officially accepted and how this uncertain status seem to influence the willingness to participate (or the participation in generally). In addition, the presentation will inform about the accomplishment of a standardized survey in collective accommodations and the challenges social scientists have to expect. A particular note is the proper handling with different reading and writing competences. The presentation informs about possible ways to deal with these challenges.
Dr Giuseppe Pietrantuono (Research Center German Federal Office for Migration and Refugees) - Presenting Author
Dr Nina Rother (Research Center German Federal Office for Migration and Refugees)
Host country language acquisition plays a pivotal role in immigrants' integration. Language is not only important in itself, but acts also as an important multiplier for labor market integration, social participation, and feelings of belonging (Esser 2006, Chiswick/Miller 1995, Dustmann/Fabbri 2003, Portes/Rumbaut 2006).
For integration scholars, a valid assessment of immigrants' language skills is of considerable importance. Due to the lack of scaled and short test procedures, surveys almost without exception rely on (more or less extensive) self-assessments. However, often the validity of these self-assessments is questioned. Previous studies on the validity of (foreign) language self-assessments often focus on a specific population (e.g., university students or pupils) (Rossi 1998, Edele 2015). It remains an open question to which degree these findings are generalizable to adult immigrants, with heterogeneous educational backgrounds.
Our contribution presents the results of a validation study conducted in Germany among participants of general integration courses in 2018. We asked approximately 200 participants at the beginning or the end of the course to self-assess their German language skills. We used both a rough as well as a detailed self-assessment scale to measure their German language skills. Additionally, we collected individual factors that theoretically confound the language self-assessment. We linked these survey data to objective test results (A1 test scores for participants at the beginning of the course and A2/B1 test scores for respondents at the end of the course) so that the validity of the self-assessments and possible influencing factors can be examined. To our knowledge, this contribution offers the first insight into the relationship between a subjective and an objective measure of language skills for a population, which is the focus of many surveys.
For future surveys, the results show how self-assessment can be used to provide a valid measure of individual language skills.