Questionnaire Translation: Achievements and New Challenges Ahead 2 |
|
| Session Organisers |
Ms Brita Dorer (GESIS-Leibniz Institute for the Social Sciences) Dr Alisú Schoua-Glusberg (Research Support Services, Chicago) |
| Time | Wednesday 17th July, 14:00 - 15:00 |
| Room | D23 |
Questionnaire translation is a field within cross-cultural survey research that has been receiving increasing interest within the past decades, as the importance of high-quality and comparable questionnaire translations has become more and more obvious for 3mc studies, both cross national and within country cross-cultural research. While in the earlier stages, approaches such as simply back-translating translated questionnaires and comparing to the source text were routinely applied, ever more sophisticated methods have been developed over the last 10-20 years. Team or committee approaches, as implementations of Janet Harkness' TRAPD model, (e.g. by the European Social Survey (ESS) or SHARE), have become the norm in many multilingual projects. New approaches for assessing and improving translation quality have been added, and research on how to make use of new technological developments in translation sciences, such as Computer-Assisted Translation (CAT) tools, Machine Translation (MT), or Speech Recognition, has been under way.
This session invites papers on a wide variety of aspects of questionnaire translation. Examples of topics may include topics or research questions from within the classical linguistic and translation research fields, such as certain translation issues or linguistic patterns in individual language pairs; discussing existing or new methods and tools for assessing or improving translation quality; aspects of the source questionnaire that affect its translation into multiple language versions; the interplay between translation and adaptation in the context of questionnaire translation; or intercultural factors affecting questionnaire translation in the cross-cultural survey context.
Keywords: Questionnaire translation, comparability, translation tools, translation quality
Mr Patrick Italiano (University of Liège)
Mr Alexandre Pollien (FORS - University of Lausanne)
Mr Simon Le Corgne (CDSP - Sciences Po) - Presenting Author
The European Social Survey (ESS) is an academically driven cross-national survey conducted every second year across Europe, in which Belgium, France and Switzerland all participate since the first round in 2002. Since ESS measures attitudes, beliefs and behaviours of populations in over thirty countries, comparative analysis between the different national contexts and over time is crucial and methodologically demanding.
In order to ensure the validity of the comparison, a key methodological condition relates to translations. ESS uses pragmatically equivalent questionnaires, that is translations aimed at maintaining the intended measurement properties rather than literal translations. The source questionnaire is designed in English and translated following the TRAPD methodology (Translation, Review, Adjudication, Pretesting and Documentation). After each country has reviewed two own, independent translations, a harmonization meeting takes place between countries sharing the same language. For the French version of the questionnaire, Belgium, France and Switzerland are involved. The aim of this meeting is to agree on a common French questionnaire as similar as possible and understandable in the same way across the three countries.
Through a deep and detailed review of the work during these harmonisation sessions, this presentation proposes to take a critical look at the issues in need of decision-making during the process toward a shared French version. Over the years, we have been confronted with multiple and varied situations, which have implied concerted decision-making regarding the harmonization of translation.
Through a typology of the discrepancies encountered, and the associated decision-making procedures, choices toward an expected convergence or differences to be preserved, we intend to propose a descriptive analysis of our practice aimed at identifying sources of difficulties in the translation process, in order to help the upstream work of disambiguating the source questionnaire and to guide the translation instructions.
Dr Anna Andreenkova (CESSI (Institute for comparative social research)) - Presenting Author
In comparative cross-national surveys the harmonisation of translations into shared languages (the same language used in different countries) is important part of the translation process. For example, in ESS six countries translate questionnaire into Russian, three countries - into German and three countries - into French. Previous analysis showed that taking uncentralized approach to the translations, the differences in translations into shared languages can be very large. Current approach to the harmonisation of different translations is based mostly on expert discussions between countries which sometimes administratively difficult or lacks the empirical validation. The paper presents the experiment where the expert discussion about the harmonisation was supplemented by cognitive interviews. The experiment was conducted using the translation from English into Russian in three different countries - Kazakhstan, Moldova and Russia. The translation passed full TRAPD cycle in each country. The resulted versions were compared and different types of differences – grammar, lexical and semantic – were revealed. Then cognitive interviews were conducted in each country - competing versions of the same question were presented and discussed with respondents in all 3 countries. The purpose of this stage was two-fold a) to separate country (culture)-specific differences in translations from linguistic-driven differences; b) to find evidences which version of the question with linguistic differences is perceived better in each country. The experiment showed that using both expert harmonisation discussion and cognitive interviews about the perception of different question versions allowed to eliminate substantial part of differences in translations into shared language and also do it with more confidence if based on expert discussion only.
Mr Steve Dept (cApStAn) - Presenting Author
For the past two decades, the literature has hailed team translation as the gold standard in survey translation. Indeed, this design has been implemented in numerous research projects, such as the ESS, with a good measure of success in terms of cross-language comparability. In the same two decades, technology has achieved spectacular progress. Two areas where such advances have been observed are translation technology - with a special focus on computer-assisted translation tools and automated quality assurance tools - and survey administration platforms.
Has there been a dialogue between survey methodologists who advocate and implement best practice, translation technologists who use and apply new tools, and platform engineers who have designed and streamlined new questionnaire administration systems?
The question this paper raises has non-negligible financial and quality implications for international surveys of today and tomorrow: best practice in survey translation was tried and tested for certain modes, first PAPI, then CATI, then CAPI, largely by-passing translation technology. Attempts have been made to tweak platforms to accommodate this best practice by mirroring existing workflows in technology-rich environments. Some platforms have attempted to replicate and integrate functionalities from translation technology. And some CAT tools have been adapted to host team translation workflows. However, one of the main challenges encountered in integrating the wisdom and the expertise from these three different areas is that efforts tend to concentrate either on duplicating processes in a new technological environment or on modifying a technological environment to accommodate an existing set of processes. Little attention is given to exploring result-oriented processes that would make the most of new advances in technology.
In this paper, the authors will examine two alternative workflows that allow a better integration of needs and requirements from all three sides, without increasing the overall translation costs and without compromising translation quality.