Questionnaire Translation: Achievements and New Challenges Ahead 3 |
|
| Session Organisers |
Ms Brita Dorer (GESIS-Leibniz Institute for the Social Sciences) Dr Alisú Schoua-Glusberg (Research Support Services, Chicago) |
| Time | Wednesday 17th July, 16:30 - 17:30 |
| Room | D23 |
Questionnaire translation is a field within cross-cultural survey research that has been receiving increasing interest within the past decades, as the importance of high-quality and comparable questionnaire translations has become more and more obvious for 3mc studies, both cross national and within country cross-cultural research. While in the earlier stages, approaches such as simply back-translating translated questionnaires and comparing to the source text were routinely applied, ever more sophisticated methods have been developed over the last 10-20 years. Team or committee approaches, as implementations of Janet Harkness' TRAPD model, (e.g. by the European Social Survey (ESS) or SHARE), have become the norm in many multilingual projects. New approaches for assessing and improving translation quality have been added, and research on how to make use of new technological developments in translation sciences, such as Computer-Assisted Translation (CAT) tools, Machine Translation (MT), or Speech Recognition, has been under way.
This session invites papers on a wide variety of aspects of questionnaire translation. Examples of topics may include topics or research questions from within the classical linguistic and translation research fields, such as certain translation issues or linguistic patterns in individual language pairs; discussing existing or new methods and tools for assessing or improving translation quality; aspects of the source questionnaire that affect its translation into multiple language versions; the interplay between translation and adaptation in the context of questionnaire translation; or intercultural factors affecting questionnaire translation in the cross-cultural survey context.
Keywords: Questionnaire translation, comparability, translation tools, translation quality
Ms Brita Dorer (GESIS-Leibniz Institute for the Social Sciences) - Presenting Author
Mr Maurice M. Martens (CentERdata-University of Tilburg)
The Translation Management Tool (TMT) is an online service for supporting questionnaire translation processes for large multilingual surveys, originally developed for SHARE. It was adapted to be usable for supporting other large-scale multilingual surveys such as the ESS and EVS.
The ESS has been using the TRAPD approach rigorously since its first round, and translation verification by the external service provider cApStAn since its 5th round. In the context of the SERISS cluster project, the TMT was used for translations into a subset of language versions in the 8th and 9th rounds of the ESS for supporting these processes.
This paper will discuss the findings from these two testing rounds in the ESS: which were the benefits, which the challenges from using this online-platform? Did they differ between the different user groups? What does this tool mean with regard to quality, both of each individual final translation as well as the overall quality for the whole survey? Does, for instance, the increased complexity of using an online platform outweigh benefits in terms of documentation or ease of online communication? Can the TMT be combined with computer-assisted translation (CAT) tools or machine translation, both of which have become state of the art in the general translation business?
Translations are of key importance for large international studies, and the translation processes for producing these translations are important in safeguarding the quality of the questionnaire. For assessing the contribution that the TMT can make to improve these processes, the challenges in further developing this technology should be discussed from different points of view. In the end, the different interests of the disciplines and parties involved will be considered: the technical development side, translation studies perspective, as well as survey operations.
Mr Steve Dept (cApStAn)
Mrs Elica Krajceva (cApStAn) - Presenting Author
Since G. Almond and S. Verba launched the Political Attitudes and Democracy in Five Nations Study in 1963, the most widespread model for 3MC surveys has long been to design questionnaires in English, to go through cognitive testing, to pilot, and to leave translation and adaptation for the end. Meanwhile, however, best practice prescribes embedding translation and adaptation in questionnaire design. How is this done? How does it benefit to comparability?
Since 2013, an exercise referred to as "translatability assessment" has been introduced in International Large Scale Assessments (ILSAs) such as PISA and PIAAC, in attitudinal studies (e.g. Pew Research Center's Global Attitudes Project), or in European surveys such as ESENER-2, EU-MIDIS, EQLS or the European Company Survey (ECS). A translatability assessment features interaction between questionnaire authors and cultural brokers, who can be linguists and/or domain specialists from selected target cultures.
A near final draft questionnaires is submitted to a panel of such cultural brokers. They may produce translations, not intended for further use but to identify potential adaptation problems inherent to the construct, the target culture, or the wording of the source. They use a framework of translatability categories to describe the issues they identify. At the same time, the draft questionnaire undergoes an automated screening to identify patterns that have been reported in earlier surveys. All this feedback is then collated by a senior linguist at cApStAn, who produces a consolidated translatability report, which goes back to questionnaire authors.
Translation and adaptation notes are drafted and, when this seems warranted, alternative wording is suggested, without loss of meaning. Several iterations may be required, and this increases the preparation time for the master questionnaire. However, it is a relatively small investment, and researchers who have participated in this exercise recognise the considerable minesweeping benefits of this early reflection on the portability and adaptability of
Ms Brita Dorer (GESIS-Leibniz Institute for the Social Sciences) - Presenting Author
In multilingual surveys, source questionnaires are the cause of a number of translation problems or errors during the final translation phase. For minimising such problems, the ESS has carried out, since its 5th round, systematic ‘advance translations’ in order to detect such problems before finalising the source text. For doing so, national ESS translation teams, consisting of both translators and survey researchers, carry out translations of a pre-final version of the source questionnaire, with the purpose of spotting translation problems. These comments are considered when finalising the source text. The problems pointed out range from intercultural adaptation issues to, for instance, grammatical or syntactical structures requiring complicated translations into a specific target language that may have a negative impact on the comparability between all resulting final translations when fielded. Changes in the source text triggered by advance translation range from rephrasing source text elements to adding footnotes to explain ambiguous source text terms.
The author tested the usefulness of this method in a series of think-loud tests: experienced questionnaire translators translated 22 items – in their version before and after the advance translation – into French and German. The think-aloud protocols were analysed both qualitatively and quantitatively. The usefulness of advance translation for enhancing the translatability of the source text was confirmed in this think-aloud study.
This paper describes the method of advance translations as applied in the ESS, some typical source questionnaire issues detected, as well as the think-aloud study and its results.
Conclusions will be drawn on questions like: does the success of advance translation depend on the source text problems detected or on the changes made because of advance translation? Which role does the interplay of languages of advance translation and think-aloud study play? Is think-aloud a valid method for evaluating advance translation? Which best-practice guidance results from this study?