Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Advancing the field of questionnaire translation - identifying problems, discussing methods, pushing the research agenda. A tribute to Janet Harkness 2 |
|
| Convenor | Dr Dorothee Behr (GESIS) |
| Coordinator 1 | Ms Brita Dorer (GESIS) |
| Coordinator 2 | Dr Gijs Van Houten (Eurofound) |
Questionnaire translation is a crucial aspect when it comes to collecting comparable survey data in different countries or among different language groups. In the widest sense, translation should ensure the implementation of 'equivalent' instruments, in different linguistic, cultural and institutional settings.
The quest for equivalence already starts at the questionnaire development stage, in which advance translation can already contribute to a more 'translatable' source questionnaire. It continues in the translation stage through procedures - such as using multiple translators with varying skills and providing them with extensive information and task specification. In the review or assessment stage, committee assessments have been argued to contribute to questionnaire equivalence, more so than the use of 'back translation' which was common practice in many multi-lingual surveys in the past. Pretesting and documentation should round off all these translation procedures (e.g., Harkness 2003).
We invite papers on all aspects related to questionnaire translation. The papers may address more specific (e.g. pertaining to a particular language combination or translation issue) or more general translation issues (e.g., pertaining to scale translation) to draw attention to where problems are located. The papers may also deal with the role of translation in source text development (e.g., advance translation). The papers may equally focus on how challenges are met in terms of methodology (e.g., translation and assessment methods) or software (e.g., translation tools). Last but not least, papers are encouraged which push the research agenda and provide deeper insights into what exactly a good and comparable survey translation is (e.g., effects of different survey translations on the data; translation vs. adaptation; different communication styles across cultures and their translation/adaptation; the notion of "equivalence" itself).
Twenty years ago, the only common method for assessing questionnaire translations was backtranslation. In this two-step process a single translator performs an initial translation, with a second translator translating back into English. Both English versions are compared, supposedly allowing the monolingual researcher to assess translation quality. Discrepancies across versions are investigated. This process is expected to uncover problems in the original translation.
Over fifteen years ago Janet Harkness clearly identified the problems in using backtranslation exclusively. Backtranslation, she observed, uncovers some problems in the original translation, some problems in the backtranslation, yet fails to uncover some problems in both steps. Harkness' developed the TRAPD (translation-review-adjudication-pretesting-documentation) model as an alternative method.
This paper will present results of a study comparing a translation obtained by backtranslation with one done by Committee Approach. This team approach includes the steps in TRAPD and has gained acceptance in the industry since the mid 1990s among federal agencies and university researchers; however, no experiments are published showing how it compares with backtranslation.
For this experiment, a 20-item survey scale will be translated into Polish via Committee Approach. The translation for team discussion prepared by the translator with the strongest credentials will be sent to a backtranslator. A comparison of the backtranslated and the committee version will be performed to determine 1) how each process best identifies translation problems and 2) how each fares in producing a translation that monolingual Polish speakers in a focus group find most idiomatic.
The European Foundation for the Improvement of Living and Working Conditions (Eurofound) carries out three recurring Europe-wide surveys: the European Working Conditions Survey (EWCS), the European Quality of Life Survey (EQLS), and the European Company Survey (ECS). In the latest wave of the ECS - an establishment level telephone survey, which is carried out in 32 countries and in 37 languages or language versions - Eurofound for the first time followed the Translation-Review-Adjudication-Pretesting-Documenting (TRAPD) approach to survey translation. The survey consists of two questionnaires: one for the manager in charge of human resources and one for an official employee representative. The questionnaires were initially translated into German and French, after which the English source questionnaire was reviewed and translation instructions were drafted. The questionnaires were pretested in three countries, using cognitive and structured interviews. After implementing changes based on the pre-test findings, the questionnaires were translated into all 37 languages or language versions. For each language version, the translation was carried out by two independent translators after which an adjudication process took place, led by a third person. In countries where different versions of the same language are used (e.g. Germany and Austria) a cross-national adjudication process was put in place between the country level adjudicators to track unwarranted differences between the language versions. This paper will further outline the approach and will focus on the conditions that proved particularly conducive or obstructive for its efficient and effective implementation.
The Programme for the International Assessment of Adult Competencies (PIAAC) is an international OECD study that compares key competencies of adults (16-65 years) in 24 countries. To implement PIAAC, countries were require to translate an elaborate background questionnaire and the assessment instruments for the domains literacy (including reading components), numeracy, and problem solving in technology-rich environments. Since translation procedures which produce equivalent measurement instruments are crucial to achieving comparability across countries, the PIAAC Consortium specified appropriate procedures in their international standards and guidelines. The translation process for PIAAC Germany will be described and reflected upon.
Sharing a common language. The constraints of joint questionnaire translation.
Translating an hour-long questionnaire for multi-nation use is by definition a hard task. Achieving equivalence of the particular questions, in question, is even harder. The translation process requires significant knowledge of "a blend of language, language use and culture" (Braun, Harkness 2005). What happens in the case of countries sharing a common language? Do joint translating efforts reduce the burden? or duplicate the strain? What are the implications for the quality of the questionnaire? What are the effects of translation verification? What are the effects on comparability when shared language spans more than two countries?
To answer these questions insights from the Greece- Cyprus joint translation of the 5th round European Social Survey questionnaire (2011) will be used. Time and human resources constraints will be discussed along with a critical assessment of the TVFF monitoring tool.