Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Advancing the field of questionnaire translation - identifying problems, discussing methods, pushing the research agenda. A tribute to Janet Harkness 1 |
|
| Convenor | Dr Dorothee Behr (GESIS) |
| Coordinator 1 | Ms Brita Dorer (GESIS) |
| Coordinator 2 | Dr Gijs Van Houten (Eurofound) |
Questionnaire translation is a crucial aspect when it comes to collecting comparable survey data in different countries or among different language groups. In the widest sense, translation should ensure the implementation of 'equivalent' instruments, in different linguistic, cultural and institutional settings.
The quest for equivalence already starts at the questionnaire development stage, in which advance translation can already contribute to a more 'translatable' source questionnaire. It continues in the translation stage through procedures - such as using multiple translators with varying skills and providing them with extensive information and task specification. In the review or assessment stage, committee assessments have been argued to contribute to questionnaire equivalence, more so than the use of 'back translation' which was common practice in many multi-lingual surveys in the past. Pretesting and documentation should round off all these translation procedures (e.g., Harkness 2003).
We invite papers on all aspects related to questionnaire translation. The papers may address more specific (e.g. pertaining to a particular language combination or translation issue) or more general translation issues (e.g., pertaining to scale translation) to draw attention to where problems are located. The papers may also deal with the role of translation in source text development (e.g., advance translation). The papers may equally focus on how challenges are met in terms of methodology (e.g., translation and assessment methods) or software (e.g., translation tools). Last but not least, papers are encouraged which push the research agenda and provide deeper insights into what exactly a good and comparable survey translation is (e.g., effects of different survey translations on the data; translation vs. adaptation; different communication styles across cultures and their translation/adaptation; the notion of "equivalence" itself).
In multilingual surveys, there is a strong trend towards performing more upstream work to reduce the need for downstream corrective action. Along these lines, a new step has been designed and implemented recently, and its output is most promising: newly developed questionnaire items undergo a Translatability Assessment before they are finalised and sent to countries for translation/adaptation.
This Translatability Assessment consists in submitting draft versions of new items to a pool of experienced linguists covering a broad range of language groups. These experts go through the exercise of producing draft translations of those items. These translations are not intended for further use, but they help contributors identify and describe the headaches translators will be confronted with. A set of 12 translatability categories is used to report on the translation, adaptation and cultural issues identified and, whenever possible, alternative wording is proposed. This new formulation proposes a way to circumvent the problem. In some cases, the linguists suggest inserting a translation note to clarify a given term or expression, or to indicate the type of adaptation that may be necessary.
The translatability report is sent to the item developers, who can take this feedback into account: they have the opportunity to eliminate ambiguities, e.g. Anglo-Saxon idiosyncrasies that may be difficult to render in certain languages, double-barrelled questions, cultural issues or unnecessary complexity. In a nutshell, an attempt is made to fine tune the initial version of the items so that it becomes a translatable source version.
Previous analysis of the translation into shared language showed that taking uncentralized approach to the translation, the differences in the translation into shared language in different countries can be very substantial and derived from different reasons. Possible sources of such differences are personal views/ experience of translator; scientific/research traditions; linguistic and socio-linguistic reasons of using languages in different countries for different respondents and also the source text. Source text can be a reason of differences in translations into shared language because of too broad/ unspecific terms; idiomatic or regionally/ nationally specific terms in the source text - incomparable terms; multi-meaning terms allowing multiple interpretations; grammatical structure which allows various interpretations and other reasons.
This paper explores major types of differences in the translation into shared languages on the example of Russian language in large comparative surveys - ESS, ISSP and WVS. Russian language is used in many countries - Russia, Ukraine, Belarus, Moldova, three Baltic countries, Transcaucasian countries, Israel, Middle-Asian countries.
The paper will show that many (although not all) differences in translations into the same language are related to the source questionnaire. Paper will show what kind of differences in the translation due to the source questionnaire can be found in large comparative projects as ESS, ISSP, World Value Survey. Differences in translations into shared languages could serve as an indicator of the problem in the source questionnaire and could be used as a part of advance translation procedure to improve the quality of the sources questionnaire.
Using the same measurement in different cultures, not necessarily mean that we can compare obtained data. It is due to problems with translation and the cultural context.
Schwartz recently proposed a revised theory of basic individual values (Schwartz et al., 2011). It distinguishes 19 more narrowly defined values and it aims to increase the predictive and explanatory power of the original theory of 10 basic values. These values express motivations that can be linked conceptually more precisely and better predict and explain behavior of individual, so by asking individual everyday behaviors we can observe individual's prevailing values.
As a measurement for this purpose Schwartz proposed a new instrument, which was called EBQ (Everyday Behavior Questionnaire). It has been assessed by five judges to be appropriate to the Russian context, ease of translation, and accuracy as expressions of particular underlying values. The main problem in the pretest of this questionnaire was whether respondents interpret the behaviors in it (and underlying values) in the way intended by the author. To reduce the amount of misunderstanding related to translation we used different options: some items were literally translated (adopted) and some items were adapted for Russian context. However we faced the problem of non-working items, when respondents used "no opportunity" answers. We suggest that the problems in translation process connected not only to item level but to concept level (values) too. Results of previous cognitive interviews showed that the meaning of concept's content differs in Russia.
To produce equivalent measurement instruments in cross-cultural survey research, some form of adaptation is likely to be needed in questionnaire translation. Generally speaking, such adaptation amounts to intentional modifications of questions to make them suitable to the new cultural and linguistic setting. In this presentation, I will look at the different forms of adaptation that can affect questionnaire items (e.g., Harkness, 2008; Harkness, 2010; Harkness, Villar, & Edwards, 2011). Moreover, I will relate these to external factors that not only determine but also limit the need and amount of adaptation. These factors are in particular the way how the source questionnaire was designed (comparative perspective from the start vs. only one culture in mind) and the goal of questionnaire translation (comparison in mind vs. no comparison in mind). The presentation shall encourage discussion on the type of changes needed in questionnaire translation and the limits that are determined by comparability needs.