Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Advancing the field of questionnaire translation - identifying problems, discussing methods, pushing the research agenda. A tribute to Janet Harkness 3 |
|
| Convenor | Dr Dorothee Behr (GESIS) |
| Coordinator 1 | Ms Brita Dorer (GESIS) |
| Coordinator 2 | Dr Gijs Van Houten (Eurofound) |
Questionnaire translation is a crucial aspect when it comes to collecting comparable survey data in different countries or among different language groups. In the widest sense, translation should ensure the implementation of 'equivalent' instruments, in different linguistic, cultural and institutional settings.
The quest for equivalence already starts at the questionnaire development stage, in which advance translation can already contribute to a more 'translatable' source questionnaire. It continues in the translation stage through procedures - such as using multiple translators with varying skills and providing them with extensive information and task specification. In the review or assessment stage, committee assessments have been argued to contribute to questionnaire equivalence, more so than the use of 'back translation' which was common practice in many multi-lingual surveys in the past. Pretesting and documentation should round off all these translation procedures (e.g., Harkness 2003).
We invite papers on all aspects related to questionnaire translation. The papers may address more specific (e.g. pertaining to a particular language combination or translation issue) or more general translation issues (e.g., pertaining to scale translation) to draw attention to where problems are located. The papers may also deal with the role of translation in source text development (e.g., advance translation). The papers may equally focus on how challenges are met in terms of methodology (e.g., translation and assessment methods) or software (e.g., translation tools). Last but not least, papers are encouraged which push the research agenda and provide deeper insights into what exactly a good and comparable survey translation is (e.g., effects of different survey translations on the data; translation vs. adaptation; different communication styles across cultures and their translation/adaptation; the notion of "equivalence" itself).
According to the Ask-the-Same-Question model, the most important consideration is to ensure equivalence of translated questionnaires and their components. However, as Janet Harkness pointed out "[...] close translation is a balancing act." (Harkness et. al. 2010, p. 47). Our paper will look at the problems posed by translation of numeric-graphic scales where ends of scales are described verbally. Such scales have been commonly used in many cross-national surveys (e.g. the European Social Survey). In particular, we will analyse verbal descriptors of extreme ends of scales, especially looking at modifying adverbs/adverbial phrases (such as extremely, completely, very etc.). The practice for many surveys has been to use the same modifier at both ends of scales (e.g. extremely bad vs. extremely good, or extremely satisfied vs. extremely dissatisfied). However, translation of those modifiers into other languages has revealed a number of problems (some phrases do not translate well, different modifiers have to be used even if the same word was used in the original English questionnaire etc.). It seems that linguistic issues such as semantic prosody may be at play here: some modifiers work well with 'positive' words but not so well with 'negative' ones and vice versa. Our query into English and Polish linguistic resources (monolingual corpora and dictionaries) seems to confirm this intuition.
In our paper we will discuss the findings and explore if and how corpora may provide a useful source to address the problems mentioned above.
Different authors consider that competencies of survey instruments' translation are an important premise for good measurement properties of translated questionnaire. Incorrectly translated questionnaires can result in the whole research project failure due to the lack of methodological quality. Nevertheless, only few attempts have been made to produce some empirical validations.
The author of this article formulated "a model of 4 competencies", while looking for an answer to a question as to what determined good results of the questionnaires' translation and cross-cultural adaptation. The suggested translation organization technique (based on "the model of 4 competences") stipulates that competencies of a researcher-translator should converge towards the "ideal" profile of two cultural and two academic-professional competencies. That is - the linguistic competence, the cultural competence, the methodological competence in survey, the competence in a measured diagnostic construct. The duality of the competencies is essential - practical and theoretical knowledge of a subject.
The effectiveness of suggested translation organization technique was tested with empirical social research. It was supported with biographical survey and interview with the scholarly group who created, applied and tested the above mentioned technique. Linguistic adaptation of questionnaires and the quality of the cross-cultural transfer was tested with psychometric statistics and methodological re-analysis of five different surveys. The methodological quality of the adapted questionnaires was confirmed from analysis of 665 primary indicators and 76 scales (when N=5516). That allowed empirically recognize that the suggested methods as effective.
Introduction: Quantitative migrant health researchers often apply questionnaires validated only for the source populations of migrants. However, after years in the host country, migrants may differ in cultural and language characteristics from the source population. This may limit the transferability of questionnaires. Based on a comprehensive analytical framework of equivalence covering conceptual, semantic and measurement characteristics, the transferability of the Turkish version of the Revised Illness Perception Questionnaire (IPQR) (part II) between Turks residing in Turkey and Turkish migrants in Germany is examined as an illustrative example.
Methods: The measurement equivalence of the IPQR was examined through multigroup confirmatory factor analysis based on data from 602 patients surveyed in Turkey and Germany. Conceptual and semantic equivalence were studied by qualitative interviews with patients (n=40 and 58, respectively) and experts (n=6 and 8, respectively) and by systematic literature reviews.
Results: The indicator-factor pattern of the IPQR differed between both samples and only a partial configural model could be established. Subsequent analyses, additionally, revealed differences in factor loadings and intercepts. While there was no evidence of a different conceptual frame between both populations, several sources of semantic nonequivalence could be identified, going back to complex wordings of certain items.
Conclusion: The study illustrates possible areas of nonequivalence in the transfer of questionnaires between source and migrant populations, some of which go back to differences in language usage. They need to be considered in the readaptation of questionnaires for which the analytical framework of equivalence can be a useful tool.
Throughout all stages of the cross-national survey life cycle there are threats to measurement equivalence. Errors can occur during the drafting, pre-testing or translation stages of questionnaire design as well as during administration itself.
In order to address potential inaccuracies in its questionnaire translations, the European Social Survey (ESS) has included an additional 'verification' stage in rounds 5 and 6. cApStAn, an external company specialised in linguistic quality control of survey instruments, was appointed to verify a selection of items taken from the translated questionnaires in each participating country. This process has involved verifiers from outside the original translation teams checking the translations for linguistic accuracy and equivalence to the source questionnaire.
This paper will compare the different strategies used to implement the verification process during ESS rounds 5 and 6 and provide an assessment of the contribution of verification to date. The types of translation problems identified by the verifiers in each round will be discussed and implications for both ESS national teams and for source questionnaire designers in general will be assessed. In particular, the overall contribution made by verification to ongoing efforts to improve translation quality of ESS items in terms of (a) general translation issues, (b) language-version-specific translation issues, (c) term-specific translation problems, and (d) equivalence between all translations in this cross-national survey will be discussed.