ESRA 2025 Preliminary Program
All time references are in CEST
Questionnaire translation in a changing world: challenges and opportunities 2 |
| Session Organisers |
Dr Alisú Schoua-Glusberg (Research Support Services)
Dr Brita Dr. Dorer (GESIS-Leibniz Institute for the Social Sciences)
Dr Dorothée Behr (GESIS-Leibniz Institute for the Social Sciences)
|
| Time | Wednesday 16 July, 14:00 - 15:00 |
| Room |
Ruppert paars - 0.44 |
With the world becoming more and more globalised and population demographics changing at an ever higher pace, a good quality level of questionnaire translations is increasingly important to develop reliable cross-cultural data.
The digital transition has also been entering the fields of translation in general and questionnaire translation in particular: While good practice recommendation in cross-cultural survey methodology is still to apply team approaches, making sure to involve appropriately trained and experienced questionnaire translation experts, ideally TRAPD (consisting of the steps Translation, Review, Adjudication, Pretesting and Documentation), digital innovations such as Machine Translation or Artificial Intelligence are more and more entering the fields of (questionnaire) translations. Technical innovations related to questionnaire translations can have manifold forms, such as platforms allowing a smooth handling of the workflow to develop, translate and later field questionnaires in manifold languages. Or, for instance, crowd-based translation schemes become more and more popular and may contribute to translating questionnaires too. Where are the strengths and where the weaknesses of such developments? Which role should AI play in the translation of survey instruments?
This session addresses various aspects of questionnaire translation, whether related to digital innovations or not. We invite papers on various topics related to the translation and interpretation of survey instruments, referring for example to: experiments on new techniques or alternative methods, studying challenges of specific language pairs or translation methods. How to translate certain questionnaire elements, such as translating answer scales, approaches to develop questionnaire translations of minority languages, or comparing effects of different translation quality assessment approaches on survey data.
Keywords: Questionnaire translation, cross-cultural surveys, survey methodology, digitalisation
Papers
The Influence of Translator Backgrounds on Survey Measurements: Evidence from a Survey Experiment
Miss Chia-Jung Tsai (GESIS – Leibniz Institute for the Social Sciences) - Presenting Author
Dr Clemens Lechner (GESIS – Leibniz Institute for the Social Sciences)
Dr Dorothée Behr (GESIS – Leibniz Institute for the Social Sciences)
Miss Ulrike Efu Nkong (GESIS – Leibniz Institute for the Social Sciences)
Dr Anke Radinger (GESIS – Leibniz Institute for the Social Sciences)
Questionnaire translation is essential for transferring survey instruments to different languages and cultural contexts, but it can unintentionally affect how questions are understood and measured. These changes may impact the reliability and validity of survey results. This study focuses on how the background of translators affects key statistical properties, such as variance, skewness, and issues with reverse coding, and explores whether these effects can affect responses of survey items.
To investigate this potential effect, we will conduct a survey experiment where a standardized English questionnaire was translated into German. The translations were created by two groups of translators: 16 professional translators and 16 social scientists. By analysing the data collected from these 32 questionnaires, we will examine how differences in translator backgrounds affect the quality of item measurements, including variance, skewness, and errors in reverse-coded items.
The aim of the study is to highlight the significance of translation choices and translator backgrounds in shaping survey outcomes. Even small differences in wording or cultural framing can introduce bias, affecting item measurements and the underlying concept constructs. This study contributes to improving survey translation methodology and data comparability in cross-cultural research contexts.
Questionnaire Translation in the Fourth European Survey of Enterprises on New and Emerging Risks (ESENER 2024)
Mr Xabier Irastorza (European Agency for Safety and Health at Work (EU-OSHA)) - Presenting Author
The European Agency for Safety and Health at Work (EU-OSHA) completed its fourth wave of ESENER in 2024, interviewing over 41,000 establishments across all activity sectors in 30 countries.
This presentation focuses on the main steps of the translation of the ESENER 2024 questionnaire into 40 national versions:
a) A translatability assessment (TA) of the English master questionnaire.
Experienced translators from three different language families -Croatian, Finnish and Greek- reviewed the draft of the new and modified questions, identifying potential translation, adaptation or cultural issues and providing recommendations for alternative wording. The draft questionnaire reviewed in the TA was the result of cognitive testing interviews that were carried out in three languages different to those in the TA: German, Polish and Spanish. This aimed to ensure a broader language coverage in the pre-testing phase.
b) Two independent translators and one adjudicator per language version.
Following training -for all translators and adjudicators- two independent translations were provided to the adjudicator, who produced a reconciled version to be discussed at a team review meeting –one for each of the 40 national versions. A help desk service was offered throughout the entire translation and adjudication/adaptation process. There was a thorough documentation of each step of the translation using different worksheets of the same Excel file (one file per national versions), including hints, instructions and specific terminology.
c) National expert feedback
Representatives of the national bodies in charge of OSH in each of the 30 countries covered in ESENER 2024 reviewed the questionnaires. The aim was to ensure the appropriateness of the national OSH terminology rather than a linguistic check. The final versions of the questionnaires were sent to proofreading, to check for correctness of the target language.
The entire process lasted from November 2023 to February 2024
