Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Issues in Cross-national Data Colection and Analysis |
|
| Chair | Dr Catalina Lomos (CEPS/INSTEAD) |
Multicountry surveys demand a different approach in order to ensure quality. We distinguish quality management in the:
- set up: objective & document transmission, sampling, questionnaire design, translation, briefing, pretest, pilot, uniform software & lay-out (e.g. central scripting);
- field execution: fieldwork visits/listening in, fieldwork monitoring: progress, deviation sample/universe, adherence to instructions (time of visit/calls, frequency, respondent selection, ...), interview duration, intermediate data checks;
- (pre-) analysis: back checks, interview(er)variance, straight lining, consistency, similar interviews, non response bias & weighting factors, evolution of data, linking data to the context of a country and/or media events.
Thorough data quality management is essential to guarantee consistency and comparability in the data and sets standards for repeated waves. However not all measures reflect an identical cost impact as some will come at a high (intensive translation process) and some at a low cost (check of interviewer variance), but one must keep in mind that the total quality management cost is still relative to the data collection cost as a whole.
The paper will highlight the above measurements in the context of the third European Quality of Life Survey (EQLS3): measures taken, recommendations and cost implications. The role of the requesting department (Eurofound), the coordinator (GfK EU3C), national fieldwork agencies and the external audit process will be referred to.
Additional focus will be paid to the difference between multicountry research involving a single coordinator (e.g. EQLS3) and crossnational surveys (e.g. ESS), to define measures applicable to both and bring up differences.
Teacher collaboration and efficient work within professional communities proved to be an important predictor of student achievement in secondary schools, especially in disadvantaged or underperforming schools. In this context, the questions that appear are how are professional communities manifested in schools around the world and what is their impact on student achievement?
In order to answer such research questions, a cross-country research approach is necessary.
The concept of professional community is a latent construct, represented by five specific factors and their related items. In the new TIMSS (Trends in International Mathematics and Science Study) study 2011, with the data available in January 2013, the latent concept of professional community has been measured using 1 or 2 items for each of the five factors of the latent construct. The data has been collected in secondary schools in more than 50 countries, by asking teachers within schools their perception on many school-related processes, including their professional community practice.
In conclusion, the present study will aim at testing the measurement equivalence of the latent concept of professional community in more than 50 countries, using Confirmatory Factor Analysis and multi-group CFA. The applicability of this study is that, if equivalent, this instrument could be used as a measurement of the professional community concept in many educational contexts. In addition, the relationship of the concept with student achievement or with many other school and classroom characteristics could be investigated, considering the complexity of the TIMSS data, focused on students, teachers and schools.
Transparency International's Global Corruption Barometer (GCB) has been gathering data on people's perceptions and experience's of corruption in more than 100 countries for a decade.
But to what extent do the differences in country contexts limit the extent to which we can use this data to make meaningful comparisons of the results across country and over time? Particularly results on such an issue - corruption - that is not only sensitive, but that also manifests itself so differently in different places and over time.
This paper will first examine and identify variables that may affect the cross-country equivalence of our measures. Firstly looking at those variables related to the survey design, for example survey mode (the GCB is conducted by face to face, telephone and online depending on the country). Secondly the paper will explore other variables which may affect results at the country level in terms of how people respond to and interpret the questions on corruption. These will include measures of freedom of expression, trust , income and education levels.
Using the most recent 2012-2013 survey data of more than 100,000 respondents world wide, this paper explore the validity and reliability of the data for cross-national analysis. The findings can be used to inform both the way we interpret the results and also the design of future international surveys on corruption and related topics.