Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Use of Paradata for Production Survey Management |
|
| Chair | Mr Kyle Fennell (NORC) |
While developing CAWI instruments, German official statistics strives to meet high demands in comprehensibility, functionality and usability in order to increase survey data quality. In addition to qualitative pretesting under laboratory conditions, server- or client-side paradata might prove beneficial for gaining deeper insights into respondents' completion behavior in the field and identifying critical aspects of the online questionnaire in need for fundamental revision. However, Destatis has not used this method yet.
Going through the scenario of a possible implementation, methodological, legal and technical considerations are combined to an integrative assessment of cost, benefit and general feasibility. Of major concern are practical challenges like how to reconcile the desired informative value of paradata and the best possible online tool performance with strict data protection rules. Finally, some gaps in the existing knowledge about web survey paradata are indicated that provide useful points of departure for future research.
In telephone surveys, designers make efforts to ensure that the households/individuals who are the hardest to interview are included in the sample by increasing the number of call attempts before abandoning a number and calling back refusals/abandonments to persuade them to participate. These efforts rely on the hypothesis that there is a continuum starting from the respondents who respond most readily, followed by respondents who require further efforts, and finally by non-respondents--in other words, that individuals who are hard to interview are similar to non-respondents.
Calibration processes can hold the unit non response. But generally, the easiness to respond is not taken into account in the process: basically all respondents as a whole are classically considered substitutable to the non respondents.
We used the data from a two-stage random probability sampling using random digital dialling (n= 8645, 15-49 years old) collected in 2010 in France on sexual and reproductive health (SRH) to test two calibration process. The first takes advantage of the paradata (number of call attempts, number of refusals) that allowed grouping the respondents in three groups (easy, middle and difficult to interview, the latter considered as a subsample of the non respondents that could have been interviewed during a specific survey) with a method inspired by Hansen-Hurwitz. The second one is the classical one.
Comparisons of the three groups of interviews were based on sociodemographic bias, differences in SRH behaviours, multivariate logistic modelling of SRH behaviours. The calibrations processes were
Survey research is relevant and appropriate for problem identification and providing objective explanations to hidden phenomena in Nigeria. Despite the preference for survey research by scholars in the fields of social sciences, management and educational management, it is a method that is hindered by low literacy level of respondents, multiplicity of ethnic groups/languages, inability of respondents to answer questions appropriately and incidences misleading responses. The study examines these environmental factors using a quantitative method. The techniques used for questionnaire administration are face-to-face approach and posting via internet. The data collected were analyzed using descriptive and inferential statistics on the basis of which informed conclusions were drawn. The paper concludes that should the observed factors be substantially redressed, survey research in Nigeria would be enriched and academic research would be better fortified.
NORC uses paradata to monitor and manage field studies. Contact attempts, item non-response, interview length, audio recordings, and other paradata are used to monitor field and enable projects to take actions to make adjustments to plans during the course of data collection efforts. The diversity of NORC studies is matched by a diversity of ways in which paradata are used to manage data collection efforts.
This paper presents the results of an effort to create a single harmonized data set from these various studies so that this dataset could be used to support our goal of continuously improving interviewer quality. We begin by reviewing the paradata related to interviewer quality NORC collects and highlighting some of the ways these data are used to assess interviewer quality. We then discuss the challenge of creating a harmonized dataset to track interviewer quality across projects and over time. Next, we outline the theoretical foundations we drew upon when building a process for using the harmonized dataset and discuss the key features of the process itself. Finally, we present the result of the initial efforts to implement the process during the first half of 2013.