Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Multi-Level, Multi-Source Survey Designs |
|
| Convenor | Dr Tom Smith (NORC) |
To more fully understand human society, surveys need to collect and analyze multi-level and multi-source data (ML-MS data). Methodologically, the use of ML-MS data in general and the augmenting of respondent-supplied information with auxiliary data (AD) in particular can notably help to both measure and reduce total survey error. In particular, AD from sample frames, databases, paradata, and other sources can be used to improve data quality and reduce total survey error. For example, it can be employed to detect and reduce nonresponse bias, to verify interviews, to validate information supplied by respondents, and in other ways. Substantively, ML-MS data can greatly expand theory-driven research such as by allowing multi-level, contextual analysis of neighborhood, community, and other aggregate-level effects and by adding in case-level data that either cannot be supplied by respondents or is not as accurate and reliable as information from AD (e.g. health information from medical records vs. recalled reports of medical care).
The ML-MS approach first collects as much information as practical about the target sample at both the case-level and at various aggregate levels starting during the initial sampling stage. The second step is to augment the sampling frame by linking all cases in the sample to other databases. As Groves (2005) has noted, "Collecting auxiliary variables on respondents and nonrespondents to guide attempts to balance response rates across key subgroups is wise." The third step in ML-MS is to take information gained from the initial case-level linkages to secure additional information. The final step is to record, process, clean, and maintain a large amount of paradata for each case.
Background and aims: a pilot study on the Italian context has verified the existence of a significant relationship between the presence of a familial situation of ADL disability and increased risk of poverty (G. Casanova, R. Lillini Disability, not self-sufficiency and socio-economic inequalities and structural treatment strategies and the correlation with the incidence of poverty. a pilot study. in: Politiche Sanitarie, 2010 Vol 11 (4).
The aim of this study is to provide a methodology able to identify specific indicators constructed and measured on individual data available to the local public services of the social and health sector, in order to make a substantial contribution to the planning of local services and to better delineate the framework of the phenomenon in whole.
Materials and methods: the variables contained in the model proposed by the pilot study, used on secondary data in aggregate form, will be revised on the basis of individual data in possession of the information systems of the participating local actors ( e.g.municipality). After an assessment of completeness, quality, update level and standardization, these variables will be used for the construction of evaluation models based on the characteristics of the individuals, with the same techniques used in the pilot study.The municipality of Genoa, will be the case study.
Expected results: the expected result is to find a stable and repeatable method of establishing indicators specific (local) to describe the relationship between the presence of severe disability and long-term care and socioeconomic deprivation of
Auxiliary data can greatly help to understand nonresponse. In a cross-national survey auxiliary data can be available on three different levels. Information on a country level (e.g. nonresponse on major social surveys, GDP, survey scarcity, social trust) can give a general background. It should be mentioned that country level information can be confounded with survey agency information, e.g. if in some countries data are collected by statistical agencies, in others by universities and in still others by market research organisations. Regions within countries are the second level. These can be large regions, such as the American states, or small regions, such as neighbourhoods or local communities. Information from regions is available from contextual databases, such as the ESS Multilevel Data, statistical and commercial databases, and, at the lowest level, interviewer observations. At the third level information on individual sample units is available, from the sample frame, from registers linked to the sample frame at a case level, from interviewer observations and from process or paradata.
One problem in a cross-national survey such as the European Social Survey (ESS) is the lack of comparable auxiliary data across countries. Even paradata may not be totally comparable. A second problem is how to link auxiliary data to respondents and non-respondents within diverging national confidentiality regimes.
The presentation will show which auxiliary data are used in the ESS and which could be used.
Surveys typically collect data about how often individuals engage in behaviors. However, when policymakers and researchers design programs or policies, they often want to understand why individuals act in particular ways. Collecting data about respondent motivations for behavior is difficult, particularly in developing countries, and there is virtually no methodological research on this topic.
We conducted a face-to-face survey of 608 Ethiopian entrepreneurs that asked if respondents engaged in three business practices (advertising, sharing storage, and switching suppliers). If respondents did not engage in each practice, interviewers asked questions why they did not. For those questions, respondents were randomly assigned to one of three conditions: (1) close-ended questions, where interviewers read pre-specified reasons and asked for a yes/no response to each, (2) open-ended questions with interviewer probing, and (3) open-ended questions without interviewer probing. We compared the total number of reasons provided, as well as the number of reasons considered socially undesirable, across each group. We assumed that more reasons (particularly socially undesirable reasons) indicate a preferable method.
Analysis shows that respondents in the close-ended question group endorsed the most reasons. In this group, 34% provided more than one response, compared to 18% and 23% for the open-ended groups with and without probing, respectively. They were also the most likely to provide socially undesirable responses. In the open-ended groups, probing had no effect on the number or types of answers provided.