Poster session |
|
| Chair | Professor Annelies Blom (University of Mannheim ) |
This papers links theories from sociological stress research to studies on social inequalities in exposure to environmental noise. We investigate how exposure to road traffic noise is related to individual level physical as well as mental health outcomes and show how these effects are moderated by two largely recognized domains of social support: family and religious involvement. Noise exposure is generally considered to be harmful for health because it constitutes an threatening environmental condition and therefore a stressor. Enduring exposure to this stressor has deleterious effects on health as numerous studies yielded evidence for. For instance, road traffic noise was identified as a significant risk factor for developing cardiovascular diseases (Babisch 2014) or even diabetes (Sørensen et al 2013). Sociological stress theory suggests that several social mechanisms increase individuals’ ability to cope with stressors (Thoits 2011). For example, the negative consequences of personal crises such as unemployment on subjective well-being and health can be buffered by social ties that provide coping assistance or emotional sustenance through, among others, social support or giving behavioral guidance. Ultimately, these social buffers foster coping either by hurdling the stressor or by managing its consequences. A large set of studies show that these mechanisms can particularly be found in family ties (Carr & Springer 2010) as well as in religious organizations (Ruiters & DeGraaf 2006). The theory emphasizes that family strengthens individuals in coping with stressors because there is stronger emotional closeness, e.g., facilitating communication about subjective suffering. Furthermore, family provides structures of belonging, companionship or simply mattering that promote self-esteem and mastery. Similary, regular religious practice is a source of social support (due to the integration into the religious communities) and advances sense of meaning and belonging. Therefore, in some circumstances, family and religion have equivalent meaning as coping mechanisms for individuals. For this reason, we investigate how characteristics of the family and the religious practice of individuals moderate the negative effect of environmental noise exposure on mental and physical health. Existing studies either use large samples with subjective measures of noise exposure or limited samples (most convenience samples) with objective measures of environmental noise exposure. The data we used for our analysis stem from the 2014 German General Social Survey (GGSS). Thus, we use data from a representative face-to-face population survey based on a probability sample. The data was geocoded at the respondents’ address level and linked to small scale administrative data on traffic noise measured according to the Environmental Noise Directive of the European Union. This linked dataset contains subjective measures of physical and mental health, objective measures on road traffic noise, subjective measures of noise annoyance and extensive information on family demographics and religious identity and practice. The results of the empirical analysis indicate that individual religiosity does not moderate the effect of road traffic noise on health outcomes. The marital status, however, significantly reduces the negative effect of road traffic noise on mental health. There is no evidence for a moderation on physical health outcomes.
Building on the leverage-saliency theory of survey cooperation, this paper examines the effect of negative factors affecting the decision to participate, focusing on the burden they place upon the respondent. Deviating from the original theory this paper considers the decision to participate not as something that is fixed in time prior to participation, but rather as an ongoing dynamic decision making process throughout the course of the survey task,
The data used comes from a research project situated within the context of the Innovation Panel portion of the UK Household Longitudinal Study: Understanding Society, which tasked respondents with scanning receipts to record their expenditures for a month long period. Alongside this respondents are asked a series of questions about their experiences being a part of the study on a weekly basis. These two data sources are combined to produce both subjective measures of respondent burden, such as, self-reported difficulties participating; and objective measures, drawn from paradata from the receipt scanning application.
Due to the situation of the research project within the main Innovation Panel it is also possible to draw on the main annual survey as a data source, which enables analysis both of past survey engagement, and some relevant behavioral and attitudinal questions asked in the ninth wave of the survey about technology use.
Three main relationships are explored: the relationship between the subjective self-reported burden measures and the objective measures captured in the data; the relationship between previous wave response behaviors and responding in this additional data collection task; and the relationship between respondents use of and attitudes towards technology and their participation in the study.
Overall one of the potential benefits of utilising alternative methods of data collection such as this is that they offer alternatives that are less burdensome than traditional methods of data collection. Whilst not making a direct comparison, this research explores the notion of burden within this different context, and hopefully extracts some insight into successfully implementing a method such as this.
Abstract
Postpartum depression (PPD) is considered a major public health problem. It is estimated that 8-15% of mothers suffer from PPD in the UK. Empirical evidence on the effects of PPD on maternal employment are not explored. Using the Millennium Cohort Study (MCS) and a wide time span (3 to 11 years after the birth), the present study explored the effects of PPD on maternal employment in the UK, expecting to gain an insight into the possible pathways through which PPD can impact maternal employment outcomes, and thus fill the gap by obtaining a more comprehensive picture of the implications of mental disturbances on female wellbeing and employment outcomes. The direct and indirect effects (through marital status, future maternal mental and physical health and children’s cognitive outcomes) of PPD on maternal employment were explored. The direct and indirect effects of PPD on maternal employment were decomposed using the Karlson, Holm, and Breen (KHB) decomposition method. This method disentangles the total effect of a variable on the outcome, in this case PPD on maternal employment, into a direct effect (the effect of PPD on maternal employment adjusting for a mediating variable) and an indirect effect (the difference of the total and direct effect) for nested non-linear models. The two measures of maternal mental health in MCS, are the Malaise Inventory and the Kessler K6 scale. The first measure was used as an indicator of PPD since it was only asked at 9 months. The second measure is designed to be sensitive around the threshold for the clinically significant range of the distribution of nonspecific distress, as to maximize the ability to discriminate between cases of serious mental illness from non-cases, and was used as an indicator of maternal mental health problems in later years (3 to 7 years after the birth). This study finds a PPD effect on maternal employment at ages 5, 7 and 11 but finds no indication of an effect at age 3. The study also finds that the effect of PPD on maternal employment at ages 5, 7, 11 is partially mediated through subsequent maternal mental health problems.
In higher education and science studies, we typically interview young, educated individuals, e.g. students, graduates and researchers. Most of them are intensive computer and Internet user and thus well reachable by an online administered survey. Therefore, this method of data collection is gaining importance at the German Centre for Higher Education Research and Science Studies (DZHW). The DZHW’s long-term survey series are successively being converted to this mode of data collection. In our view, this leads to a number of methodological questions regarding mode effects.
(1) To date, little is known about where within the invitation letter the survey link should be placed, in order to minimize nonresponse. (2) It is still not clear to what extent the mention of paradata within the informed consent form (IC) influences participants’ decision to take part in the survey. (3) If the acceptance for such an extended IC is lacking, it would be possible to ask participants to directly deliver data that is otherwise collected automatically (e.g. about the device used & its configuration, screen orientation & resolution, question time & duration). But it is unclear if respondents are willing and able to provide valid information on this. (4) With increasing mobile Internet usage, a growing proportion of mobile respondents are taking part in online surveys, which have been designed to be answered using desktop computers. Responsive web design is used to optimize the survey experience for both user groups. It is currently uncertain to which extent this varies with data quality. (5) Gamification (i.e. integrating playful elements) could be used to encourage more participants to complete the survey. However, hardly any experience is available for this innovative approach. (6) A further question posed is the form of administration of complex question types. Extensive item batteries could be divided in a variety of ways and thus may provide better data. (7) The survey setting also plays an important role in online surveys. The social situation of mobile and non-mobile responders is very different. It can be assumed that this is reflected in the response behavior and should be taken into account in data analysis. (8) Online surveys are often associated with low participation. Thus, the acceptance of a short survey for non-responders should be determined in order to at least obtain information on a pre-defined set of core variables.
In 2017, an Experimental Study on the Methodology of Online Surveys (ESMO) will be carried out. It will investigate the eight questions outlined above regarding participation and data quality which have not been sufficiently been taken into consideration in existing method research. About 25,000 participants of an online access panel, consisting of randomly recruited students at German higher education institutions, are invited. They are randomized into various conditions of the first seven scenarios. Non-participants receive the invitation to a short survey instead of a last reminder.
At the conference we will present the complete randomized controlled trial design and initial results, including recruitment figures and power analyses.
A common objective of epidemiological surveys is to provide population-level estimates of health indicators. Survey results tend to be biased under selective non-participation. One approach to bias reduction is to collect information about the health of non-participants by contacting them again and asking them to fill in a questionnaire. This information is called re-contact data, and it may be utilized to adjust the estimates for non-participation.
We analyzed data from the FINRISK 2007 and 2012 health examination surveys (participation rates 62.6% and 58.9%), where re-contact data were collected. We assumed that the respondents of the re-contact survey were similar to the remaining non-participants with respect to their health given the available background information. The validity of this assumption was evaluated using data on hospitalizations before and after the survey obtained through record linkage of survey data to the administrative registers.
We applied multiple imputations tailored for the assumption to deal with non-participation in the estimation of the health indicator levels. The estimated smoking prevalences adjusted for non-participation were 27.1% in 2007 and 23.7% in 2012. The estimates based only on participants were 21.8% and 19.6% for 2007 and 2012, respectively. The estimated prevalence of heavy alcohol consumption was 6.8% in 2007 and 7.1% in 2012. The estimates based only on participants were 5.2% and 4.8% in 2007 and 2012 respectively. The estimates adjusted for selection bias due to non-participation revealed clear differences in comparison to participants' estimates.
We conclude that utilization of re-contact data is a useful method to adjust for non-participation bias on population estimates in epidemiological surveys.
References:
Karvanen J, Tolonen H, Härkänen T, Jousilahti P and Kuulasmaa K. Selection bias was reduced by re-contacting nonparticipants. Journal of Clinical Epidemiology, 2016;76(1):209–17, doi:10.1016/j.jclinepi.2016.02.026
The Programme for International Student Assessment (PISA) is a macro-survey project carried out by the Organisation for Economic Co-operation and Development (OECD) every three years since 2000. It aims to evaluate education systems worldwide by testing the competencies in mathematics, reading and science of a sample of 15-year-old students living in any member or associated country of the OECD. A cursory analysis of the PISA data allows for comparing the performance of students between countries. A more thoroughgoing study of the data may also be useful to identify the determining factors of the differences detected. According to a number of studies, the gender is one of the most influent aspects on the academic performance of the students.
On this basis, we performed an analysis of the PISA data by gender using multilevel regression models. We have considered two levels of variables: those related with the students and those others regarding the schools in which the students are enrolled. Our study sample is obtained from the 2012 edition of the study, which examined some 510,000 students drawn from a population of 28 million students in 65 countries. In Spain, over 25,000 students from 900 schools located in 14 regions were evaluated. Our analysis focuses on the evaluation of scientific competence according to the data collected for the students in the region of Andalusia. The objective of this analysis is to determine the variables that have a significant influence on the scientific performance of boys and girls.
Our results show that the factors influencing the scientific performance are different for boys and girls. For example, the immigrant status does not affect all students in the same way: the immigrant girls recorded a scientific performance that was, on average, 45.24 points lower than that of native-born girls. However, the immigrant status has not a significant effect on boys. Furthermore, the scientific performance of boys who live both with his father and his mother is, on average, 18.63 points higher compared to boys with any other familiar structure. However, the familiar structure does not influence the performance of girls. Both the immigrant status and the familiar structure are variables belonging to the student level. But there are some other variables of the school level affecting the scientific performance of boys and girls differently. As an example, girls attending private schools achieved, on average, 30.180 points more than girls attending public schools. Nevertheless, the type of centre does not affect the scientific performance of boys.
Because cross-national survey research generally analyzes large-scale, publicly available data sources, this field seems to be relatively well protected against the replication crisis that affects social-psychological experimental research (e.g. Open Science Collaboration, 2015). However, the idea of replicability rests on the assumption of transparency. Only when cross-national survey researchers communicate transparently about their empirical strategies and decisions, the scientific community can evaluate, build upon or refute research results. As such, transparency is one of the foundations of the scientific method - cfr. Merton’s (1973) scientific norm of communalism.
This study investigates the degree of transparency in the field of cross-national research. We consider a random sample of studies using publicly available cross-national datasets that were published in one of 30 sociological and political science journals. As a first step, we examine whether basic information regarding the empirical analysis (e.g. the research population, the selection procedure, coverage, response rates, survey mode and the specific question wordings used) is reported adequately. These findings will offer an overview of what information is most likely to be left out in empirical sections; whether researchers report correctly the information about the survey data used, or, most importantly, whether they provide sufficient details to replicate a study. As a second step, we examine how certain characteristics of articles (e.g. length), authors (e.g. academic rank) or journals (such as impact factor) are related to the level of transparency (i.e. amount and correctness of the information given) in the method section. The overall goal is to provide a set of guidelines that will facilitate efforts to replicate cross-national survey analyses.
Health examination surveys are conducted to obtain information about health and health determinants at the population level and their trends over time. Over the past decades, the participation rates of such surveys have decreased. This leads to declining representativeness if participants and non-participants differ from each other. Thus, the interest towards the reasons and consequences of non-participation has grown. Non-participation increases uncertainty of estimates and, more seriously, may introduce bias. For example, health behavior and health status have been seen to differ substantially between participants and non-participants.
Participation rates have varied between different population groups. Low occupational class and educational level have usually been associated with a low participation rate. Knowledge on the characteristics of non-participation can help us to develop more effective, subgroup-specific recruitment methods and statistical non-participation adjustment methods for the estimation of health indicators.
Planning the allocation of health care resources and evaluating health interventions or prevention programs require long-term health data. The decreasing participation rates may complicate the utilization of the collected data. If we do not know how well the population is represented by survey participants, we cannot know whether the observed changes in health indicators are due to real changes in the population or changes in the representativeness of the survey results.
We provide new information on long-term changes in participation rates and their associations with occupational class and educational level in Finland between 1987 and 2012. We will also explore the effects of sex and age and their interactions to deepen our understanding of non-participation in health examination surveys and its relationship with socio-economic position. Some predictions of participation probabilities until the year 2022 are presented. Data from six cross-sectional FINRISK surveys conducted between 1987 and 2012 in Finland are linked to national administrative registers which are available for both survey participants and non-participants.
Our results show that individuals with low occupational class or low level of education were less likely to participate than individuals with high occupational class or high level of education. Participation rates decreased in all subgroups of the population over 25 years but the decline was fastest among those with low level of education. This can partly be explained by the radically changed distribution of educational levels in Finland. Our future predictions indicate that participation rates will continue the decreasing and educational level differences may become even greater. The differences in participation rates must be taken into account to avoid biased estimates because socio-economic position has also been shown to be associated with health behavior. Particular attention should be paid to the recruitment of the less-educated population groups.
Although the study of political trust received considerable attention, research did not yet yield concluding results. Research still focuses on explaining the relationship between political trust and other variables that either affect or explain it cross-sectional or longitudinally in several parts of the world. Besides that each of these approaches faces particular challenges given the limitations of available data, most of the studies ignore measurement error. We therefore illustrate how correction for measurement error (CME) can be done when explaining trust in political institutions and how the substantive conclusions change after correction. We use data from the European Social Survey and measurement quality information from the Survey Quality Predictor (SQP 2.1.), the work of Alwin (2007), and calculated the measurement quality of the sum scores ourselves. In order to illustrate the change in substantive conclusions we run the same linear regression model twice, once before and after CME. We find that the results are very different: the effects of the predictors changed their significance, some increased, some decreased, and overall the model with CME explains much more of the variance of the dependent variable. This illustrates that measurement errors cannot be ignored.
Surveys rely on different types of scales to measure attitudes. Two commonly used types of scales are the Agreement/Disagreement scale and the Item Specific Scale. The study focuses on items that measure political efficacy in the pre-election study of American National Election Study 2012. The political efficacy scale has been applied with a randomisation on the scale direction, in two modes: web and face-to-face. I test whether the options that appear first tend to get selected more often. Moreover, I research the extent to which the mode of application (face-to-face versus web). The findings suggest that each scale performs differently on the web and on face-to-face depending on the scale direction. I will conclude with a discussion on the findings and their implications for survey design.
This presentation examines the relation between features of return questionnaire in mail survey and characteristics of respondents. We especially focus on two points. One is how the questionnaires were folded by respondents in order to enclose in an envelope. Because the return envelope which we enclosed with the questionnaire was about one third of the qustionnaire, the respondents had to fold the questionnaire in order to put it in the envelope. Most respondents folded the questionnaire twice horizontally, but some respondents folded the questionnaire into four or fold it vertically. We examined the characteristics of respondents who folded the questionnaire in such irregular ways, and found that more males fold the questionnaire vertically than females. Age or educational background was not related to the ways of folding.
Another point we focused on is whether the address printed on the return envelope was overwritten. A word "Iki", which means "To" in Japanese, was printed along with the return address. In Japanese traditional culture, it is regarded as a polite manner for respondents to overwrite it with a word "Onchu", which means "Dear" in Japanese. But some respondents did not overwrite the word "Iki". We examined who followed the traditional custom, and found that more aged respondents wrote over the word "Iki".
The recent challenges in polling the British Election of 2015, the Brexit vote in 2016 and the US Presidential Election have resulted in an increased focus on the content and methods of the polls that were conducted. A major review by the British Polling Council of the outcomes of the British election polls was released this year (http://eprints.ncrm.ac.uk/3789/1/Report_final_revised.pdf), and a review has been announced by AAPOR to examine the polling in the US election. A similar experience with state polling in the 2008 presidential election lead to the establishment of the AAPOR Transparency Initiative (TI), which was formally established in 2014 following several years of development, and now has 40 approved member organisations.
In the case of both the Transparency Initiative and the BPC review, there is a strong emphasis on the documentation of the methods and outcomes of the survey results, and the potential for the release of anonymised microdata from the polls themselves. This mirrors a growing trend in academic research (such as the Horizon 2020 Data Management Guidelines) and government data (such as the Australian data.gov.au public data statement) for the release of both data and documentation associated with the creation of survey and other data collection activities. Thus the challenge to improve transparency has been put to data production organisations across all sectors.
This challenge however creates new demands on data producers. The creation of relevant, manageable documentation and data puts an additional burden on the production process. Are producers therefore ready to meet this burden? This poster explores the potential for the use of metadata standards as a means for enabling this.
The poster will present an overview of four core metadata standards in this areas – the Data Documentation Initiative, the Transparency Initiative (AAPOR), Dublin Core (libraries and information standard) and DCAT (data.gov systems in Australia, UK and USA), that can be used to enable transparent reporting of research results. It will present an overview of the reporting requirements of the four standards, a comparison of their content, and the integration between the standards. Finally, the poster will provide an approach for implementation of standard survey reporting requirements, based on the DDI metadata framework, to support reporting against all four standards within a joint information systems framework.
Compared to the European average, the proportion of Germany’s landline households is still relatively high (86%). Nevertheless, relying solely on landline surveys may substantially bias the estimates due to undercoverage, since mobile-only-households differ systematically from those that can be sampled by the landline frame. Thus, in case of nationwide surveys, dual-frame approaches, containing landline and mobile samples, appear to be an appropriate solution to deal with these frame imperfections.
The benefits of such surveys in Germany have been extensively examined, e.g. by both CELLA studies (CELl-phone and LAndline) that GESIS, the Leibniz Institute for the Social Sciences, conducted in cooperation with the Dresden University of Technology in 2007/2008 and 2010/2011. Besides numerous insight regarding the population of only-households and the potentials of dual-frame surveys, a key-development has been the weighting approach that is nowadays commonly practised when weighting such surveys. The weighting approach adjusts for unequal inclusion probabilities by employing the Horvitz-Thompson-Estimator and for frame-independent nonresponse using the GREG-Estimator.
However, it does not consider nonresponse in dual-frame surveys that is frame-dependent. A further investigation of CELLA 2’s data suggests that there might be factors apart from socio-demographic characteristics that may affect individual response behaviour. Since CELLA 2’s call-outcomes of the landline and mobile sample differ considerably, another source of nonresponse might be determined by varying usage-patterns of specific means of communications within the overlap domain of both frames. Therefore, this paper will introduce a weighting approach that takes such frame-dependent, device-specific response behaviour into account by subdividing the overlap domain into several, disjoint domains, determined by usage-patterns, and employing a composite design weighting approach. To test this particular weighting approach, it will be applied to the data sets of the CELLA 2 study and the survey Influenza that has been conducted by the Robert Koch-Institute. Furthermore, the precision of such an estimate will be compared to various other approaches such as the Kalton-Anderson Approach, Composite Weighting and the Pseudo-Maximum Likelihood Approach.
Anthropometric measures, and parameters derived there from such as Body mass index (BMI), are widely used in epidemiological studies to evaluate the prevalence of obesity. However, measuring anthropometric parameters is a cost factor of importance in large studies, as measurement requires face-to-face contact, specifically trained personnel and time. Self-reported data on the other hand, can be obtained relatively easy and at low cost. In societies, where tallness and leanness are socially favored, self-reported body weight and height may though not always be accurate. BMI derived from self-reported weight and height accumulates the errors of both variables which is particularly important when subjects are classified into standard BMI categories. Hence, reporting errors in anthropometric measures may have serious consequences as they may lead to biased estimates of the relationship between body size, shape and health outcomes.
Considering the above and in view of the known cultural differences in the three main linguistic regions of Switzerland (German, D-CH; French, F-CH; Italian, I-CH), both self-reported and measured anthropometric data were collected in the first National Nutrition Survey menuCH, a cross-sectional population survey in 18 to 75 years old adults. Of the 2086 participants of menuCH 2071 reported their body weight and height prior to their face-to-face interview while 2051 allowed measuring it as part of their face-to-face interview. Pregnant or breastfeeding women and immobile people were exempt from measurement. For 935 men and 1103 women both self-reported and measured weight and height are available and were used to analyse the degree and direction of reporting error.
Participants generally self-reported their weight, height and BMI very accurate. Mean difference (SD) between measured and self-reported was 1.18 (1.96) kg for weight, -1.04 (1.71) cm for height and 0.39 (0.90) kg/m2 for BMI.
Accuracy for self-reported weight and height was equal for men and women while for BMI it was men > women. For the AGE GROUPS accuracy for self-reported weight, height and BMI was ‘18-29 years old’ > ‘30-44 years old’ > ‘45-59 years old’ > ‘60+ years old’.
In the LANGUAGE REGIONS accuracy for self-reported weight was F-CH > D-CH > I-CH while for height and BMI it was D-CH > F-CH > I-CH. Accuracy for self-reported weight, height and BMI was for WEIGHT SATISFACTION ‘very satisfied’ > ‘satisfied’ > ‘dissatisfied’ = ‘very dissatisfied’; for DIETING it was ‘not on diet’ > ‘on diet now or during the last 12 months’; and for BREAKFAST it was ‘breakfast-skippers’ > ‘non-skippers’;
EDUCATION: accuracy for self-reported weight was ‘primary’ > ‘secondary’ >‘tertiary’ while for height and BMI it was ‘tertiary’ > ‘secondary’ > ‘primary’.
Due to the misreporting 10.5% of the participants would have been classified in the wrong BMI class, namely 1.1% as underweight, 6.2% as normal weight, and 3.2% as overweight, instead of one BMI class higher.
Surveys often rely on subjective measures for various concepts such as wellbeing, health or happiness. For these measures it is important that different groups of the population understand the measured concepts and items in the same way. Otherwise conclusions drawn from analyses may be misleading.
The SF-12 is a widely used scale to measure health-related quality of life. It consists of 12 questions covering both mental and physical health and health restrictions. It has been widely used by former research to assess the effect of unemployment on individual health. However, there is some reason to question the validity of the SF-12 for the comparison of employed and unemployed respondents because of some critical questions which refer to work or other regular activities. Employed persons who feel restricted at work due to their health may rate their health lower than unemployed persons of the same health status.
While there is research on the intercultural equivalence of the SF-12, there exists no previous research, testing if the SF-12 is equivalent across groups with different employment status. I assess, whether the SF-12 is suited to analyse health effects of (un-)employment using the large German Panel Study Labour Markets and Social Security (PASS). PASS consists of a benefit recipient sample and a sample from the general population. The household panel has been conducted since 2006 and covers 12,436 respondents in 2012, the year I am using for my analyses.
The overrepresentation of benefit recipients who are most often unemployed, allows detailed comparisons between employed and unemployed persons. Assessing measurement invariance between employed and unemployed persons is of utmost importance because it is a prerequisite for drawing correct (causal) conclusions. I use the CFA-framework in order to tackle equivalence of health measures for employed and unemployed persons.
First results indicate problems in modelfit for survey population as a whole. However, measurement invariance can be achieved using a modified model including measurement factors accounting for different item scales.
In our advancing technological society, school-age children increasingly need access to computers and the Internet at home in order to progress through their education. In 2015, approximately 81 percent of children under age 18 in the United States lived in a house with Internet access (ACS 2015). However, prior research has shown that disparities in home Internet access exist by poverty, geographic region, race/ethnicity, and parent’s educational attainment. Therefore, it is important to be able to examine differences in Internet access for small subgroups of school-age children. Different data sources are available for measuring Internet access from home and each offers advantages and disadvantages. Since 1997, the U.S. Census Bureau has reported on Internet use from home using the Current Population Survey (CPS) supplements. This paper will study the recent period of 2010 to 2015 when questions on Internet access were more consistent. Based on the July and October CPS supplements, Internet access from home for school-age children fluctuated between 2010 and 2015 with no clear pattern of increase. This paper will examine these puzzling findings and investigate whether context effects may be a factor. The July CPS supplement included a detailed battery of questions on household and individual Internet use, while the October CPS supplement included a short subset of questions on these topics. In addition, the Census Bureau has reported on Internet access from home starting in 2013 using the American Community Survey (ACS). The CPS and ACS differ in several aspects, including sample size, mode of administration, questionnaire design, and reference period. This paper will investigate fluctuations in CPS estimates of Internet access for school-age children between 2010 and 2015 and also present recent ACS estimates on this topic by different demographic characteristics. The analyses will help researchers determine which data source best fits their needs.
Web surveys are increasingly used for data collection in social science research since they offer several substantial benefits: cost-effectiveness, saving time, and most importantly, they enable researchers to capture a variety of paradata (e.g., response times). Web mode, however, might also support respondents’ distraction during survey completion due to “multi-tasking” (e.g., checking incoming emails, changing to other websites, or staring software). Until now, it lacks empirical evidence in which specific way distraction during survey participation affects response behavior. In this study, we therefore investigate whether there are systematic differences between respondents who process the survey steadily and those do not. For this purpose, we use a new paradata tool called “SurveyFocus (SF)” – enabling survey researchers to gather the activity of the web-survey page. This cross-sectional study (n = 1,751) is based on an onomastic sampling approach and contained single as well as grid questions. The statistical analyses reveal substantial differences between continuously and discontinuously processing respondents. This implies that respondents who leave the web survey for a certain time period produce significantly longer processing times (after correcting for the “time-out”). These respondents, additionally, produce lower response quality in terms of item non-response and error of central tendency. Furthermore, there are considerable differences between single and grid questions. All in all, our empirical findings suggest that the continuity of web-survey processing matters. For this reason, survey researchers and practitioners should take this circumstance into consideration when analyzing and interpreting web-survey data.
Traditionally, the European Company Survey (ECS) has been a telephone survey of establishments in Europe, in which interviews are carried out with a management representative (the most senior person in charge of personnel) and – where available – an employee representative responsible for the establishment. The unit of enquiry for the survey was the establishment. The target population was all establishments with 10 or more employees in all economic sectors except those in the NACE Rev. 2 categories A, T and U. The countries covered were all 28 EU Member States, as well as 5 candidate countries.
With survey response rates dropping throughout Europe, there is a challenge to both find ways to involve selected respondents and to learn more about the non-respondents (to better control for non-response error). Because of the special circumstances and characteristics of business surveys, response rates are relatively low due to the difficulties involved in reaching a respondent (gate-keeper). The overall response rate of the third ECS 2013 for the management interviews was 38%, ranging between 18% in Austria and 62% in Slovenia. It is assumed that the very low response rates in some countries are in part due to the survey mode. Response rates were higher for the employee representative interviews with an average of 60% ranging from 39% in Ireland to 83% in Croatia.
Presenting multiple survey modes in sequence has been demonstrated as a successful way to improve response rates. A major advantage offered by mixed-mode survey designs is an increase in the coverage of the population of interest. Mixed-mode designs may offer a more inclusive frame with respect to the target population. Other advantages include a reduction of cost/increased efficiency, the establishment of credibility and trust with the respondent. Problematically, however, since each mode of data collection is unique in terms of the transmission of information and the environment of the interview, survey questions are received and processed by respondents in different ways. Mode effects are an important downside to the advantage(s) offered by a mixed-mode design.
In this paper, we will therefore present several scenarios for potential mixed-mode data collection for the next ECS which is believed to improve data collection and overall response rates. Question discussed are : Is an option to respond on-line a viable way to collect some characteristics from at least a part of non-respondents and to design better survey modes to have them take part in the future?
In this task, the contractor will be also requested to provide cost evaluation associated with introduction of new survey modes.
The overall objective is to scrutinise and compare several survey mode scenarios as well as mode effects and to develop an optimal scenario for the next wave to be carried out in 2018.
In the last years Log-linear analysis (LLA) of contingency tables became a widespread method of data analysis in several fields, including social science. This method allows to analyze dependencies between two and more nominal variables and their interaction effects. Log-linear analysis is especially fruitful in the field of social sciences because it is applicable for most variable types and can detect non-linear relationships.
Today, most software implementations of LLA are based on iterative maximum likelihood estimation (MLE). This, however, leads to the main limitation of the method: its inability to work with contingency tables with one or more zero counts. Zero counts, however, are very common in the poll and other social science data, especially when working with large sparse multidimensional contingency tables.
Statistics and statistical software developers propose several techniques to tackle this limitation, but their use may lead to biased coefficients estimation and probability of non-existence of MLE. The most common techniques include adding small constant to some or all cells in contingency tables and changing the order of categories in the variables. Alternatively, it is possible to fit the LLA model using iterative Pearson Chi-squared minimization, but this strategy is not yet adequately described and analyzed in the literature.
I use Monte Carlo experiment to estimate the stability and bias of different techniques for working with zeros in contingency tables in Log-linear analysis. The key question of the paper is if all techniques are equivalent and lead to same conclusions, or some of them may lead to invalid interpretations.
Background: During adolescence, assessing body composition and health in cohort studies and surveys is a challenge as puberty is a dynamic period of development marked by rapid changes in body size, shape, and composition, all of which are sexually dimorphic. At any given age in adolescence, children will be at varying stages of puberty. Consequently, analyses of adolescent body composition and other health outcomes that vary with pubertal stage need to take account of this. We conducted a review of evidence to examine potential methods for assessing pubertal stage within cohort studies and surveys.
Methods: Our aims were to: identify methods used to assess pubertal stage; look for evidence of their validity; identify similarities that could contribute to harmonisation of datasets; and seek barriers to pubertal assessment and assess acceptability of the various approaches. We conducted a review of published literature and gathered expert opinion. Searches of bibliographic databases (Medline, Psycinfo, Scopus, Sociological abstracts, CINAHL, ERIC) were carried out by an information specialist. A group of 26 experts was invited to a workshop at which consensus approaches were used to shape recommendations for pubertal assessment within cohorts and surveys and for future research. Barriers to pubertal assessment, and acceptability of different approaches were explored through consultation with a children’s Patient and Public Involvement panel.
Results: Searches of bibliographic databases led to the identification of 11,935 abstracts. Detailed assessment of these led to identification of 50 studies describing approaches to pubertal assessment in the three broad areas of interest: self-assessment (20 studies), growth (15) and biological sampling to assess hormone levels (15). Self-assessment approaches agreed with the gold standard of clinical assessment of Tanner stage to within one pubertal stage. Accuracy increased if assessments are done by parents as well as the adolescent themselves. Peak height velocity is a retrospective marker of pubertal stage but serial height measurements are required to derive it. Growth markers at earlier stages of puberty include measurement of hand and foot size. Regular measurement of foot size as part of a cohort study could be used to indicate progression through puberty. None of the assessments of hormonal change currently offer a practical means of assessing pubertal stage within research studies. Early morning urine or saliva samples have potential in the assessment of LH, inhibin and testosterone, although serial measures may be needed to assess pubertal stage. The children we consulted thought self-assessment was the most acceptable approach, and preferred completing questionnaires by hand to digital approaches such as apps. They stated they were more likely to agree to clinical assessment if it were conducted by a doctor/nurse of the same sex.
Conclusions: Overall, the evidence on assessment of pubertal stage was patchy. We identified few validation studies. Most studies included small numbers of participants. Many of the approaches that have the potential to identify pubertal stage require frequent anthropometric measurements or biological sampling and so have resource implications for cohort studies and could not be used for cross-sectional surveys.
Due to its complexity and abstractness, the social space paradigm is difficult to transfer not only to practical social work but also to the scientific analysis of practical social work and education. To analyse how problems related to social space are perceived in social work and educational practice it is therefore crucial to make these concepts more accessible.
My approach suggests the qualitative digital pin-method as an appropriate example of such accessibility. This method originates in social work. Children and adolescents are asked to pinpoint places where they usually visit, meet with friends or avoid entirely on an analogue map. Thus, social workers become acquainted with movement patterns and uses of public space. The adaptation of this method to a digital format and to qualitative social research is innovative. Combined with a set of qualitative questions, it can be used for visualising social space related problems like expulsion, functionalisation and conflicting or overlapping uses of public space on a low-threshold level. It can also be used to analyse the participants’ fundamental understandings of the constitution of social space.
The qualitative digital pin-method will be demonstrated using data acquired during my dissertation. The dissertation focusses on the German discussion on educational landscapes and the enhancement of equal educational opportunities regardless of social background. The latter shall be achieved via the collaboration of multifaceted educational actors based on a common social space. Specific comprehensions of Bildung, education and social space are implied as the base of this collaboration. This includes the consideration of the shared importance of formal and non-formal educational settings as well as their interdependencies. Furthermore, this includes the consideration of non-institutionalised learning of children and adolescents tied to their individual social spaces.
In practice, educational landscapes are usually centred around schools. Non-formal learning opportunities are merely added as a bonus. Schools rarely reach out to their surrounding social spaces. Non-institutionalised informal learning and learning experiences based on social space are seldom considered. These empirical results suggest a breach between the theoretical and conceptual principles of educational landscapes and the actor’s comprehension of Bildung, education and social space as well as their realisation. So far, no studies explicitly explored the comprehension of Bildung, education and social space by actors in educational landscapes. This analysis is therefore part of my dissertation. Additionally, I study which roles these actors ascribe themselves and how these roles and comprehensions match in different fields of Bildung and education. By including the concept of social space as an important opportunity for Bildung and education my dissertation exceeds the common reception of social space as a merely administrative condition in regulating educational systems.
My survey was conducted in four educational networks in Germany. The respective actors were questioned in qualitative semi-structured interviews. In addition, qualitative vignettes and the qualitative digital pin-method were used to enable access to complex concepts like Bildung, education and social space. Qualitative content analysis has been used to analyse the obtained data.
Survey data can be usable only if accompanied by comprehensive metadata. These are ‘data about the data' necessary to transform the numbers into meaningful knowledge. In international survey programs, metadata are paramount to document data quality across countries and evaluate issues of cross-national equivalence between country methodologies and datasets. This is especially crucial for survey programs based on a decentralized management model that is relatively dependent upon post hoc harmonization of data. Such surveys require full documentation of country specificities in fieldwork methodologies, as well as of data harmonization procedures.
This paper presents the ways in which metadata are provided in the framework of the Generations and Gender Programme (GGP). This is a longitudinal survey of 18-79 year olds in 19 countries in Europe and beyond, aimed at studying socio-demographic and economic challenges such as low fertility, changes in family structures, and population ageing. It is run by a consortium of research institutions following a relatively decentralized management model. The national teams may either adapt the survey instruments and the fieldwork guidelines to the different national contexts or incorporate them into existing surveys (such as in the case of Australia, Hungary, and Italy).
The challenge of documenting GGP data is the need to combine country specificities in fieldwork methodologies and country deviations from the standard questionnaire, with information on the harmonization process. Metadata are provided in compliance with the Data Documentation Initiative (DDI), the international standard recommended by the Consortium of European Social Science Data Archives (CESSDA) for documenting social science data. DDI provides a framework and a format that allow for data and metadata online publication. To publish data and metadata online, the GGP implements the software package Nesstar.
The GGP national teams are to provide country-specific documentation on national deviations in the survey methodology and in the questionnaire implementation. They fill in a template of DDI items chosen by the GGP central coordination team. This information can be browsed in the GGP Online Codebook and Analysis tool, through the software Nesstar. Here surveys are presented in three ways. First, to provide users with cross-national overviews of data, consolidated data files are published for each wave. In these files, variable metadata account for country specificities, whereas general survey metadata only regard the international guidelines. Second, country-specific data files are also published and fully account for cross-country deviations from the international guidelines. A third type of data file documents availability of variables across countries and waves thus providing insights into country compliance to the standard questionnaire.
To properly document GGP specific methodology, a great amount of metadata is required. The national teams and the central coordination put a lot of resources on metadata preparation. The implementation of DDI right from the beginning of the fieldwork could optimize the management of the entire documentation.
The poster introduces a process to integrate two formerly distinct panel studies. The Student Life Cycle of the German Center for Higher Education Research and Science Studies (DZHW) is aiming to provide individual data about the transition to higher education, educational careers, the transition to and establishment of graduate employment as well as alternative work modes (such as family work) in the approx. 9 to 10 years after graduation. The project has two main objectives:
- The linkage and coordination of the large quantitative Panel Studies “DZHW Panel Study of School Leavers with a Higher Education Entrance Qualification” and the “DZHW Graduate Panel”
- The generation of panel data over longer observation periods.
The integration of the DZHW panel studies and the long-term observation of individuals provide several improvements for the analysis of the student life cycle. Educational careers can be examined more extensively in their chronological and sequential structure as well as in their embeddedness in the life course. The interactions of educational careers with other living areas, also taking into account socio-demographic characteristics, offer analyses that go far beyond the possibilities of official statistics. In addition, determinants of switching habits between tertiary education and alternative activities as well as their effects on employment opportunities and non-monetary returns on education can be taken into account.
Against this background, the SLC project has two main goals: on the one hand, the long-standing and proven surveys are to be continued in order to gather data for education monitoring and not to lose valuable time series. On the other hand, by harmonizing and integrating the panel studies, new analytical potential is to be identified by observing long time periods with a uniform modularized panel survey. By that, new comparative groups are implicitly made available (e.g. for a comparison of graduates with persons without a university degree).
The poster gives insights in the challenges which occur when integrating existing panel surveys, making use of a design with multiple cohorts and sequences. It addresses problems of sampling procedures as well as the harmonization of survey contents given the need of preserving long-lasting time series demanded by education policy. In addition it introduces a new source of data for research in higher education.
The increasing number of refugees in Germany since 2015 poses substantial socio-political challenges, especially for the social security system. In 2016 more than 400.000 recognized refugees acquired entitlement to welfare benefits which leads to a strong structural change. One of the central dataset in the field of labor market, welfare state and poverty research in Germany is the Panel Study "Labor Market and Social Security" (PASS), which is established by the Institute for Employment Research. Since 2006, approximately 12.000 persons in more than 8.000 households were interviewed every year. In order to continue ensuring representative data and to provide reliable information on refugee’s labor market and social integration, network composition and a variety of socio-demographic characteristics and subjective indicators such as contentment, fears and problems, it is necessary to interview this special migration population. With an oversampling of Syrian and Iraqi refugees almost 500 households with over 800 personal interviews could be conducted in the current wave (wave 10).
There is a growing body of literature which shows lower participation rates for migrants and deals with specific problems and obstacles that arise when integrating these populations into survey designs. In PASS different strategies are implemented in order to ensure a high response rate and high quality of interviews. For example, in the CAPI-mode refugees could choose to either take the chance to consult a third person for interpreting the interviewer or to read the questions by themselves in an Arabic version of the questionnaire. In the CATI survey the interviews were conducted by Arabic-speaking interviewers.
In the poster presentation we would like to give an initial general overview of the refugees in PASS and their availability, accessibility and participation by using additional information of follow-up interviews. More precisely, each phase of the survey (e.g. contact, cooperation) is analyzed separately while several aspects of the fieldwork, like the use of bilingual interviewers and questionnaires, were taken into account. Moreover, we investigate how interviewer characteristics and the different mode strategies affect sample selectivity and response behavior.
In large scale face-to-face surveys such as the European Social Survey (ESS) interviewers are usually assigned to sample cases that are close to their area of residence. Unfortunately, this lack of interpenetration does not allow to properly attribute which part of the observed effects (e.g. response behaviour or the variability of target variables) are caused by interviewers and which part is caused by areas. As inevitably effects of interviewers, as well as area differences may occur, it is important to quantify them in order to plan features of the survey such as the sampling design, interviewer training and interviewer assignment. Ultimately it can become of great importance for conducting statistical inference to understand the magnitude of such effects, as they can be used in the modelling of the variance of estimators. Furthermore, this can also help to optimize the allocation of available funds to help minimize the observed effects more effectively. In this paper, we will present our current knowledge on interviewer and area effects in the ESS, the unavoidable problems their entanglement raises and discuss possible solutions to improve the quality of the ESS based estimation.
Due to recent technological change and input from the task-approach it is believed that we currently see what could be called a “second wave of polarization”. Here, medium skilled workers are shown to loose employment shares and also experience (relative) wage losses, while low and high-skilled workers gain in employment and wages. This is seen as an effect of occupational change caused by technological change.
But occupational change does have more than one cause, with actors and changes being multidimensionally intertwined. Occupational change happens over time, but also within and between branches, it is driven by firms requirements, technological realities and opportunities but also by qualification supply.
Occupational change is visible in diverse data: there are surveys of firms and also employees, covering aspects of workplaces, labour input requirements and qualifications in two perspectives; one can anticipate occupational change by analyzing job advertisements but also ex post in curricula of regulated trainings and degree programmes.
Using substantive work on the question of polarisation in Germany as an example, our presentation will show how to combine and compare different perspectives with different survey methods and methods of analysis in a combining methodological and theoretical framework describing occupational contents and their changes. This is done in an ongoing research project at BIBB. We use data from the BIBB Qualification Panel for the analyses of firms’ perspectives on whom they appoint which tasks, where they take on new production technologies and their (future) recruitment. Data from the BIBB/IAB/BAuA Employment Surveys are analysed for the individual employees’ perspectives with information on singular tasks, requirements, changes at the workplace and more. Data from a pool of nearly 3 million job advertisements are analysed with a document analyses and also learning algorithms in order to capture changes in recruitment practices and requirements of firms. Further qualitative analyses are conducted in expert interviews and curricula analysis.
Methodologically comparable items and scales have been developed with which we can analyse all data within one theoretical framework. This framework (regarding tasks, possibilities for substitution and knowledge requirements) allows us to depict (changes in) occupational contents in the different data-sets and different perspectives on a number of aggregation levels and also individually.
This set-up helps answering the question whether there is polarization in German but it also gives us the possibility to assess potentials for substitutability, potentials for the use of new technologies and thus work equipment and working devices. Working devices can empirically be assessed in different settings, but they can also be used to link different data sources. Among these are employee and firm Level data, linked employer-employee data, and Job advertisements.
Research about the efficacy of the different methods of surveying with children is relatively scarce. Especially when children are asked to contribute opinions, like in self-reported research, studies that examine the validity and reliability of the children’s responses are rare. Bell (2007) provides evidence that survey research can be effectively conducted with seven-aged and older children and, if adapted and age-appropriated, quantitative questionnaires can bring valid and reliable results.
This abstract aims to provide a contribution to the subject by presenting some techniques used during a prior investigation to the survey PrisvEs (Vesuvius Risk Perception) on the perception of volcanic risk in young adolescents. Preliminary research was extremely important to get form the young population a few concepts about our subject of study and give an adequate operational definition to the items of the questionnaire. The preliminary investigation involved two steps: the first was a sampling search, through the administration of a questionnaire with open questions. The second phase developed focus groups, organized with some students of the schools involved in the research.
These two phases will be described with the following operational definition in the final questionnaire. The first phase gave us the representation of two concepts: the dangers and the advantage perceived in living in a volcanic area. Furthermore, the construction of the semantic differential scale will be shown. The second phase, characterized by focus groups, has been particularly useful to bring out the most important conceptual categories for the children about their well-being in their social and relational contexts. In fact, for each dimension emerged we elaborated appropriate items.
The objective of this work is to show both the techniques to get preliminary data from a young population, and the techniques to make those data operational in the questionnaire, for example, the text analysis.
The aim of the study was to check how Structural Equation Modeling (SEM) results are influenced by various data weighting strategies. The results were compared between different statistical software such as M+ and Stata. Interesting was to verify whether alternative strategies of weighting used in various statistical packages applied to one model imply the same, or at least comparable results, and if software used has impact on research replicability. As an example, used for checking this aim, it was chosen examination of G. Bouckaert and S. Van de Walle model on the relation between public performance, satisfaction and political trust. Analysis were performed based on three rounds of European Social Survey (5th, 6th, 7th) and two countries: Poland and Germany. Especially various weights for analysis within country (design weights) or between countries comparison (population size weights) offered in ESS can be sensitive to SEM output/results. Multigroup confirmatory factor analysis was used with models built for three ESS rounds separately as well for all three jointly, and for single country, between countries and between years. In general, looking at various results, like coefficients, latent means differences etc. even different software was used, the analysis showed that we cannot explicitly indicate whether the differences are small/large. It is interesting for further discussion to notice that while analyzing single country, the differences are much smaller, than in the case of multigroup analysis, where data weighting has more significant impact. At end, we would like to share our experience with a critical discussion of these approach if it is worth to apply and if and how using different data weighting in SEM allows to draw convergent conclusions.
Mixing data collection modes provides an opportunity to compensate for the weaknesses of each individual mode. Face-to-face interviewing may be the best mode to keep the survey error as small as possible; it is however, the most expensive. Web interviewing is much cheaper, but leads to under coverage. Mixing modes may however reduce data comparability since different modes a) provide access to different types of people, b) attract different types of respondents, and c) elicit different responses. The decision to mix modes requires survey practitioners to evaluate and quantify the impact of mode on data quality.
This study explores some of the issues surrounding the use of CAWI-Computer-assisted web-based interview surveys, in particular the extent to which data from a CAWI survey can be matched to data from a CAPI-Computer-assisted personal interview. Some hypotheses about what causes differences in data from web-based surveys and face-to-face surveys are discussed. These include i) interviewer effect and social desirability bias in face-to-face methodologies, ii) the mode effects of web-based and face-to-face survey methodologies, including how response scales are used, and iii) demographic differences in the profile of the respondents. Parallel surveys were conducted using CAWI and CAPI methodologies, and data were compared before weighting and following demographic weighting.
Checking the income data obtained from the survey questionnaire revealed problems. The figures did not reflect the real income level; they were deficient in the most part. Therefore, regarding the income of households and individuals, two main processes using administrative data to correct deficiencies are used:
1. Adding receipts from the National Insurance Institute to the total household income.
Table 1: Report of Total Income per Household in the 3rd Wave, Before and After Adding National Insurance Institute Data
As can be seen from the table, 22.4% of households did not report on income at all. After adding data on National Insurance benefits, 95% of the households have an income.
2. Correction and imputation of income data for an individual who reported working last year but did not report on the amount of income from work. This figure is taken from the income tax file according to the work status of sampled person – employee or self-employed.
The individuals' report on income from work was checked according to their employment status. It was found that the extent of answering questions about the individual's income from work in the Longitudinal Survey was low: About 20% of employees and more than half (53%) of self-employed persons did not report on the amount of their income. For these sampled persons – who did not report on their income – the income that appears in the income tax files (worker-employer file) were paired with their ID number. Table 2 below summarizes the process:
Table 2: Extent of Reporting on Income from Work, by Work Status, in the 3rd Wave Before and After Income Imputation
To test the quality of the pairing and data quality after using administrative income data, a comparative analysis was conducted on income data from the Longitudinal Survey versus the income data from the Household Expenditure and Income Survey:
Table 3: Gross Income Estimate in the Longitudinal Survey and Household Income and Expenditure Survey, Before and After Making Improvements in the Longitudinal Survey Data
The table above shows that the imputation reduced the gap between the surveys to only about 4-4.5% in the estimates.
In view of the above results and considering the limitations of the survey, this process of completing income data, as described above, was introduced on an ongoing basis.
Latent class analysis (LCA) has become increasingly common in academia, official statistics and beyond to correct for measurement error in categorical data. In the context of longitudinal data, a special version of LCA, the Hidden Markov Models (HMMs), are more and more commonly being applied for measurement error correction (Biemer, 2004; 2011).
HMMs rely on the assumption that the true (latent) value, which is observed with an error, is related to itself over time in an autoregressive process- i.e. the Markov assumption. Furthermore, they also assume conditional independence of errors (ICE): the probability of the observed value occurring at a given time point depends only on the true value at that time point. These assumptions are often seen as fairly strict and considered disadvantageous. To illustrate, the ICE assumption precludes the possibility that any errors that occurred at the previous time point were copied over to the current time point. Furthermore, by strictly maintaining the ICE and Markov assumptions, it is impossible to model a scenario whereby either the error probabilities or the true initial states and transition probabilities depend on covariates (Biemer, 2011; Blunsom, 2004; Dias et al., 2008; Van de Pol & De Leeuw, 1986).
One possible solution, which allows overcoming these shortcomings, is the use of extended, multi- source versions of HMMs in which several indicators of the true variable are observed for each time point (Pavlopoulos and Vermunt, 2015). While such an extension allows for the relaxation of the assumptions of HMMs, it also introduces new potential challenges. Namely, as the indicators most often come from various different data sources which need to be linked, this methodology introduces the potential for linkage error, which might bias the estimates (Di Consiglio & Tuoto, 2014).
This paper investigates the bias introduced by simulating various degrees and types of both false-negative and false-positive linkage error when using a two indicator Hidden Markov Model. More specifically, the paper looks at the effect of linkage error on the latent transition probabilities between employment with a permanent contract, employment with a fixed-term contract and non-employment using Dutch linked data on individuals from the Labour Force Survey (LFS) and the Employment Register (ER).
The results suggest that linkage error leads to substantial bias only when the individual probability of being subject to false-negative (i.e. to be excluded from the sample) or false-positive (i.e. to be mislinked with another individual) linkage error depends on a covariate that is highly correlated with the true transition estimates. In all other cases, our results are very promising: our HMM corrects to a very large extent for linkage error and produces estimates that are very close to the case of perfect linkage.
In many cases the researcher faces the challenge of measuring a latent construct, which assumes an ordinal level of measurement. There are several models of latent class analysis that allows to obtain the latent variable with ordered classes. Is there any difference between the results obtained by different models? How big may it be?
In our paper we've compared 3 modelsof latent class analysis which allow to get latent variable with ordered classes. First model is based on approach proposed by M. Croon (Croon, 2002). In this model an order relation is defined on the set of latent classes by imposing inequality constraints on the item response probability in such a way, that one may say that probability of a positive response increases when one runs through the set of latent classes from the lowest to highest one. Items for this model can be dichotonious or ordered.
The second model is based on the approach of C. Clogg (Clogg,1979). С. Clogg considered a model with 3 latent classes based on 3 thrichotomious indicators. In the first latent class only item responses "high" and "medium" is allowed, in the second all responces is allowed, and in the third alowed responces "medium" and "low", and such restictions is the way to get ordered 3-class latent variable.
The third model, proposed by P. Lazarsfeld and N.Henry (Lazarsfeld, Henry 1968), is called latent distance model: individual items are assumed to correspond to points on an underlying continuum and to divide it in an item-specific “positive” and “negative” part. A probability to get "positive" item responce in "positive" part is constant and it higher than in "negative" part
We applied these models for measuring religiosity of a student group. Conceptualization of religiosity was based on approach C. Glock and R. Stark (Glock, Stark 1966; Glock, Stark, 1968), where were suggests four domains of religiosity: intellectual, ideological, ritualistic, experienced. In fact we applied 3 latent class models for each of these domains. We worked out a special questionnaire, wich allow to conduct all 3 models on the same data.
We compare the results by following criteria: the number of classes in the model, the quality of the model, the interpretation of the classes in the model, similarity of the results obtained by different models for the same respondent. In addition we'll analyse correlation between domains of religiosity and gender, parental education, income level, religiosity of the environment and respondent's religious denomination
Clogg C. Some latent structure models for the analysis of Likert-type data//Social Science Research .-USA, 1979. – Vol.8 (4)- pp. 287-301
Croon M. Ordering classes// Applied Latent Class Analysis, Cambridge: University Press, 2002. – pp.137-161
Lazarsfeld, P. F. and Henry, N. W. Latent Structure Analysis. - Boston: Houghton Mifflin, 1968
Glock, Ch.Y. and Stark, R. Christian Beliefs and Anti-Semitism. New York, London: Harper and Row, 1966.
Glock Ch.Y., Stark R. American Piety: The Nature of Religious Commitment. Berkeley: University of California Press, 1968.
Reflecting on social surveys CATI techniques applications, allows the perception of how a hard process it was. Ten years after the first telephone interviews, Statistics Portugal identifies 4 complementary management areas:
(a) Contact management: adaptive tools using available information prior to and during data collection cycle. This allows to hierarchize by initial priority position, after each attempt, as result of call’s history analysis and to act on specific sets to maximize response rate.
(b) Respondent management: multi-channel communication strategy, to deal with difficult respondents and get higher response rates and data quality.
(c) Interviewer management: assuring proper resources allocation, giving feedback on performance and online support to technical and behavioral questions linked to interviews.
(d) Monitoring quality: off line listen a sample of total interviews to evaluate questionnaire application and improve quality; reminders to interviewers with newer or better procedures; data analysis and validation maps to verify data coherence.
This areas aim to provide tools to monitoring, communicate and manage all the CATI processes, users and tasks in a integrated way .
Some indicators of our recent past activity are showed in the poster.
Modeling count data has wide applications in sociology, engineering, medical studies and other fields. A classical approach for fitting count data is Poisson regression, but it is of limited use because there exists a number of problems. The first problem is that the variance of the response variable typically exceeds the mean of the variable for the count data, or the so-called over-dispersion problem. The second problem in relation to the count data is that some particular answers dominate the responses.
The most well-known distribution for inflated data is the zero-inflated Poisson distribution, which is a mixture of a Poisson distribution and a distribution at zero. The most popular model is the zero-inflated Poisson (ZIP) regression model. This regression setting allows for covariates in both the Poisson mean and weight parameters. The problem with ZIP is that it can only model one single inflated data value and the inflated value has to be zero. If the over-dispersion problem still remains after modeling inflated values, a single component of the ZIP regression model may not be sufficient to describe the non-zero distribution. Unobserved heterogeneity often results in this over-dispersion phenomenon, which occurs when the sample of responses is drawn from a population consisting of several sub-populations. Poisson mixture models are used to deal with this problem. This approach provides a framework to classify observations into different components of the mixture model.
In this study, we would like to extend the zero-inflated Poisson regression model by incorporating a latent class variable to better solve the unobserved heterogeneity problem. That is, we would like to develop a new model that solves inflation and heterogeneity simultaneously. We formulated a new latent class inflated Poisson regression model and provided the model’s estimation algorithm. We evaluated our model with a simulation study. The simulation studies show that our model performs well with inflated data. The models are applied to sleep data and compared with the existing approaches. The results show that our model with two latent classes provides the best fit. The latent class approach is an alternative when the data are heterogeneous with inflations. Our model demonstrates a better fit than the standard Poisson regression and zero-inflated Poisson regression models for the inflated counts.
Data from web-panels may be susceptible to a special form of non-response bias. Unlike face-to-face or telephone interviews, surveys conducted via the Internet require that respondents are both capable, and willing to interact with an Internet-enabled device. According to the Oxford Internet Survey, 78% of the UK’s population had access to the Internet in 2013, and only 39% of people aged 65 and over use the Internet. How can we build representative web panels when there are systematic differences between the overall population, and those who are capable of participating in Internet surveys?
Work package 7, “A survey future online? Constructing a cross-national probability based web panel system”, also known as “CRONOS”, in the H2020 funded project “Synergies for Europe's Research Infrastructures in the Social Sciences” (SERISS), is well underway. Individuals participating in the 8th round of the European Social Survey in Estonia, Slovenia and the United Kingdom are being invited to participate in the CRONOS web-panel. Interviewers have been instructed to record all concerns a respondent may share when they are invited, enabling researchers to identify individuals who are unwilling to participate due to a lack of digital literacy.
Using data from CRONOS, this presentation will explore how individuals who agree to participate in the web-panel differ from the net samples, and how representative these panellists are with respect to the ESS target populations. We will investigate whether conventional weighting strategies are sufficient, or if additional measures are needed to adjust for non-response bias in online surveys. In CRONOS, all individuals without Internet access were offered free Internet-enabled tablets along with their web-panel invitation. The results of this experiment will be investigated, hopefully shedding some light on whether or not the digital divide within web-surveys can be bridged through targeted incentives directed at those who lack digital literacy.
The main objective of sampling is to obtain a representative sample for an unbiased and efficient estimate within a budget constraint. A population characteristics or measures of N population elements could be interpreted as a vector on N-dimensional space. Similarly, a sample characteristics or measures of n sample elements could be interpreted as a vector on n-dimensional subspace, imbedded in N-dimensional space. The length of a population vector is defined as the square root of the sum of squares of N components. The length of a sample vector is the square root of the sum of squares of n components. A weighted length of a sample vector could be obtained by weighting the n components with sampling weights. We can measure the sample representativeness as a ratio of the weighted length of a sample vector to the length of the population vector (2016). We proposed an index of sample representativeness using vector length and showed that our index was effective in choosing a representative sample with a simulation using Fisher’s Iris data. Our index could be an approximate quality measure of a sample. We also found that auxiliary variable needed to be moderately correlated with the variable of interest to be useful. Choosing a good auxiliary variable is difficult since the correlation between the auxiliary variable and the item of interest is unavailable since the item of interest is unknown before data collection. The current paper is to evaluate the proposed index using simulated data. The 2015 National Health Interview Survey will be used to generate various data sets simulating non-probability samples.
In recent years Social Science Surveys have added a variety of additional ways to gather respondent data in addition to the interview questions such as including cognitive tests, collecting biological specimens, completing physical measures as well as measurements and observations about the respondent’s environment. While it is exciting to see the expansion of ways to measure health and behaviors, the variety of tools continues to present new challenges to systems development and testing, interviewer training and respondent burden. This presentation will focus on the challenges for interviewer training, respondent issues and compliance for a recent study conducted by the University of Michigan in conjunction with the University of Texas-Austin.
For this study respondents 65 years old and older were asked to wear for 5 days an activity monitor and a handheld computer that delivers questionnaires throughout the day as well as records environmental sounds. These devices needed to be individually programmed to match respondent sleep and wake times and dates of participation. This required interviewers to do set-up via websites as well as utilize a variety of software and set-up equipment and protocols. The handheld computer (cell phone) is also used to take pictures in the respondent’s home to gather additional information about the environment.
Interviewers have the tasks to set up the devices, explain their operation and purpose to the respondent and then upload the data from these devices and re-set them for the next interview. Determining the best ways to train all new interviewers presented challenge; finding the right pace to introduce the various types of technology to the group required flexibility by the training staff. Combining instruction manuals, enough hands on-practice and an integrated sample management system was key for this project’s success.
Respondents were on the whole quite willing to give the devices a try, if they agreed to the initial interview then 100% of them also agreed to this additional data collection. They were offered a token of appreciation of an additional $100 for these activities (they received $50 after completing the face-to- face interview). Their actual compliance with each aspect did vary however. In this presentation we will review actual rates of acceptance and of compliance for the various devices.
In a changing world of work aspects of working time regimes play a central role for the health and well-being of employees. However, for Germany no longitudinal representative data on the in depth assessment of aspects of working time and its consequences existed. For this reason the Federal Institute for Occupational Safety and Health (BAuA) established the panel study "BAuA Working Time Survey (BAuA-Arbeitszeitbefragung)". The first round was conducted in 2015 with the aim of describing working time realities in Germany as well as to investigate relationships of working time with health and satisfaction of employees. Further rounds are planned after every two years.
In 2015 about 20.000 employees took part in computer assisted telephone interviews (CATI). Employees were selected if they were in salaried employment of at least 10 hours per week. They were called via random landline (70%) and mobile (30%) numbers. The average interview duration was 35 minutes. The participants reported on different aspects of their working time, further working conditions as well as aspects of their health and satisfaction.
Based on the survey data a working time report was issued to give a first overview of the prevalence and distribution of aspects of working time as well as their role for employees’ health and satisfaction. Amongst others, the working time report gives an overview on working time duration and overtime, shiftwork and weekend work, working time control and working time accounts, variability of working time, on-call work and permanent availability. The results show that several working time demands are quite common in Germany, such as weekend work and working overtime. Also, there is a general tendency, that working time demands are related to worse health and less satisfaction of employees and working time resources are related to better health and more satisfaction of employees.
The BAuA Working Time Survey constitutes the beginning of a long term project on working time reporting for Germany with the aim to provide representative, reliable and longitudinal data on working time and its consequences.
An emphasis of many surveys is the measuring of subjective indicators concerning a wide field of topics. The measurement instruments used for these purposes (e. g. batteries of multiple items) rely on the assumption of measurement invariance. This means, that all respondents have a similar understanding of the measured underlying construct as well as each individual item. To compare means of different groups of respondents we must ensure that these groups understand and respond to the questions in similar ways. Otherwise comparison between groups can lead to incorrect conclusions.
Previous methodological research has shown that measurement invariance is not given for all instruments and groups. Particularly, differences appear between groups of different cultural background (Davidov et al., 2014). Therefore testing measuring invariance is of utmost importance for surveys including respondents with cultural diversity.
The Panel Study “Labour Market and Social Security” (PASS) is an ongoing yearly household panel study of German welfare benefits recipients and is concerned with their living conditions, socio-economic situation and the dynamics of welfare receipt. Culturally the PASS respondents are very heterogeneous due to the large proportion of individuals with a migration background. This is intensified by the rising number of refugees from Syria, Iraq and Afghanistan over the last month, which enter the study through annual refreshment samples.
The recent developments and their socio-political implications increase the need for valid sociological insights. To assure comparability between groups of different cultural background we test multiple measurement instruments for multiple subjective indicators in PASS (e. g. life satisfaction, self-efficacy, gender role, psychometric measures) using a multi-group CFA framework (see Vandenberg & Lance 2000). We operationalize cultural background using questions on migration background, spoken language, religion and nationality.
Undertaking any research can be challenging and research with and for refugees, asylum-seekers and other migrants is no exception. Conducting studies with these populations presents unique challenges due to their diversity and greater vulnerability associated with their migration status. The plight of refugees, asylum-seekers, and migrants, and their resilience to thrive, however, is inspiring and motivates many researchers to assume this challenge. During a workshop organized at a symposium held in Montreal (Canada) in November 2016 on the theme of "Welcoming Refugees: Practices and Policies", researchers, practitioners and community organizations came together to share their research experiences and discuss “lessons learned” and “keys to success” in conducting research with refugee, asylum seeking and migrant populations. Building from this workshop, this presentation provides a summary of these “lessons from the field”. Three themes are discussed: Methodology and methods (recruitment, interviews, interculturality); Ethics (partnerships and collaboration), and Funding (peripheral activities, knowledge translation). Main challenges include: attending to the uniqueness as well as the diversity across and within migrant populations; addressing the complexity of their circumstances and range of factors influencing their health; and assuring that the research is beneficial, and not harmful. To conduct relevant research that addresses the needs of migrants it is essential to: integrate community organizations into the research process; seek input from, and exchange with migrants; maintain a flexible approach; and use research as a means to advocate for and empower migrant communities to effect change.
Rockville Institute has developed and tested a mail survey alternative to the 50-State Survey of Fishing, Hunting, and Wildlife-associated Recreation Survey, which has been administered by telephone or in person interviews since 1955. This survey collects information on the number of anglers, hunters, and wildlife watchers, how often they participate, where they participate, and how much they spend on their activities in the United States. Our goal in developing a mail survey alternative was to create an alternative approach that addresses the challenges presented by declining responses rates and increasing costs for interview studies, but still produces high quality data.
During this conversion, we have pretested various methodology and design alternatives such as color vs. black and white, picture vs. no picture, official vs. less official look, incentive type and mail delivery methods. Our survey design included an initial household screener that collected information on the fishing, hunting, and wildlife-associated activities of each household member 6 years or older. From that data we then sampled household members for each of the three survey types – Fishing, Hunting, or Wildlife-Associated Recreation.
This paper will compare the cost-per-completed-survey metric to the effectiveness of the different approaches we have tested during this conversion. Specifically, we will compare the effectiveness of different incentive levels, of variations in the mailing package, of different sequences of mailings, and cost of different mail delivery methods. Since the survey will be used to estimate totals, such as number of trips and expenditures for outdoor recreation activities, there is concern about “avidity bias,” or the likelihood that those responding will include a higher proportion of participants in these activities than among non-responders. We will examine the relationship between the cost of different data collection approaches and the proportion of participants identified by those approaches.
In the last few years we have witnessed a multiplication in the use of probability-based panels that collect data via online or mixed-mode surveys as an answer to the growing concern with the quality of the data obtained with traditional survey modes (Blom et al. 2016; Bosnjak, Das, y Lynn 2016; Callegaro et al. 2014). The Institute for Advanced Social Studies (IESA), part of the Spanish National Research Council (CSIC), recruited and maintains one of these panels, a tool that provides cross-sectional samples that are representative of the Andalusian households. 3.439 people living in 2.809 households compose the Citizen Panel for Social Research in Andalucía (PACIS) that have served as a sampling frame for three studies conducted in 2015 and 2016.
PACIS members were recruited face to face using the official address list (catastro) to select the units in the sample and off-line members are contacted and interviewed by telephone (landline and mobile). As a result undercoverage is a very limited threat to data quality. However, PACIS is still affected by nonresponse in the recruitment phase (RR3 23.9%) and for each of the cross-sectional studies. Response rates for these surveys go from RR1 47.4% to RR1 54.1% even if respondents receive a 5€ conditioned incentive.
In this paper, we describe patterns of unit and item nonresponse to the second wave of the PACIS survey. The survey was conducted in November 2015 and the questionnaire deals with the opinions about political decision-making. We propose a new estimation technique suitable for survey data affected by nonresponse where auxiliary information exists at both the panel level and the population level. The objective is to benefit from the combined information through a calibration that is as efficient as possible. The primary objective is a reduction of the nonresponse bias. A secondary objective is to achieve a small variance for the calibration estimator.
Background: Low individual and contextual socioeconomic position is associated with cardiovascular disease. Greater cumulative life-course exposure to low socioeconomic position level such as low education and low social class are significantly associated with higher levels of C-reactive protein and fibrinogen inflammatory biomarkers, which may cause increased cardiovascular disease risk. Thus, inflammatory biomarkers could be potential mediators to the association of socioeconomic position and cardiovascular risk.
However, in longitudinal studies, researchers often use complete data for analysis and ignore missing values resulting in the loss of important information from non-response data. Particularly, the English Longitudinal Study of Ageing has a high attrition rate caused by individuals non-randomly dropping out of the survey over data collections.
Aim of this paper: The primary aim of this paper is to examine whether the levels of adulthood inflammatory biomarkers of C-reactive protein and fibrinogen can be explained by life course socioeconomic position characteristics (i.e paternal occupational class, educational level, wealth and adulthood occupational class).
The secondary aim is to evaluate the typologies of missing data and investigate different methods for compensating for missing data. Based on the general mechanisms of missingness: 1. Missingness completely at random (MCAR): when then probability of missingness is the same for all units. 2. Missingness at random (MAR): when the probability of missingness depends only on available information 3. Missingness not atrandom (MNAR): when missingness depends on unobserved predictors and also depends on the missing value itself.
In this presentation, I will describe the different types of missing data in wave 2 (2004) of the English Longitudinal Study of Ageing, a multidisciplinary study with three different data collections in each wave (main interview, nurse visit and blood sample) which includes 9,432 men and women aged over 52 living in England and demonstrate the impact when ignoring missing data in biosocial research and in particular the effect of socioeconomic position characteristics on inflammatory biomarkers. The non-response rates for participants who gave consent for main interview but failed to give blood sample is 20.1% and 20.5%, respectively for C-reactive protein and fibrinogen.
Particularly, I will present models of response that describe missingness in two different data collections (nurse visit and blood sample) of the English Longitudinal Study of Ageing.
I will discuss the findings based on strategies for dealing with the different mechanisms of non-response such as complete case analysis (listwise deletion), weighting, multiple imputations and selection models (Heckman selection model) while investigating the association between socioeconomic position and inflammatory biomarkers.
Primary findings of complete case analysis suggest that people from lower quintiles of wealth have higher levels of C-reactive protein and fibrinogen compared to people of the highest wealth quintile in approximately 6.000 men and women in England after controlling for possible confounders.
Impact: This project will make an important contribution to current understanding techniques and methods for handling missing data in longitudinal studies. Evidence from this first study will inform best practice in biosocial research.
Cognitive interviewing is a common method used to evaluate survey questions while web probing has recently been advocated as a promising new method. Due to the increasing availability of internet nonprobability panels web probing allows a time and resources saving recruitment of respondents and a realization of larger sample sizes. The goal of this study is twofold: 1) Comparing f2f cognitive interviewing with web probing by asking: Does web probing produce similar results as f2f cognitive interviewing? 2) Analyzing how the quality of probe answers changes within the progress of an online pretest. Based on these results, we will propose practical recommendations for web probing as pretesting method.
We will present the results of 508 respondents drawn from a nonprobability online panel who completed an online survey including four questions from the ISSP 2013/2014. The same questions had been tested previously via f2f cognitive interviewing. It is examined whether web probing and cognitive interviewing identify similar problems in these four questions. Additionally, we present data from five online pretests conducted by GESIS. The online pretests consisted of 1 to 14 questions to be tested with between 5 and 18 probing questions, both open as well as closed. The quality of the answers to these probes is evaluated by the percentage of probe nonresponse, the percent of unintelligible responses, and response length.
Vignettes are complex descriptions of situations that respondents have to understand and evaluate. Therefore it is important to offer vignettes that are comprehensible and relevant to everyday life. Visual stimuli are often easier to understand and more intuitive, but creating visual vignettes with pictures or videos takes a lot of effort and it arises the discussion on how visual vignettes are able to improve reliability and validity. Hence the question is, how and to which extend do visual stimuli in vignettes effects the outcome compared to textual vignettes.
The goal is to create a concept of how to use visual stimuli as vignettes. The construction of visual stimuli needs more thought and consideration. It is important to ensure that the stimuli are comparable to another and respondents must take them seriously. Maybe respondents are getting tired more quickly if they have to rate visual instead of textual vignettes. Advantages could be decreased social desirability since respondents understand and evaluate the vignettes quicker and more spontaneous. Also the depiction of the situation may be more realistic and therefore raise more realistic answers. The poster depicts a project to compare both methods and their limitations.
Large scale surveys such as the National Educational Panel Study (NEPS) aim to produce data which are suitable for making population-based inferences. Since their samples are usually not drawn via simple random sampling but rather by complex sampling designs, the data collection institutions usually provide survey weights which reflect the applied complex sampling scheme and advise to use them in data analysis. For descriptive analysis there is broad consent among statisticians that survey weights should be used by applying weighted estimators such as the Horvitz Thompson estimator. However, the role of survey weights is less clear in analytic inference. In fact, sampling design features only have to be considered in the model if the survey design is informative for the respective analysis model in order to avoid biased estimates. If the sampling design is not informative, but is modeled nevertheless, unnecessarily high standard errors might result, possibly blurring otherwise statistically significant findings. Therefore, it is crucial to investigate whether the sampling design is informative for the model at hand. In this talk I present different test procedures for assessing the informativeness of sampling designs.
If the sampling design points out to be informative, the design information has to be incorporated in the model estimation process. In general, there exist two different approaches for realizing this. One approach adds all necessary design variables as covariates to the analysis model. This procedure is referred to as model-based inference. The other approach involves weighted estimators and is referred to as design-based analysis. Often a pure model-based analysis is not feasible since the required design variables are not available. Furthermore, the inclusion of design variables in the model equation might lead to mingling of design information and the scientific problem to be studied which causes considerable complications when interpreting the estimated model outcome.
Here, design-based analysis using survey weights in the model estimation process leading to Pseudo Maximum Likelihood estimators seems to offer a way out. However, these methods are designed to design weights defined as inverse selection probabilities. Thus, their application to the nonresponse-adjusted and calibrated survey weights, usually provided to the data users, is questionable.
I address this problem by studying existing design-based methods for their applicability if only survey weights and not the pure designs weights are available. In this context, I also sketch possible solutions if the available design-based methods cannot be applied appropriately with the survey weights at hand.
There is broad evidence from the field of survey methodology that empirical results differ depending on the respective mode and response scales. Thus, my research examines response scale effects and the impact of survey mode on responses and data quality in surveys about political attitudes towards security and defense policy, which are assumed to be sensitive topics.
To examine these measurement effects, a large-scale method experiment was conducted in two concurrent surveys (CATI and CAPI) with probability samples of the German population. Both surveys included the same 44 items with varying content, stimuli, response labels and number of response options (split-ballot: 4-, 5- and 7-point scale). To analyze interaction effects of the response scale format / survey modes with interviewees, various characteristics were documented. To get more information about the interviewee and the interview situation, the interviewer was asked to assess respondents’ characteristics and behavior during the interview and the interview situation in general. In addition, metadata (response latency, day and time of interview) were recorded. To control for interaction with interviewer characteris-tics, socio-demographics and job experience were included in the dataset.
In the poster presentation, empirical results are presented and discussed.
This poster will describe a project, currently in progress by the Inter-university Consortium for Political and Social Research (ICPSR) and funded by the Sloan Foundation, which aims to develop a broadly accepted system of researcher credentialing. The purpose is to increase the willingness of potential data providers to share data and the ability of researchers to undertake creative analyses of multiple, naturally occurring datasets with the least possible risk to privacy and confidentiality.
Objectives
The project has two main objectives: 1) to survey and analyze the current characteristics and procedures organizations use to determine the conditions of access for prospective data users and to make recommendations to help streamline those procedures and 2) to develop and test software and a Web interface that will verify researcher credentials and match those credentials to the requirements of the data custodian or dataset. ICPSR will seek to establish a set of generally accepted credentials (“ICPSR Seals of Approval”) that would facilitate researcher access to (multiple) restricted data sets and simplify the process for data custodians considering making the data available to the research community.
Proposed Activities
Project staff will review existing requirements for researchers to gain access to restricted data from multiple data providers and build on existing work in this field already completed by the Data without Boundaries project and CESSDA. Staff will also:
• Interview individuals responsible for producing and disseminating restricted data, users of restricted data, and data custodians of commercial and administrative data.
• Analyze type and scope of requirements and their relationship to types of data and modes of access.
• Produce white paper summarizing the current state of the field and proposing standardized credentialing criteria.
• Build software and an accompanying web interface to automate the credentialing and data access processes.
• After testing, garner feedback on both the recommendations and the software from users, data providers, and organizations who provide administrative and other restricted data to researchers.
Expected Products
A written report (white paper), disseminated on ICPSR’s Website and in other ways, and software and interfaces needed to serve as a bridge between researchers’ credentials and data access requirements.
Expected Outcomes
The software and recommendations will become part of the infrastructure of social research. A clearly defined set of standards that protect individual’s privacy while allowing for valuable social research would be made available to accompany the ever-expanding realm of data used by social and behavioral scientists to understand the world.
The PhD panel study of the German Centre for Higher Education Research and Science Studies (DZHW) examines the learning conditions, career entry and career development of doctorate holders who successfully completed their PhD at a German university in 2014. One of the project’s main research focus is to investigate how different learning and development conditions (learning environments) encountered by doctoral candidates during their doctoral studies influence their transitions after completing their doctorate and their further career paths within or outside the academic field.
Since an appropriate survey instrument did not exist so far, we developed an instrument to assess learning environments during doctoral studies. To meet the requirements that result from our target population and research design, the instrument firstly has to be suitable for doctorates from different subjects and forms of doctoral studies (for example doctorates within a research assistant position, graduate school or grant program). Secondly, it needs to be applicable in multi-topic surveys with large samples and thirdly, it should be usable in paper-pencil as well as in online surveys. As the instrument will be used in other surveys like the German National Educational Panel (NEPS) as well, we also had to consider that it has to be equally suitable for doctoral students and PhD holders.
The proposed poster will illustrate the theoretical conceptualization of our survey instrument, show how we preceded with its empirical development and testing, and finally introduce a thoroughly tested instrument that can be used to measure the learning environment during doctoral studies in standardized questionnaires.
As a theoretical framework, we use the SSCO model which distinguishes four major dimensions of learning environments (structure, support, challenge, and orientation). Based on these theoretical considerations and existing research findings, we identified subdimensions that seemed especially relevant for the learning environment during doctoral studies and constructed corresponding items. To assess the instrument’s validity and reliability, we initially carried out an expert interview and cognitive pretesting. Subsequently, a first quantitative pilot-study with 1.810 participants was performed using an online access panel. Item analyses, nonresponse analyses, confirmatory factor analyses and multiple group comparisons identified areas for item reductions and improvements. The modified instrument was then used in the initial wave of the PhD panel study. We finally developed a parsimonious item set that proved to be a valid and reliable instrument for assessing learning environments of doctoral students.
The survey instrument includes three dimensions and 11 subdimensions (structure: supervision quality, supervision intensity, and topic consistency; support: professional support, emotional support, support with networking, and support with career planning; challenge: internationality, interdisciplinarity, collaborative research, and discourse participation), each measured with three items.
It is used in the currently running survey of doctoral students in NEPS Stage 7 (From Higher Education to the Labor Market).
The study of self-identification, which usually relies on survey data, poses serious methodological challenges to scholars since the notion of self-identification, as Rogers Brubaker argues, is inherently dynamic. This implies that the context which might entail articulation and contestation of identity should be accounted for in the research. Therefore, prior to factors influencing self-identification could be identified, a researcher should explore if self-identification is stable through time.
This paper is devoted to study of national and sub-national self-identification in Spain. Spain is a
state of autonomies with complex institutional structure designed to meet demands of the heterogeneous citizenry. According to the Constitution, Spain comprises 17 autonomous communities with a status either of a historical nationality or a historical region. We are interested in how different levels of self-identification relate, if self-identification is resistant to challenges, and what factors determine it. The non-referendum popular consultation on the political future of Catalonia held in November 2014 set the agenda which made self-identification a subject of articulation and reflection. This is the reason the study focuses on 2013-2014 period.
Traditionally, national and sub-national self-identification is explained by cultural and socio-economic factors. Those factors might include linguistic diversity, ideological attitudes, unemployment rate, and economic development. Moreover, self-identification might be driven by both personal experience and structural conditions. To estimate the effects by both groups of factors, multilevel regression analysis was conducted. Three sets of binary response models were estimated: the first one for Spanish self-identification, the second one for autonomous community self-identification, the third one for dual self-identification. Level-one data comes from monthly public opinion polls held by Spanish Centre of Sociological Research. There are tree time points to check if self-identification is stable: October 2013, April 2014, and November 2014. Level-two data comes from National Statistics Institute. To account for not immediate influence of structural factors, data points of 2011-2013 were chosen. To get better estimates, the province is used as a level-two analysis unit with total N = 52.
The following result were obtained. Firstly, there is no evidence of time affecting self-identification. This means a feeling of belonging to a community is rather consistent in a short run period and experience no changes under identity contestation. Secondly, the distinction between national and sub-national self-identification should be drawn, since the factors which influence each type of self-identification do not coincide. Finally, the results show that national self-identification is driven by socio-economic factors, while self-identification with an autonomous community is determined by the interaction between socio-economic and cultural factors.
An integral part of making fundamental rights a reality for all in the European Union is collecting comparable data that can be used to inform relevant legislation and policies. Surveys can provide the necessary evidence by exploring people’s experiences, attitudes and opinions with regard to fundamental rights – the extent to which people are aware of their rights and how the laws have an impact on their everyday lives. In order to provide evidence-based advice, the European Union Agency for Fundamental Rights (FRA) is establishing a new EU-wide survey on fundamental rights, with the aim of repeating the survey on regular intervals. However, in order to fulfil the objective of measuring trends over time through a cross-sectional survey means that the survey design needs to be fit for future challenges and changes in the way large-scale population-based surveys are implemented in Europe.
In 2017, FRA will pilot the Fundamental Rights Survey, testing the feasibility of EU-wide data collection using a mixed-mode survey design, where – based on a representative probability sample – respondents are first recruited with a letter to complete the survey online, and a sub-sample of online non-respondents will be contacted by interviewers in person to complete the survey in CASI mode. The pilot will assess the challenges of implementing such a mixed-mode design in each country, to develop a tailored approach for carrying out the full-scale survey in each of the EU Member States while at the same time ensuring the overall comparability of the results as the survey will need to provide results at both EU and Member State level. The pilot builds upon the results of a study, which FRA carried out in 2015-2016 to map the available sampling frames that could be used for mixed-mode surveys, to assess possible design scenarios and to pre-test the survey in six EU Member States, comparing three different interview modes in each country.
Based on the results of the feasibility and pre-test study, as well as the on-going pilot in all EU Member States, the presentation will discuss the challenges of implementing a mixed-mode survey design that ensures representativeness and comparability (across countries and over time) while also tailoring its approach to the boundary conditions that have an impact on survey research (such as availability of and access to sampling frames) in the different EU Member States.
In social research, we very often gather information relating to highly sensitive issues. In these situations using the direct method of interview (asking questions directly to the respondents), the respondents provide often untrue response or even refuse to respond because of the social stigma and or fear. Such systematic response errors lead to social-desirability bias in prevalence estimates of the sensitive behaviors of interest, underestimating socially undesirable activities.
To overcome these problems, methods such as the randomized response technique (RRT) may be used to collect more reliable data, protect respondent’s confidentiality and avoid unacceptable rate of nonresponse. In the RRT, respondents use a randomization device to generate a probabilistic relationship between their answers and the true values of the sensitive characteristic. The RRT has been applied in surveys covering a variety of sensitive topics like racism (Ostapczuk et al., 2009, Krumpal, 2012), drug use (Kerkvliet, 1994, Dietz et al., 2013, Goodstadt and Gruson, 1975, Striegel et al., 2009), abortion or delinquency (Fox and Tracy, 1986, Holbrook and Krosnick, 2010, Lara et al., 2006, Kuha and Jackson, 2014), AIDS (Arnab and Singh, 2010) or academic cheating (Fox and Meijer, 2008).
RR technique was originated by Warner (1965). Warner developed a data collection procedure, the randomized response technique, that allows researchers to obtain sensitive information while guaranteeing privacy to respondents. This method encourages greater cooperation from respondents and reduces their motivation to falsely report their attitudes. The most important claim made for RR is that yields more valid point estimates of sensitive behaviour: there have been many reports that RR achieves more accurate estimates of the prevalence of socially undesirable behaviour than when sensitive questions are asked directly (Coutts and Jann, 2011).
Standard RR methods are used primarily in surveys which require a binary response to a sensitive question, and seek to estimate the proportion of people presenting a given (sensitive) characteristic. Nevertheless, some studies have addressed situations in which the response to a sensitive question results in a quantitative variable, but these are not used as commonly.
In our study, conducted in Spain (where RRTs are not generally used for studies of drug consumption), we took into account quantitative variables in order to make the scope of the study as complete as possible.
This work describes a probabilistic, stratified sample of 1146 university students asking sensitive quantitative questions about cannabis use in Spanish universities, conducted using the RRT, specifically we are using the Barlev method (Bar-Lev et al. (2004)). After analyzing the survey the results of the direct question (DQ) survey and those of the randomized response survey, we find that the number of cannabis cigarettes consumed during the past year (DQ = 3, RR = 17 approximately), and the number of days when consumption took place (DQ = 1, RR = 7) are much higher with RRT. The advantages of RRT, reported previously and corroborated in our study, make it a useful method for investigating cannabis use.
Although CAWI seems to become a standard mode in surveys, methodologists still find differences in data quality when comparing results from CAWI to other modes. It is known that besides the absence of an interviewer, explanations for the differences can be found in the design of the web questionnaire. But web questionnaire design depends to a great amount on the online data collection tool in use. To know more about the sources of data quality differences it is therefore worth to also research the functionality of the web data collection tool itself.
This paper tests how the design and functionality of a newly developed online data collection tool (STATsurv) effects respondent behavior. It researches respondent behavior from the point of receiving the invitational letter, through to the logging in to the survey, filling out the questionnaire until finishing the survey. The paper aims to give practical advice on how to display different question types, error messages, plausibility checks and helping texts in a web survey.
The testing of the online data collection tool was done in the manner of a usability test, the gold standard for testing in the field of software development. It was enriched by the following quantitative and qualitative methods common in the field of survey research:
1. Video taped observation of the respondents logging into the survey and filling out the web questionnaire
2. Self administered standardized respondent debriefing (via an online debriefing questionnaire)
3. Cognitive Interviewing about the different question types and technical features of the web survey
In sum 18 respondents took part in the test. To research as realistic as possible the testing was done at the home of the respondents using their own device. For the questionnaire, easy questions from existing surveys in official statistics were selected so that every question type was represented.
The test gave a deep insight on the following issues:
• Login procedure: Problems arose due to the way login information are displayed in the invitational letter, the complexity of username+password and the appearance of the survey in the google search result list.
• Question types: The more complex questions were displayed, the more unintended effects occurred. Problematic question types are dropdown fields, classification lookup lists and 4 or more column tables.
• Error messages and plausibility checks: Messages were valued to long and not precise enough.
• Explanation and helping texts: Because of the amount of improvement potential the test brought to light here, we suggest giving this topic more room in survey methodology literature.
In the presentation of this paper we will give practical tips about how to tackle these problems when designing a web survey.
In sum, usability testing as a testing method for web surveys appeared as a very useful addition to the toolbox of questionnaire designers. It detected many possible pitfalls for respondents taking part in web surveys. Moreover, it helps producing ideas for design improvement so that response burden is lowered and data quality increased.
The Office for National Statistics (ONS) is investing in the introduction of an online mode to its existing Social Surveys. For social surveys this vision of a future data collection model presents substantial research and survey response challenges. All social surveys in the UK are voluntary and retaining response over longitudinal surveys is heavily dependent upon a positive respondent experience at the first wave of interview.
The focus of this presentation is the Labour Force Survey (LFS) which is the UK’s principal measure for employment statistics. At ONS we are transforming our LFS by redesigning our questions and flow to be more respondent centric whilst still meeting output need. ONS is also taking an online-first, mobile-first. This combined with the respondent-centered approach to mixed-mode design necessitates significant changes to what is the UK’s largest household survey. During this talk we will give a brief overview of the survey rationalisation but concentrate on the questionnaire redesign work including evidence from qualitative and user research using traditional (e.g. focus groups, cognitive interviews) and innovative methods (e.g. usability testing, pop-up research) to gather insight to inform the design approach.
It is important that the research and development work incorporates wider developments and strives to be innovative in order to keep pace with societal changes. This presentation will also set out plans for ongoing research and the context within which this work is being taken forward. It will discuss some of the challenges faced to date and those that we foresee to be on the horizon. We invite discussion and encourage others to share their experiences and recommendations with us.