Assessing Non-Cognitive Skills in Large Scale Assessments |
|
| Chair | Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences ) |
| Coordinator 1 | Dr Daniel Danner (GESIS - Leibniz Institute for the Social Sciences ) |
| Coordinator 2 | Dr Clemens Lechner (GESIS - Leibniz Institute for the Social Sciences) |
Due to time constraint in survey situation, we must typically rely on a simple and straightforward method of assessing respondents’ skills. However, for any skill to be practiced, a person must both possess (a given level of) the skill and be willing to actually use it. We employed this biaspectual concept of skills in the Human Capital Study in Poland (2010-2014), gathering data for more than 50,000 persons randomly sampled from working-age population. This allows us to analyze links between skills self-assessment and willingness to perform jobs requiring particular skills with many relevant variables, e.g. gender, age, education, occupation, etc., under control. By applying the same method to both cognitive and non-cognitive skills, we are in position to analyze differences between particular groups of general skills. In our paper, we present skills framework and research design used to collect data in the project. We report the results of self-assessment vs. willingness correlation analysis within the context of respondents’ occupational status in the labor market.
Non-cognitive skills such as personality traits can predict relevant life outcomes such as education, income, or health. These personality traits can be assessed on an abstract domain level that covers a broad spectrum of individual differences or on a more specific level that captures particular facets of personality. There is a lively debate on whether broad domains of specific facets are better predictors of real life outcomes. We will contribute to this debate by testing whether the broad Big Five domains Extraversion, Agreeableness, Conscientiousness, Negative Emotionality, and Open-mindedness or more specific Big Five facets are better predictors for cognitive ability, education, employment status, income, life-satisfaction, and health. We assessed domains and facets with a 60 item Big Five inventory in a heterogeneous sample of 948 US respondents. Multiple regression analyses reveal that the facets scores are able to predict more variance in the outcome variables (4% on average). Furthermore, the analyses reveal that different facets are differently related to the outcome variables which allow a deeper understanding the relation between personality and life outcomes. The robustness of the analyses is demonstrated by replicating the results in three independent samples (910-1,053 respondents) with different item formats and scale length. In sum, these results emphasize the usefulness of personality variables and its facets for the social sciences.
Further development of both cognitive and non-cognitive skills always depends on the actual level of an individual’s skills to build upon. Hence, the level of non-cognitive skills in early childhood lays the foundation for all coming investments in non-cognitive abilities. Additionally, the malleability of non-cognitive skills is higher in early years. The fact that the formation of non-cognitive skills particularly matters in early years makes it necessary to learn more about relevant determinants of this process. Prior research showed that the family conditions a child grows up with play a central role. Due to increasing maternal employment, the influence of non-parental care on early childhood development gains in importance, and leads to the necessity to consider also the impact of child care characteristics on non-cognitive skill development.
The data of the Kindergarten cohort of the German National Educational Panel Study (NEPS) allows investigating these relationships of family and child care aspects together with social skills of preschoolers. In order to measure this construct, the questions of the prosocial subscale of the well-established Strength And Difficulties Questionnaire (SDQ) (Goodman 1997) were implemented. It was supplemented with the items of the subscale concerning disruptive behavior of the Teacher Assessment of Social Behavior (TASB) (Beelmann 2007).
In this paper, the relationships of the social skills of preschoolers and their socio-economic circumstances are investigated initially. For this purpose, information on parental employment as well as aspects of family composition were included in multivariate regressions. A special focus, however, lies on the consideration of child care information and the past child care biography to learn more about the influence of these experiences on the preschoolers' social skills. With this in mind, data like age at first child care experience, group composition, or the educator’s occupational education also were considered.
Due to the multi-informant perspective of the NEPS surveys, an evaluation of children’s social skills is available from the perspective of the responding parent as well as the responsible educator. The comparison of these assessments shows that they differ strongly, with parents appraising the abilities of their children more positive than their educators. This leads to different results when identifying the relevant determinants, depending on the perspective of evaluation. For example, there are significant relations for parental education and the child’s social behavior only in the regression with the educator’s assessment as dependent variable. These results suggest that it is not only the social behavior which depends on the considered aspects, but also the subjective perceptions might be influenced by these factors. This leads to the challenge that, in order to answer the underlying research question, it has to be quantified how strong the considered determinants correlate with the social skills per se, and which relation can be reduced to the bias of the subjective assessment. This paper tries to find a solution for the derived issue stemming from the subjectivity of the skill measure used in these analyses.
Acquiescent response style (ARS) occurs when respondents systematically agree with survey items with Likert-style response scales. ARS is a threat to cross-cultural survey research, as multinational research indicates that use of ARS differs across cultural groups. In the United States, ARS may be of most concern when surveying Latinos, who have been shown to engage in ARS at substantially higher rates than non-Latino whites. Little is known, however, about what drives some Latino respondents to acquiesce. This study proposes that ARS is not a product of ethnicity but, rather, a response to specific, respondent-level, item processing and cultural variables that encourage acquiescence. This premise is explored by testing three hypotheses among a sample of 1296 Mexican American, Puerto Rican, and Cuban American telephone survey respondents: (1) ARS is associated with cognitive item processing variables (inversely with education, health literacy, and numeracy; positively with age); (2) ARS is positively associated with relevant cultural traits (e.g., collectivism, simpatía, personalismo); and (3) ARS does not differ across Latino ethnic groups, acculturation levels, or other sociodemographic variables when controlling for item processing variables and relevant cultural traits. This research will provide a deeper understanding of the determinants of ARS and complements prior research in which ARS and cultural traits have been measured at the national or group levels.
Data were obtained through a telephone survey of Mexican American (n=446), Puerto Rican (n=426), and Cuban American (n=424) adults. Prospective participants were screened for ARS using a 20-item eligibility screener, and the sample was stratified by both ethnicity and acquiescence (50% acquiescers/50% non-acquiescers). Eligible participants defined themselves as being from one of the three Latino ethnic groups (and not bi- or multi-ethnic), were classified as being an acquiescer or non-acquiescer, were between ages 18 and 90, and spoke English or Spanish. ARS was measured using the 20-item eligibility screener and an additional 15-item scale, both which were developed for this study. Four variables were assessed as indicators of respondents’ ability to cognitively process survey items: education, health literacy, numeracy, and age. Additional demographic variables were also measured, including income, gender, acculturation, and years lived in the United States. Nine cultural traits were assessed: collectivism, familism, more polite vs.submissive male/female gender roles, dogmatism, value for sincerity in interpersonal communication, simpatía, personalismo, trust in others, and respect for elders. These cultural traits were assessed using pre-existing measures and scales developed specifically for this study through formative research.
As our data collection was recently completed, we do not yet have findings to report. In our presentation, we will report the results of multi-level analyses to test our hypotheses, identify influential determinants of ARS, make comparisons on key variables related to ARS among the three Latino ethnic groups, and evaluate the relative influence of item processing, sociodemographic, and cultural variables on respondents’ tendencies to engage in ARS.