All time references are in CEST
New Developments and Challenges in Measuring Socioeconomic Status |
|
| Session Organisers | Professor Natalja Menold (Technische Universität Dresden) Mrs Luise Richter (Technische Universität Dresden) Mr Roberto Briceno-Rosas (GESIS) |
| Time | Tuesday 18 July, 11:00 - 12:30 |
| Room | U6-28 |
Socioeconomic status (SES) is a relevant measure in social science research and beyond, serving as fundamental information for public opinion, quality of life, social mobility, health and inequality research. Crises like the COVID-19 pandemic, or energy and economic crises, can impact individuals’ educational paths, labor force biographies, and wages and sources of revenue. The macro-level impact of crises on societies can also impact the frame of reference for evaluating the SES. Hence, an inspection of the current measurement design and quality can be deemed necessary. Typically, socioeconomic status is measured by a combination of information on respondents’ education, occupation and income, which are indicators of the so-called objective measure of SES. In addition, subjective measures of the SES, which aim to capture respondents’ evaluation of their position in society has been commonly used. The operationalization of indicators for both objective and subjective SES is largely heterogeneous.
This session offers the opportunity to discuss new developments and challenges of the measurement of SES in general and in times of crises in particular. The papers address issues like the operationalization of different SES indicators (in national and cross-cultural context), correlations among objective and subjective measures, their economic measurement and data analysis, measurement quality, and potential biases (like social desirability bias).
Keywords: Socioeconomic Status, Operationalization, Measurement quality
Mr Roberto Briceno-Rosas (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Given the importance of the educational attainment of survey respondents for the social sciences, its measurement should be highly reliable. This includes the reliability across survey modes. Survey designers have given abundant attention to the construct validity and international comparability of this measurement, but lesser attention has been directed towards assuring its reliability.
This paper focuses on evaluating the reliability of measurements of education across modes. It asks whether the same instrument design can produce consistent data about the education of respondents when implemented in face-to-face mode and self-completion. For this purpose, a test-retest study of the educational measurement was conducted using data from the European Social Survey (ESS) and its follow-up Web Panel (CRONOS) across three European countries. This allows the evaluation of the measurement error using an instrument with high construct validity and comparability across countries. The reliability of measurement is estimated based on the highest educational qualifications reported at two points in time by same respondents while controlling for changes. The paper also analyses the relationship between inconsistencies and dimensions of the target construct, like the educational levels and orientation of the education, as well as implementation characteristics of the survey, like interviewer effects and device.
The results indicate that the reliability of the measurement of education is yet to reach the expected gold standards. It suggests further research on the reliability of the measurement of education is needed. The paper invites researchers to exercise caution in the use of the educational variable. It also contributes to the discussion about instrument designs more suitable for comparability across survey modes.
Ms Afife Yasemin Yilmaz (SHARE Berlin Institute) - Presenting Author
Social class and status are central to social science research, as they predict and shape an important array of outcomes and are the formative drivers behind numerous social inequalities. To study these outcomes, researchers rely mainly on surveys and use a combination of education, income, and occupation data. However, these measures all pose important limitations, stemming mostly from operational and methodological challenges in the collection and harmonization of data. To circumvent these shortcomings, researchers use derived status indices, such as ISEI or EGP, which can capture class and status better by adding nuance through the consideration of additional variables. However, I argue that these existing indices suffer from three important limitations, which can be traced back to the underlying data and the nature of its collection. The first relates to occupation classifications that are at the heart of the derived status indices, as they do not consider the relational aspects of occupations, particularly power, autonomy, and authority, and thus limit the operationalization of class and status. The second stems from the level of analysis being the individual, which neglects household totality. The focus on individuals’ occupation as the building block of the indices necessarily excludes homemaker partners, which itself can become a status symbol at higher levels of the income and wealth distributions. The third limitation is related to data collection being bound by reporting snapshots of current events and not taking a long-run view of family history and prior endowments. Using data from the Survey of Health, Aging, and Retirement in Europe (SHARE), I aim to propose an index of status that takes considers the relational and cumulative aspects mentioned above. By integrating data from survey questions not usually considered, the proposed framework would nuance our use of status and class as concepts.
Professor Frederick Conrad (University of Michigan) - Presenting Author
Subjective socioeconomic status (SSS), a survey measure of how well respondents believe they are doing in life relative to others, has been shown to reliably predict objective socioeconomic status (SES) as well as overall health and well-being. We investigate the scale used to measure SSS in a web survey of people enrolled across the world in University of Michigan MOOCs, conducted to better understand this global population. The survey under investigation also provided a natural methodological experiment: during the field period (09/2020 - 09/2021) the scale’s orientation was changed from a vertical image of a ladder (higher rungs indicated higher SSS) to horizontally arrayed radio buttons (SSS increased from left to right) to accommodate users of screen readers. This allowed us to examine the effect of scale orientation on responses in the entire sample as well as by the income and culture (Hofstede’s cultural dimensions) of learners’ countries. Scale orientation had no overall impact on data quality (mean response, variance in response, or probability of extreme response) but did interact with Hofstede’s Power Distance dimension: in countries with a higher Power Distance index (strength of social hierarchy) mean responses were associated with scale orientation, thus increasing respondents’ perception of their social standing. Although subtle, the orientation of scales used to measure SSS can affect the point estimates produced in cross national/cross cultural surveys, potentially leading to different substantive conclusions and even policies. But as with other subjective survey measures (e.g., question context effects), it’s hard to say if the difference can be attributed to measurement error or momentary shifts – caused temporarily by the scale’s orientation – in how respondents think about SSS. A within-respondents experiment that disentangles order from scale orientation is needed.
Mrs Luise Richter (Technische Universität Dresden) - Presenting Author
Mrs Natalja Menold (Technische Universität Dresden)
Socioeconomic status (SES) has been an essential concept in social science research and beyond, that been used as a relevant indicator in the research on social inequalities or social societies’ structures. There has also been a long-standing tradition to use objective or subjective measures for SES and to differentiate between objective and subjective socioeconomic status. The objective SES (OSS) is captured by factual information on income, education and occupation aiming to describe material resources, positions and prestige of individuals. Subjective socioeconomic status (SSS) is measured by self-perception of one´s own position in society. A relevant methodological question is about the validity of the measures of SSS, which can be evaluated by means of correlation between both measures. In the literature, different results with respect to this correlation are reported. The objective of this Meta-Analysis is to investigate the correlation between OSS and SSS. Furthermore, we evaluate the use of certain indicators, different measurement instruments and effects of study design, such as study type, sampling strategy or survey mode on the size of the correlation between OSS and SSS. We conducted a systematic literature search following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scheme. 2001 records until July 2021 were identified, of which 353 full texts were retrieved after duplicates were removed and titles/abstracts were screened. 68 Studies assessing OSS and SSS that reported a bivariate correlation between these indicators were included and relevant information from the studies were extracted. First results provide a low, but significant correlation between the OSS and SSS measures, which was mainly independent from the scientific area of the study, but varied dependent on the operationalization of OSS or SSS. We provide implications for both, validity of measures and consequences for substantial interpretation of the SES measurement.