New technologies to improve the measurement of household finances |
|
| Chair | Mr Alexander Wenz (University of Essex ) |
| Coordinator 1 | Professor Annette Jäckle (University of Essex) |
| Coordinator 2 | Professor Mick Couper (University of Michigan) |
Real Time Measurement of Household Electronic Financial Transactions in a Population Representative Panel
Marco Angrisani
Arie Kapteyn
Swaroop Samek
Center for Economic and Social Research
University of Southern California
In this paper we report on our experiences collecting data on household financial transactions, balances, and expenditures from electronic records in a population-representative household Internet panel (the Understanding America Study, UAS). The data are combined with survey information from the panel and made available to the research community with minimal delay.
To date, the measurement of financial and consumer behavior mainly relies on self-reports, which are burdensome for respondents and tend to suffer from serious measurement and selectivity issues. Since most individual financial and spending behavior (use of payment instruments, investments, assets and liabilities) is recorded in electronic form, direct access to electronic records potentially both improves accuracy of measurement and reduces respondent burden. In this project, we collect financial electronic records directly through the financial institutions of respondents in the household panel. We have designed and implemented a data collection system, using a contract with a major financial information aggregation firm. Participating panel members have consented to share with us their financial information through the aggregator platform.
Fundamental to this project, by relying on the UAS, we can combine survey information with information from the electronic records. In the paper we document various features of the project: consent rates, completeness of electronic records, and comparisons with self-reports. The comparison with self-reports serves two purposes: it yields insight into the differences between self-reports and electronically measured expenditures, thereby providing information on the accuracy of both, and it documents selectivity of the group that consents to participating in the project (about 60% of those who are asked). Panel members are incentivized to participate by monetary rewards depending on the number of accounts they share with the financial aggregation firm and on how well they keep that information current. The size and structure of the incentives is varied experimentally across respondents so that we are able to determine which incentives are most effective.
Smartphones offer new measurement possibilities, because they can deliver short surveys ‘close to the moment of experience’ and can employ in-built features such as cameras, sensors or GPS tracking. Nevertheless, the use of approaches of this kind beyond small-scale, volunteer-based research depends on the level of uptake that can be achieved and an assessment of any biases. In this paper, we examine participation in a supplementary study designed to measure household expenditure using a receipt scanning App as part of a nationally representative household panel study, Understanding Society. We invited 2,432 members of the UK Household Longitudinal Study Innovation Panel to download an App to report all their spending on goods and services for a month. Respondents were asked each day to scan all receipts received, report spending for which they did not have receipts, or report that they had not made any purchases that day. The incentive scheme included an experiment with a conditional incentive of £2 versus £6 for downloading the App, followed by a daily conditional incentive of £0.50 for using the App, and a £10 bonus for using the App every day. The App data collection and its accompanying web surveys will be completed in December 2016.
We first consider the level of participation in the study at different stages of the data collection. What proportion of the sample had the required technology (a smartphone or tablet) to participate in the task? What proportion had said they were hypothetically willing to participate in such a task at a prior wave? What proportion actually downloaded the App? How did participation evolve over the month? What impact did the conditional incentive for downloading the App have? We then examine biases in the types of sample members who participated, considering social-demographic characteristics, financial position and behaviours, mobile device ownership, usage patterns and self-rated skills, and indicators of cooperativeness collected in previous waves of the panel. This paper will contribute to an assessment of whether a Mobile App survey can be used to improve the measurement of household spending and to the broader debate about the potential for innovative data collection approaches of this kind.
Smartphones offer new measurement possibilities. They can be used to capture hard-to-report behaviours through passive monitoring or by delivering short surveys ‘close to the moment of experience’, thereby reducing reliance on respondent recall. At the same time, they can employ in-built features of mobile devices such as cameras, sensors and GPS tracking to capture new kinds of data that go beyond traditional survey responses. As well as depending on reasonable levels of response, these kinds of approaches will only be widely adopted if their promise of better measurement is delivered.
In this paper we examine the quality of the data collected in a supplementary study designed to measure household expenditure using a receipt scanning App as part of a nationally representative household panel study, Understanding Society. Members of the UK Household Longitudinal Study Innovation Panel were invited to download an App and to report all their spending on goods and services for a month. Respondents were asked each day to scan all receipts received, to report spending for which they did not have receipts or to report that they had not made any purchases that day. Scanned receipts were coded to extract information such as category of spending and product codes of itemized purchases. This approach has the potential to increase the accuracy of reports of household expenditure by capturing easily forgotten shopping occasions and by removing the need for respondents to sum their spending over time, as well as providing more detailed information than could be achieved through a standard survey including linked data from nutritional databases.
This paper evaluates the quality of measurement in two ways. Firstly, we consider outcome quality. We examine estimates of monthly individual and household expenditure, comparing the App data with expenditure data from previous waves of the panel, as well as drawing on respondent self-assessments of the proportion of expenditure missed due to non-participant household members, drop-out and under-reporting. Second, we consider process quality. We examine the pattern of receipts returned over time, the proportion of spending occasions verified by a scan, the proportion that are successfully coded and the quality of coding. If time allows we will consider factors which may explain variations in quality of data such as the socio-demographic characteristics of participants, their financial position and behaviours, their mobile usage patterns and self-rated skills, and indicators of cooperativeness collected in previous waves of the panel.
The U.S. Bureau of Labor Statistics (BLS) is currently working towards a redesigned Consumer Expenditure Survey (CE) in response to increasing evidence in measurement error, declining response rates, and the emergence of new data collection technologies. Moving away from a two-part survey, with independent household diary and interview samples, the redesigned CE proposes to use a single-sample design. Each household will complete two personal interviews, and each household member 15 and older will complete a one-week spending diary. Each respondent will be given the option to keep an online diary or a paper diary to record their daily expenditures. The online diary was incorporated into the redesign plan to allow greater ease of use for the respondent and to increase the potential for “in the moment reporting”. The use of an online diary also provides the opportunity for real-time monitoring by interviewers, allowing them appropriate information to follow-up with the respondents to encourage the diary keeping task and address any potential issues during the diary keeping week.
Prior to implementing this significant design change, many large and small research efforts have been undertaken to evaluate both individual features as well as the design as a whole. The most recent test was part of the 2015 Proof of Concept test, fielded to test the feasibility of the overall redesign plan. This presentation will provide an overview of this test, focusing on the diary keeping protocols, and highlighting results. We will look at online take-up rate of the diary, quality comparisons between the online diaries and paper diaries, and comparisons of expenditure reporting counts and levels between the two modes. Additionally, we will discuss some of the limitations encountered to date with the online diaries and next steps.