Power of Survey Research in Evidence-Based Policymaking 2 |
|
| Chair | Dr Young Chun (PSI Institute for Data Science and Interdisciplinary Research ) |
| Coordinator 1 | Ms Giang Nguyen (ISR Foundation Center for Interdisciplinary Research) |
| Coordinator 2 | Ms Clara Kyung (McGill University) |
Introduction
High quality data, including from surveys, are essential to inform evidence-based policy-making. However, considerable investment goes into developing government policies which are often based on poor evidence, and there is usually little rigorous data on their impact. This is especially the case for complex interventions that have multiple interacting components.
The Integrated Care Pioneer programme in England, launched in 2013, exemplifies a complex intervention. The 25 Pioneer sites are integrating health and social care services which aim to improve patient experiences and outcomes while saving costs. We discuss our use of surveys as part of a 5 year evaluation of the Pioneer programme.
Methods
Our evaluation involves an interdisciplinary team carrying out a mixed method study that includes surveys and qualitative data collection alongside the use of linked administrative data (eg, the use of health and social care services). Survey research will be used at two levels: first, at a policy level, to assess the impact of the overall Pioneer programme over its 5-year lifespan; second, to determine the cost-effectiveness of specific interventions Pioneers implement locally to deliver integrated care.
Discussion
At the policy level, we ask a panel of key informants of managerial staff from all 25 Pioneers to complete an online survey 1-2 times/year. The 25 sites vary considerably in geographical and population size, and service integration typically involves a range of organisations including local authorities, health care providers and the voluntary sector. In order to obtain survey results that are comparable across sites and over time, some of the issues that need to be resolved include deciding the types of individuals to include on the panel, how to deal with the different levels of knowledge about the programme among panel members, and how to handle panel turnover.
For looking at the cost effectiveness of specific interventions, we will distribute postal questionnaires to patients, carers and frontline staff, 2-3 times over an 18 month period, to ask about their experiences of the new service delivery models and to measure their effect on quality of (working) life. Signed consent is required to be able to link survey results with health and social care administrative data, which raises the issue of how best to obtain informed consent as part of a postal survey. We will also survey patients, carers and frontline staff in areas without the service intervention in order to make inferences about the integration of health and social care services.
Conclusion
Evidence-based policy making relies on evidence from rigorous evaluation. Surveys can provide an essential component for evaluating complex interventions, but survey implementation is not necessarily straightforward and needs to be tailored for the specific context and circumstances of the policy (and evaluation) in order to provide high quality data that can be used by policy-makers.
The purpose of the paper is to demonstrate the power of data visualization to complement an evidence-based policymaking approach to addressing the national needs of developing countries. As a case study, we the issue of renewable energy in North Korea. Supply of electricity in north Korea is characterized by widespread dependence on hydroelectric sources, comprising of 60% of total output. Severe droughts in 2015 caused extreme shortages of electricity, with the majority of households connected to the power grid receiving only a few hours of electricity per day. This drought exacerbated and brought to focus the ongoing issue of the North’s precarious electricity situation. It is the objective of this paper to explore the complex problem of and solutions to North Korea’s electricity needs, by leveraging data visualizations and creating evidence-based policy prescriptions to address the problem. Analysis is done at both the regional and national level to form a comprehensive picture. Data is gathered from the North Korean Census, meteorological data sources, and international outreach agencies. To characterize north Korean electricity demand, we first map the population density along regional and demographic characteristics. We then go on to overlay the electrical grid with solar and wind maps to develop integrated information and provide an understanding of the potential of renewable energy in North Korea. We conduct a comparative analysis of the North Korean case with nations facing similar energy issues. We demonstrate that renewable and alternative sources such as wind and solar energy can be used to diversify and improve the resiliency of power supply in North Korea to weather and other shocks. We show the extent to which renewable energy capacity may be developed on the local and regional level. Such an approach may contribute to targeted improvements in electricity supply while circumventing the need for costly investment in grid infrastructure. Data-based evidence from this research may inform energy policy decisions of UN organizations, nongovernmental organizations, and neighboring countries.
The economy and labour market are subject to structural change over time. Demographic change, technological progress, and globalisation will frame and steer the behaviour of economic actors. Political planners have a special interest in having some knowledge about the future – be it for budgetary planning, preliminary policy assessments, or the mere identification of possible areas in need of action in the future. More specific, regarding future developments of the labour market, a concern is whether the supply of skills will suffice the demand of the economy, such that growth can spur, or whether there is a possibility of labour shortages.
In this area of research, long-term labour market projections have become a more and more popular tool for policy consulting and as such contribute to the advancement of evidence-based policymaking. The reason for this especially is that evaluating alternative scenarios with a projection model can yield insights on new developments which are expected to be important in the future, however cannot or poorly be traced in past data yet. Examples for this are plentiful; they may grasp topics from digitalized production and polarization of work, to the labour market integration of refugees, or the increased usage of electric cars.
In such analyses, long-term projections can produce broad assessments to give an intuition of the direction and rough size of effects given the scenario formulations. Based on this, areas in which political action may become necessary can be identified and recommendations for policy interventions can be deduced. Furthermore, they visualize the strength of existing interdependencies in the model and by that can give valuable information about the limitations of policies. At the same time, however, there are naturally limitations to the explanatory value of the identified effects which have to be addressed carefully in the outward presentation of results.
In the presentation, we discuss the opportunities but also the challenges of long-term labour market projections in greater detail. We introduce a projection model for the German labour market and display its scope of application to evaluations of future developments and policies. Using this example, we discuss how detailed such a model should be to accurately reflect specific heterogeneities and when too great detail jeopardise the transparency of the results. Further, we highlight the possibilities and limitations of the model to present accurate results and policy proposals.