The setting
Policy makers around the world must adopt ambitious climate policies to limit global temperature rise to the levels agreed upon in the Paris Agreement and to prevent catastrophic climate change. To keep warming below 1.5°C, this means nothing less than turning around centuries of global greenhouse gas emissions growth to net zero emissions in about three decades (IPCC 2022). In fact, increasing greenhouse gas (GHG) emissions during the last decade means that the world has to reduce emissions even faster now (Höhne et al. 2020). Given the urgency of the situation and the historic challenge ahead, there is little room for error in implementing effective, socially just and widely accepted climate policies. Learning from failures and successes of already implemented climate policies is crucial to boost the chances for success of future policies. This will allow for evidence based decision-making and help to start the age of climate solutions at all levels of governance from local to national and international (Minx, Haddaway, and Ebi 2019; Berrang-Ford et al. 2020).
The problem
Despite the requirement of good evidence for decision-making, we still lack a systematic understanding of the relative effectiveness of policy instruments. While a large body of qualitative as well as quantitative evidence on the outcomes of implemented climate policy instruments exists, this evidence is scattered across thousands of primary studies. As scientists especially in health science have long realized, primary research alone is insufficient for policy advice since it is by design limited in its scope, is subject to publication bias, and often lacks generalizability of its findings (Minx, Haddaway, and Ebi 2019). Against this backdrop, there is a strong need for advanced research synthesis methods to bridge the gap between primary research and policy advice.
What I do
In my dissertation I will identify, critically appraise and synthesise existing quantitative ex-post evaluations on the outcomes of climate policy instruments to answer the question which (combinations of) climate policy instruments are successful in reducing emissions, for which sectors, in which countries, and at which socio-economic costs. My research will contribute to the ongoing effort to establish rigorous research synthesis in the field of climate policy analysis and will thereby facilitate further research to analyse other outcomes of climate policies, such as labour market effects, in the same way. To achieve this goal I will
(1) combine the classification of literature with machine learning methods to create a systematic map of the available literature on the outcomes of climate policy instruments, and
(2) develop a framework for a guided and transparent critical appraisal of the quantitative methods for causal inference, employed in the literature on the outcomes of climate policy instruments, and (3) integrate systematic review with meta-analysis methods to rigorously compare the effectiveness of climate policy instruments in a statistical setting.
To carry out my research, I will utilize existing systematic evidence synthesis methods that are specifically designed to address the limitations of traditional reviews. Systematic reviews, based on guidelines by the Collaboration for Environmental Evidence (Pullin et al. 2022), provide a framework for the comprehensive, transparent and reproducible synthesis of evidence (Haddaway and Pullin 2014). Within this systematic review framework, systematic maps provide comprehensive and efficient Heinrich-Böll-Stiftung Promotionsvorhaben Klaas Miersch access to a literature field and are an indispensable basis to guide more detailed evidence synthesis efforts (Minx et al. 2017). In light of the massive and rapidly growing size of the literature, machine learning assisted classification methods are necessary to produce such systematic maps (Callaghan, Minx, and Forster 2020). A second key element of systematic reviews is the critical appraisal of study quality (Haddaway et al. 2020). Despite the existence of general guidelines for critical appraisal, these guidelines are insufficiently detailed to be readily applicable to quantitative evaluations of the outcomes of climate policy instruments. Meta-analyses are an established method to synthesise findings from quantitative studies, grounded in statistical theory (Ringquist 2013; Gurevitch et al. 2018).
References
Berrang-Ford, Lea, Friederike Döbbe, Ruth Garside, Neal Haddaway, William F. Lamb, Jan C. Minx, Wolfgang Viechtbauer, Vivian Welch, and Howard White. 2020. ‘Editorial: Evidence Synthesis for Accelerated Learning on Climate Solutions’. Campbell Systematic Reviews 16 (4): e1128. https://doi.org/10.1002/cl2.1128.
Callaghan, Max, Jan Minx, and Piers Forster. 2020. ‘A Topography of Climate Change Research’. Nature Climate Change 10 (2): 118–23. https://doi.org/10.1038/s41558-019-0684-5.
Gurevitch, Jessica, Julia Koricheva, Shinichi Nakagawa, and Gavin Stewart. 2018. ‘Meta-Analysis and the Science of Research Synthesis’. Nature 555 (7695): 175–82. https://doi.org/10.1038/nature25753.
Haddaway, Neal R., Alison Bethel, Lynn V. Dicks, Julia Koricheva, Biljana Macura, Gillian Petrokofsky, Andrew S. Pullin, Sini Savilaakso, and Gavin B. Stewart. 2020. ‘Eight Problems with Literature Reviews and How to Fix Them’. Nature Ecology & Evolution 4 (12): 1582–89. https://doi.org/10.1038/s41559-020-01295-x.
Haddaway, Neal R., and Andrew S. Pullin. 2014. ‘The Policy Role of Systematic Reviews: Past, Present and Future’. Springer Science Reviews 2 (1): 179–83. https://doi.org/10.1007/s40362-014- 0023-1.
Höhne, Niklas, Michel den Elzen, Joeri Rogelj, Bert Metz, Taryn Fransen, Takeshi Kuramochi, Anne Olhoff, et al. 2020. ‘Emissions: World Has Four Times the Work or One-Third of the Time’. Nature 579 (7797): 25–28. https://doi.org/10.1038/d41586-020-00571-x.
IPCC. 2022. Mitigation of Climate Change. Contribution of Working Group III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge, UK and New York, NY, USA: Cambridge University Press.
Minx, Jan C., Max Callaghan, William F. Lamb, Jennifer Garard, and Ottmar Edenhofer. 2017. ‘Learning about Climate Change Solutions in the IPCC and Beyond’. Environmental Science & Policy 77 (November): 252–59. https://doi.org/10.1016/j.envsci.2017.05.014.
Minx, Jan C., Neal R. Haddaway, and Kristie L. Ebi. 2019. ‘Planetary Health as a Laboratory for Enhanced Evidence Synthesis’. The Lancet Planetary Health 3 (11): e443–45. https://doi.org/10.1016/S2542-5196(19)30216-5.
Pullin, A, G Frampton, B Livoreil, and G Petrokofsky. 2022. ‘Guidelines and Standards for Evidence Synthesis in Environmental Management Version 5.1’. Collaboration for Environmental Evidence.
Ringquist, E.J. 2013. Meta-Analysis for Public Management and Policy. Meta-Analysis for Public Management and Policy. San Francisco: John Wiley & Sons