Solid methodology key to trustworthy climate policy findingsAssessment included multiple stages of review by subject-matter experts

Blog - July 23, 2021 - By Nichole Dusyk
Canadian provincial and territorial flags

Economic stability depends on planetary health, which is impossible without a safe climate. But what policies and measures are necessary to help governments limit warming to 1.5 degrees Celsius and deliver on the promise of a safer, sustainable world for all?

Climate success does not require a uniform approach for every government, but it does require a strong policy framework, under which regionalized elements can fit. Along with a commitment to net-zero by 2050, every jurisdiction should have: increasingly ambitious carbon targets and decreasing carbon budgets for every sector of the economy; a climate plan based on credible modelling showing how targets will be achieved; progress reports for each milestone period; and a requirement to course-correct when targets aren’t met. Importantly, these key elements of successful climate planning should be enshrined in legislation.

Further, to ensure jurisdictions put people first, each must demonstrate a strong understanding of the socio-economic and demographic impacts of climate change and climate policy, provide transition planning for workers and communities, and respect the rights of Indigenous Peoples.

To measure their readiness to act on climate, Pembina Institute collaborated with Simon Fraser University’s School of Resource and Environmental Management to assess the performance of provinces, territories, and the federal government on 24 policy indicators across 11 categories. The indicators represent foundational climate policies and measures to reduce emissions in key sectors of the economy contributing to emissions in Canada: oil and gas, transportation, buildings, and electricity. We hope to add agriculture and waste in future research.

Developing the framework

Cover of All Hands on DeckThe evaluation framework used in All Hands on Deck: An assessment of provincial, territorial and federal readiness to deliver a safe climate is derived from a number of previous studies. The framework was developed and employed in a number of steps.

The first step was a literature review to establish indicators assessing the climate planning process (e.g. setting quantifiable objectives and targets with timelines, developing a plan to meet the objectives, monitoring results and making revisions, and engaging with the public), key components and tools of effective climate policy (e.g. carbon pricing, decarbonizing the electricity sector, decarbonizing the transportation sector) and outcomes (e.g. GHG emissions).

The second step involved compiling and refining the list of indicators to eliminate overlap and redundancy. With input from subject matter experts, a rating system was developed to measure the degree to which each of the criteria were met. For some criteria, the rating system is a binary yes/no or increasing or decreasing rating, and for other criteria a three-tier rating system is used with a green indicating fully meeting or exceeding the criterion, yellow indicating somewhat meeting the criterion, and red meaning not meeting the criterion. For some indicators, major findings of the International Energy Agency’s recent report outlining a path to net-zero emissions were used to determine evaluation criteria.

Verifying the research with governments

The indicators and the rating system were pilot tested on several provinces and the results were reviewed and revised by a research team based on the pilot test. The details of the rating system for each criterion are outlined in Appendix 1 of the report. Because of the relatively low emissions and unique circumstances of the territories, they are evaluated on a reduced list of indicators.

As a third step, climate policies were collected for each jurisdiction. This was initially completed by internet reviews of government sites searching for climate legislation, policies and reports. The relevant information was compiled in a table format organized under each of the evaluation criteria categories. The compiled information was sent to all provincial governments who were asked to review the accuracy and completeness of the policies and data.

Sample indicator rating system - electricityThe fourth step was to complete the ratings for each jurisdiction’s climate policy using the rating criteria and the climate policy information. Initial ratings were made by individual researchers and then reviewed by the research team to reach consensus agreement on the ratings. Governments were invited to review the accuracy and completeness of the data and summary for their region prior to publication.

Summarizing findings, recommendations

Finally, the major findings were summarized and recommendations were developed.

Several qualifications should be noted in reviewing the results of this evaluation. First, climate policy is always changing and the results are only current to June 30, 2021. Second, judgement is required in defining the rating criteria and ratings for each jurisdiction.

Although subjectivity has been reduced by having the rating criteria and ratings reviewed by experts, different assessors may reach different conclusions.


Nichole Dusyk
Nichole Dusyk

Nichole Dusyk was a senior analyst with the Pembina Institute, researching policy recommendations for federal energy decision-making, until 2022.


Subscribe

Our perspectives to your inbox.

The Pembina Institute endeavors to maintain your privacy and protect the confidentiality of any personal information that you may give us. We do not sell, share, rent or otherwise disseminate personal information. Read our full privacy policy.