Photo Credit: Canva
This blog was co-authored by Morgan Rote, Director, U.S. Climate.
In the last year, the U.S. secured historic investments from the Infrastructure Investment and Jobs Act (IIJA), Inflation Reduction Act (IRA), and CHIPs and Science Act that position our country to drive transformational progress on emissions, energy security and jobs in clean energy and manufacturing. Getting those laws through Congress to President Biden’s desk was no easy feat — they’ve been decades in the making — and the next chapter may be just as challenging: implementing those clean energy and innovation investments swiftly and fairly across the nation.
To ensure these laws make the impact we need this decade, federal agencies like the Department of Energy need to design, build and evaluate programs that maximize this funding. Two reports commissioned by EDF offer valuable insights and recommendations on how to best design and evaluate clean energy and innovation programs.
In the first report, Harvard professor Joe Aldy examines how integrating evaluation in the design and implementation of these new clean energy policies can facilitate the learning necessary for policymakers to make policy better over time. In the second, MIT professor Jessika Trancik and postdoctoral associate Dr. Micah Ziegler describe a new methodology developed by the Trancik Lab for identifying promising mechanisms of technological change that can be targeted by policymakers to accelerate innovation. The insights provided in these reports can help ensure that funding from recent climate laws is directed toward maximizing positive impacts for both people and planet.
Here are four key takeaways policymakers should consider as they carry out the programs included in the IIJA and IRA:
1. Program evaluation can improve the outcome and acceptance of public policies like the IRA
Knowing more about the outcomes of policy — the chief goal of policy program evaluation — can be beneficial on multiple fronts, as Aldy describes. It allows us to extract lessons and best practices to improve future policy design. It can enable more effective coordination across federal agencies and offices by providing key details about program delivery and impacts. And by demonstrating the social, environmental and economic benefits of federal investments, program evaluation can enhance public and policymaker support for more clean energy and innovation policies moving forward.
This is not a new concept for the federal government. Aldy outlines how regulatory agencies, like the EPA, offer a template for setting up program evaluation. Energy and environmental regulations are required to go through a rigorous cost-benefit analysis, which means that program designers and implementers must identify key metrics and create plans for measuring outcomes from the outset. Agencies tasked with rolling out new programs under the IRA and IIJA can take a similar approach.
For example, the clean energy tax credits from the Inflation Reduction Act — which have potential to be a major contributor to climate pollution reductions this decade — would be even more effective if paired with a robust program evaluation framework. As EDF recommended in submitted comments, the Department of Treasury should design data collection and management processes for these tax credits on the front end, so we can lessen the reporting burden and unlock an invaluable set of learnings down the line.
2. Program evaluations should take advantage of “natural” experiments to assess policy impacts
One of the main challenges to program evaluation is isolating which factors caused a given outcome. How do we know if a policy enabled breakthroughs in new technologies, or whether the same outcome would have happened regardless? Luckily, social scientists have developed an array of techniques for taking advantage of natural variations in program implementation, such as differences between participating states or grant recipients, to provide more robust estimates of program impacts.
For example, in the report by Aldy, he describes how the roll-out timing around rebates for EnergyStar appliances (under ARRA) in different states provided an opportunity to test whether the rebates truly incentivized new purchases — or if those purchases would have occurred anyway. For example, a state with a rebate that is on for a given appliance could be compared to another state program that is off for that appliance. This approach could be applied to estimate the impact of energy efficiency rebates and other state grant programs funded by the IRA, to the extent that states differ in their implementation timelines.
Aldy also demonstrates how agencies can better determine the impact of competitive grant programs by tracking the outcomes for both grant winners and those just below the eligibility cut-off. Looking at outcomes just above and just below the threshold for funding allows evaluators to develop a reasonable counterfactual scenario for what might have happened to grant recipients had they not received federal funding. For example, with the Small Business Innovation Research Grants, researcher Sabrina Howell focused on pairs of small businesses — one just above the grant cut-off and one just below — to estimate how grant receipt affected subsequent patenting and follow-on financing outcomes. This approach could be used to assess programs in the IRA that operate like competitive grant programs, such as the 48C tax credit.
Taking advantage of these natural experiments can help reveal whether a policy program is successful in meeting its goals, and isolate which parts of a program work best, allowing policymakers to make informed decisions about what funding initiatives may be worthwhile and how to improve a program.
3. Studying how technologies improved in the past can help us accelerate innovation going forward
Understanding how past technologies have either thrived or lagged during their development and roll out can provide useful insights to drive further, rapid improvement. The Trancik lab uses a new methodology to pinpoint those exact mechanisms or “drivers” of technological change, which can help guide funding and policy decisions toward achieving the greatest impact. For example, by applying this methodology, the Trancik lab found that both R&D funding and market expansion policies heavily contributed to the dramatic cost declines in solar PV modules. The importance of both technology-push and technology-pull policies is not unique to solar and likely applies to many other emerging climate technologies, including direct air capture and industrial decarbonization technologies. For example, the Buy Clean program in the IRA can be used as a demand driver for clean building materials and other low-carbon solutions — and similar demand drivers may be needed to help bolster other technologies as well.
4. Data on cost and technology performance is critical to identifying key levers of technological improvement
The Trancik lab’s methodology allows researchers to identify which technological features are more suitable to advancement, as well as which mechanisms are most helpful for driving progress. For example, their research finds that technologies with more flexible designs (meaning parts of the technology can be improved independently of other parts) can advance more rapidly than those with dependencies between components. Lithium-ion batteries, which are commonly used in electric vehicles, offer a good example. A diversity of materials and chemistry combinations can be used to create lithium-ion batteries, explaining why the Trancik lab found that R&D in chemistry and materials science played a big role in driving down their cost.
By collecting data on technologies (e.g., their components, manufacturing processes, performance and cost) and how they change over time, policymakers can identify which aspects are working and which ones still require further innovation — and target policy support accordingly. This does require dedicated and granular data collection, which can be incentivized and streamlined through forward-thinking policy guidance. In many cases, this data collection requires only small tweaks to existing IRA procedures, such as the reporting to determine tax credit bonus eligibility. But the potential technological progress that this stands to unlock could have game-changing implications.
The clean energy and innovation investments in recent federal climate laws are transformative, and they present an enormous opportunity to unlock valuable learnings that enhance progress and drive support for future policy.
We need policymakers to make the most of this opportunity with thoughtful program and technology evaluation.
Read the full reports from Professor Aldy and the Trancik Lab here.
One Comment
Great article!