Proving the negative: The challenge of calculating energy efficiency

Andrew Barbeau, senior clean energy consultant for EDF, contributed to this post.

“Efficiency is good.” That’s the mantra, a known truth, shared by both business executives and environmentalists, who eliminate waste to increase profits and reduce pollution.

When it comes to electricity, efficiency also has proven effective. Whereas power consumption a few decades ago was rising annually at more than 7 percent, the introduction of inexpensive and efficient lightbulbs, refrigerators, and smart heating and cooling has recently led to slight declines in energy consumption, even as the economy boomed and population increased.

Efficiency may be good and effective, but it is inherently hard to calculate. How do you prove the negative? Virtually every state has wrestled with the same questions of how and why electricity use didn’t happen. States with energy efficiency standards – requirements for local utilities to incentivize customers to reduce their energy use year after year – want to know if the investments are cost-effective. With new approaches to calculating energy efficiency, Illinois is tackling that question head on.

Questioning efficiency’s impact

Over the years, some have challenged the merits of energy efficiency programs. A particular target has been federal and state weatherization programs – public-sector investments to insulate low-income homes in order to reduce heating and cooling costs and improve quality of life for the people who live in them.  

The programs also were designed to improve the welfare of low-income residents and provide jobs in target communities.

A few studies have questioned whether these programs can be justified based on electricity cost savings alone, but this inquiry misses the bigger picture: The programs also were designed to improve the welfare of low-income residents and provide jobs in target communities.

Others have wondered whether it’s possible to measure the impact incentives have on efficiency changes. Catherine Wolfram, a director for the Economic and Analysis and Policy Group at the University of California, Berkeley’s Energy Institute at Haas School of Business, appropriately describes how measuring the impacts of energy-efficiency programs requires “disentangling which energy consumption changes can be credited to the program, and which would have happened anyway.”

So, an important question for assessing energy efficiency programs is this: Is there a system in place to separate the direct results (gains in energy efficiency) from the reductions that would have happened regardless?

Fortunately, most states with mature utility-run energy efficiency programs have been tackling this challenge for some time. State public utility commissions, which regulate utilities, typically implement the following measures:

  • Hire third-party evaluators to determine whether utility-run programs create more benefits than costs (known as the Total Resource Cost test),
  • Investigate the Net-to-Gross ratios for efficiency efforts, determining how much savings a utility can actually claim credit for as a result of its policy measures, and
  • Utilize randomized control trials to compare the behavior of regular customers versus those that get incentives to save energy.

And improvements are on the way. As smart meter deployment spreads in states throughout the country, the new granular data presents opportunities for utilities and regulators to get an even more accurate measure of the impact of energy efficiency programs. Peering into half-hour blocks of energy use across a utility’s service territory, for example, could assure regulators that savings will be achieved. 

Is there a system in place to separate the direct results (gains in energy efficiency) from the reductions that would have happened regardless?

To take efficiency to the next level, it’s essential that regulators and utilities are confident in the impact of energy efficiency measures.

Illinois efficiency

Environmental Defense Fund has already helped Illinois become significantly more energy-efficient, but the state is on a path to go further.

One of the key elements of the Future Energy Jobs Act, which became law late last year, is a shift in energy-efficiency goals to an improved approach called “cumulative persistent savings.” This ensures customers see the benefits of energy efficiency savings over not just months, but over decades. Specifically, the bipartisan legislation calls for ComEd, the state’s largest utility, to achieve a 21.5 percent long-term reduction in energy use by its customers by 2030. In order words, even if the utility’s previous energy savings disappear, “cumulative persistent savings” guarantee the utility will meet ambitious long-term efficiency goals. The utility is already beginning to update its energy-saving options as a result of the Future Energy Jobs Act.

[Tweet “Proving the Negative: The Challenge of Calculating Energy Efficiency”]

Illinois also is shifting to incentivized, performance-based ratemaking for energy efficiency programs; the utilities will lose revenue – by up to 200 basis points, or up to 2 percent – if their programs’ verified savings don’t meet targets. Conversely, it also provides a bonus when the companies exceed those goals. Such performance-based ratemaking provides clear financial incentive for utilities to run their efficiency programs effectively, and for regulators to get the data right.

If we are to slash energy-use – creating significant greenhouse gas reductions – we need to get beyond past debates and push into data-driven calculations, understanding causality and mobilizing incentives. By incorporating bold new approaches, Illinois will become part of this emerging debate and analysis, helping show that efficiency is more than “good” and “effective” – it has real, measurable results.

This entry was posted in Energy Efficiency, Illinois. Bookmark the permalink. Both comments and trackbacks are currently closed.


  1. Ned Ford
    Posted April 20, 2017 at 6:51 am | Permalink

    The bullet points above are the core message here. But there is an additional bullet point that ought to be the target of advocacy – which is that the regulatory authority in any state or similar jurisdiction needs to retain its own independent evaluator to review the work of the utility evaluators. This provides an extremely valuable check and balance function which is fairly easy to achieve, but seldom seen as important enough that it actually happens.

    Many states already have utility programs which are reviewed by utility-retained independent evaluators, who determine the net to gross values and incorporate random control evaluation, as well as a number of other approaches to verification. Most of these states make top-down decisions about program scale, budget, compensation and incentives with zero comprehension of the reliability of the knowledge about the efficiency programs because state regulators don’t report the results in simple format that lawmakers can understand, with or without their own verification.

    Incentives are the critical component of efficiency program scale. The failure of the advocacy community to recognize this continues to be the largest barrier to a true market-based impact from efficiency. Incentives can and should be based on verified net savings, and they can and should be a relatively small fraction of those verified net savings in order to make efficiency more profitable to the utilities than supply. There’s no way to generalize about the proper level of incentives – it must be relative to the rates allowed in each state and for each utility, and that varies tremendously around the nation. But the principles are fairly straightforward.

    We should pay the utilities most for doing work that is most valuable to us. Conventional ratemaking pays them most for charging customers the most amount of money, and that generally means serving them in the least valuable manner. No one has figured out a ridiculously simple way of automating performance based ratemaking. But proper incentives for efficiency are certainly worth working for, and they are simple enough that regulators who handle the rest of a utility’s business can handle incentives as well.

    • Dick Munson
      Posted April 24, 2017 at 1:14 pm | Permalink

      Thanks for your good comments, particularly about the value of independent evaluators. We hope Illinois’ Future Energy Jobs Act provides the needed incentives through performance-based ratemaking that rewards utilities when their verified energy-efficiency savings exceed their goals and reduce utility revenues when they fall short.

  2. Kevin Brauer
    Posted April 23, 2017 at 12:02 pm | Permalink

    Healthy & safety are EE externalities seldom given value, and they may be among the most important. Especially when EE building shell treatments improve children’s health.

    • Dick Munson
      Posted April 28, 2017 at 4:15 pm | Permalink

      Good points on the positive health and safety externalities associated with energy efficiency.

  3. Subrata Ray
    Posted April 26, 2017 at 2:35 am | Permalink

    Interesting Katherine.
    I work for a Generation cum Distribution Co in Kolkata India and have a customer base of over 15 Million and counting. We are a regulated Company as well.
    The following drives have yielded results as below:
    This has been divided into two categories viz 1) Commitment 2) Measurement

    1) STAR Rated appliances like white goods are a norm in India now and are slightly highly priced but consumers do not mind. STAR Rating usually starts from Two STARS to Five STARS (obviously the higher the stars more the pricing). However there is no particular regulation as to who should buy what STAR rating. The ususally come with single / double inverters in built for seamless function (thus avoiding step function as in older models) .
    2) Ministry of Power had initiated Lamp conversion with incentive from changing over to CFL and LEDs each having a certain incentive.
    3) The Distribution Company has Slab rates like First 25 Units will be charged a particular tariff and every incremental of 50 Units the rate of tariff increases proportionally until final charge is calculated for the month.
    4) In the Consumers Bills, a comparative Histogram is mentioned about the consumption for the same month a year before for every month billing going forward which definitely serves as an eye opener.
    5) Deployment of smart meters are on and encouragement with incentives are being floated by respective State Electricity Commission towards a Smart meter policy and installation of PV panels in homes.

    There is a dedicated Loss Control cell which measures the distribution losses on a continuous basis and improvement is tracked Y-O-Y.
    Advanced cable fault kits are imported to detect a flaw faster for an even faster remedy as all cables in Kolkata are sub ground.
    Increasingly the Government distribution Companies are leasing to Private Distribution Companies and efficiency gained from the base Loss assessment goes as an incentive to the Private Company. The private company on other hand improves the Sub station equipment quality thereby spending revenue over the period to capture the incentives in the long run as the lease period ranges from 25 years to 30 years.

    • Dick Munson
      Posted April 28, 2017 at 4:15 pm | Permalink

      Thanks for the update on India’s effort. Very interesting.