What We Can Learn from Energy-Saving Program Critiques

Energy efficiency program critiques can help improve programs but we should be cautious not to generalize results.

Energy efficiency can be used to save customers money, reduce dangerous climate pollution, and stimulate the economy. Some recent studies that call into question the magnitude of those benefits may provide useful information to make improvements to particular programs, but we should be cautious about generalizing their findings to all energy efficiency efforts.

There are also additional benefits that programs provide beyond cutting energy waste and protecting the environment. Many offerings greatly improve comfort and health and by saving customers money, provide broader stimulus, such as job creation, beyond direct benefits experienced by the individual participants.

What’s in the studies?

As noted in a 2016 study by the renowned think-tank American Council for an Energy-Efficiency Economy (ACEEE): “to be most useful, a study should not only look at what is happening, but also seek to understand why, and then make recommendations on ways the problems identified can be addressed.” This is good advice and it informs our review of two 2017 critiques that received attention in the efficiency community and news media.

Stick with me as we take a look at the following studies, explore what we should be cautious of, and identify what we should take to heart and use to make sure efficiency energy programs yield as much benefit as possible for everyone.

1. In September 2017, a study used a new approach to evaluate how well actual savings matched the state’s expected savings in California school efficiency programs. While the conclusion showed the resulting savings were lower than originally predicted, it is important to focus on the fact that estimating savings is not easy. Yes, actual savings turned out to be lower but that doesn’t mean these schools didn’t reap extensive benefits.

For example, the California Energy Commission’s snapshot of the Proposition 39 programs shows they have thus far yielded more than $94 million in energy cost savings, which they would not have experienced had there been no program. In addition, some schools fixed or improved the heating, ventilation, and air conditioning systems, which could show fewer savings than expected because they can now run the system more for the same or lower cost. This would provide substantial uncounted benefits to student and teacher comfort and classroom experience.

Nonetheless, the study revives a needed focus on how to improve predictions since saving estimates are used in assessing costs and the resulting benefits—and customers rely on these anticipated benefits to make the energy efficiency investment (such as upgrading windows) worthwhile. However, what this study does not provide is a way to close the gap between predicted and actual savings. Thankfully a lot of smart people have been thinking about ways to solve this problem. Now we need to ensure improved approaches to predicting savings are integrated into future efficiency program design.

2. Earlier last year, a study assessed a U.S. Department of Energy program in Wisconsin that is targeted at low-income households. The study concluded that energy savings were less than what the energy audits estimated, costs of the program were greater than value of energy savings and environmental benefits, and the program could be improved by reducing audit subsidies and targeting incentives to measures with the greatest environmental benefits. While these findings should be investigated for how to improve the program, it is also necessary to put them in a broader context and remember that these results don’t mean the program was a failure or that similar programs are not worth pursuing. Instead, we should take this opportunity to enhance efforts moving forward.

Similar to the first study, there can be misalignment between predicted and actual savings and that is an issue to address in future program designs. As was referenced in the report, it is also critical to keep in mind that these programs provide much more than energy and environmental benefits, especially in the low-income sector where families can now keep their heat at a comfortable level. As seen in the graphic, there are also myriad benefits that people don’t necessarily associate with programs, such as water and sewer savings or wider societal improvements.


In addition, efficiency programs for residences (low-income or not) are among the most costly programs out there. Yet jurisdictions across the country continue to invest in these programs for their benefits beyond energy and include them in a package of programs so more cost-effective programs can balance out the seemingly less cost-effective approaches. This ensures that total benefits overall outweigh the costs.

What’s next?

While we should certainly examine any studies related to the success of efficiency programs and enhance programs whenever we can, we must also remember that efficiency delivers huge benefits—not all of which can be immediately or easily quantifiable. Making sure the overall energy system is using the lowest-cost resource to deliver affordable, safe, and reliable ways to keep our lights on and our heat running is a plus in addition to increasing our comfort and health while saving us money and addressing the energy burden felt most directly in low-income households.

However, we also should keep in mind that no analysis is perfect. Asking detailed questions and scrutinizing the results will allow us to make smart policy and investment decisions that rely on the most relevant and helpful findings from these analyses. This will ensure that all customers receive the best opportunities to cut energy waste and lower their bills while curbing dangerous climate pollution from energy generation.