A Recent Evaluation of a Key California Efficiency Program Is Far Too Pessimistic
In an earlier blog, I described how a recent evaluation of a key energy efficiency program that promoted compact fluorescent lamps (CFL) concluded that nearly half of the reported savings would have occurred even without the program.[1] As detailed below, the history of CFL sales in California and across the nation over the past decade suggests that this estimate is deeply flawed and far too pessimistic.
Evaluation of Lighting Efficiency Program Draws Surprising Conclusion
This past spring the Energy Division of the California Public Utilities Commission (CPUC) released its evaluation of an energy efficiency program for lighting implemented by the three largest utilities in the state from 2006 through 2008. [2] This evaluation is particularly important because the program, known as the Upstream Lighting Program (ULP), accounts for over half of the net reported energy savings from the portfolio of programs run by these utilities over that three-year program period. [2]
In brief, the ULP provides incentives to manufacturers of CFLs in order to lower the price of the bulbs and encourage consumers to buy them instead of inefficient incandescent bulbs. Compact fluorescent lamps save a lot of energy and money. For example, a CFL using only 15 watts provides the same amount of light as a traditional 60 watt incandescent bulb. From 2006 through 2008, PG&E, SCE and SDG&E provided incentives to manufacturers averaging $1.57 per bulb on nearly 100 million CFLs. [2]
The factor the evaluation study tried to estimate that had the biggest impact on the final estimate of savings was how many of the CFLs would have been sold in the absence of the program, an estimate known in the energy efficiency literature as the net-to-gross ratio (NTGR) . Unfortunately, as the study reported, none of the complicated statistical analyses used to try to estimate the NTGR produced a useable result. [2] The authors instead chose to use an estimate of 54% based on “best judgment.” [2] In other words, the evaluators estimated that the utility program was only “responsible” for about half the savings that were achieved. Since this program was such a big part of the portfolio, this one estimate has led some to conclude that the utility programs provided far less benefits than anticipated and even that incentives should no longer be provided for CFLs. But an analysis of CFL sales growth trends shows that the opposite is true.
Actual Growth in CFL Sales Far Exceeds Historical Trends
In 2002, total sales of CFLs in California were around 4.5 million per year. Utility incentive programs are estimated to have been responsible for sales of approximately 2.8 million of these lamps. The remaining 1.7 million was due to consumer demand from early adopters and represents the best estimate of what sales would have been in 2002 without the utility incentive program. Total statewide sales dropped somewhat from 2001 to 2002, but there was a modest uptick in sales of approximately 170,000 in 2003. If we assume this growth was not due to program effects, then the rate of sales growth outside the program from 2002 to 2003 was about 10% per year. [3]
As the utilities prepared to implement the new upstream CFL rebate program, the question facing the utilities and the CPUC was what would happen to sales over the next five years in the absence of a CFL rebate program in the state. The year 2003 is a reasonable starting point both because it immediately preceded the significant expansion of the ULP program to its current status and because it could reasonably be considered the most up-to-date sales data available when the utilities developed and the CPUC reviewed the 2006-08 program plans. Obviously, one plausible forecast is that the 2002 to 2003 growth rate of 10% would be sustained. A growth rate of 10% beginning in 2003 would have resulted in sales of 3 million CFLs in 2008. [4]
There is no way to know what would have happened if the California utilities hadn’t run the ULP program, but we do know what actually did happen. In 2008, CFL sales in California totaled 52.1 million lamps with rebates provided to customers for 42.6 million of those lamps. [2] If we adopt the estimate that CFL sales would have continued to grow at 10% per year without the program, then a total of 49.1 million lamps were sold as a result of the program (52.1 million minus 3 million). Since rebates were provided for only 42.6 million lamps, the estimated NTGR should be 115% (49.1 million divided by 42.6 million), more than double the 54% rate based on the consultant’s “best judgment.” [5]
Of course, one might argue that CFLs were actually poised in California in 2003 for even faster sales growth. For example, if sales in other parts of the country grew even faster than 10%, that would be an indication that this forecast sales growth rate was too low. In fact, it turns out that the average rate of growth in CFL sales from 2003 to 2008 in the rest of the U.S. was 37% per year. [6] Of course, efficiency programs in other states were also providing incentives to consumers that led to increased sales. [7] In addition, the California programs were run in part through national retailers and likely resulted in additional, unrebated sales outside California, at least in neighboring states. However, for the sake of argument, let’s assume that the California ULP and programs in other states had absolutely no effect on sales in other states and that, in the absence of the ULP program, California would have enjoyed the same rapid growth in CFL sales of 37% per year. Under this hypothetical scenario, California CFL sales in 2008 would have totaled 8.8 million lamps. That estimate, when compared to actual sales in 2008, results in a NTGR of 102%. [4]
In other words, even if one assumes that the rapid growth in CFL sales in the rest of the US from 2003-08 was not due in any way to the energy efficiency programs implemented over that period and one also assumes that California would have experienced similarly rapid sales growth, then the net benefits of the program are still roughly twice as large as the ULP evaluation concludes.
The Pessimistic Conclusion in the Evaluation Report is Implausible
Let’s look at this in a different way. How large would the growth rate in sales have to have been in California without the efficiency programs to justify the estimated NTGR of 54%? In order to justify a NTGR this low, CFL sales would have had to grow from under 1.7 million in 2003 to 29.3 million lamps in 2008 without utility incentives. The growth rate required to get to this level of sales is 74% per year. (See figure below) In other words, the growth rate in CFL sales in California without a program would have to have been twice as large as the growth rate that actually occurred in the rest of the U.S., during a period in which there were ambitious, well-funded programs in a number of states. As noted earlier, the ULP evaluation provides neither conclusive evidence nor analysis to support this highly implausible result.
Actual Benefits of Lighting Efficiency Program Are Likely Twice as Large as Estimated
To summarize, the estimated NTGR of 54% is based on an implicit assertion that in 2003 sales were somehow poised to grow at the extraordinarily rapid rate of 74% per year for five years running, despite modest at best sales growth from 2001 to 2003. Even if one assumes that sales would have risen at 37% per year based on sales growth in other states, the NTGR in 2008 should be 102%, nearly twice the “best judgment” estimate from the ULP evaluation study. A lower growth rate based on the actual sales growth from 2002 to 2003 would result in a NTGR of 115%. The bottom line is that the proposed NTGR of 54%, which is based solely on a consultant's judgment, is unsupported by readily available evidence and is far too low. The programs were likely responsible for savings twice as large as the evaluation report estimated.
References
1. http://switchboard.nrdc.org/blogs/pmiller/cpuc_should_scrutinize_ulp_eva.html
2. "Final Evaluation Report: Upstream Lighting Program". The Cadmus Group, Feb. 8, 2010.
http://www.energydataweb.com/cpucFiles/18/FinalUpstreamLightingEvaluationReport_2.pdf
3. Sales data is from: "California Lamp Report 2003." Itron, July 15, 2004. This calculation assumes a NTGR of 80% for 2002 and 2003. A lower NTGR would result in more sales in the absence of the program, but a lower growth rate. For example, a 60% NTGR would increase the estimate of 2003 sales without the program by 40% but decrease the annual growth rate to 7%.
4. Assuming a starting point of 1.7 million lamps, sales growth of 10%/year for 5 years results in 2008 sales of 3 million lamps. Similarly, a 37%/year growth rate for 5 years results in 2008 sales of 8.8 million.
5. A NTGR of over 100% means that the program resulted in sales of some lamps in addition to those that got rebates. This could happen because of increased customer awareness and acceptance among other factors. A number of states, including New York, Vermont, and Massachusetts currently credit their CFL rebate programs with NTGRs of greater than 100%.
6. Ecos Consulting analysis of USA Trade Online data
7. "CFL Market Profile." U.S. DOE. March 2009