The Cost of Saving Electricity: A Multi-Program Cost Curve for Programs Funded by U.S. Utility Customers

: This study analyzed the cost performance of electricity efficiency programs implemented by 116 investor-owned utilities between 2009 and 2015 in 41 states, representing about three-quarters of the total spending on U.S. efficiency programs. We applied our typology to characterize efficiency programs along several dimensions (market sector, technology, delivery approach, and intervention strategy) and report the costs incurred by utilities and other program administrators to achieve electricity savings as a result of the programs. Such cost performance data can be used to compare relative costs of different types of efficiency programs, evaluate efficiency options alongside other electricity resources, benchmark local efficiency programs against regional and national cost estimates, and assess the costs of meeting state efficiency policies. The savings-weighted average cost of saved electricity for the period was $0.025/kilowatt-hour (kWh). The cost of saved electricity for programs that targeted residential customers was $0.021/kWh, compared to $0.025/kWh for programs for commercial and industrial customers. Ultimately, we developed an aggregate program savings “cost curve” for the actual electricity efficiency resource during the period that provides insights into the relative costs of various types of efficiency programs and the savings contribution of each program type to the efficiency resource at a national level.


Introduction
Energy efficiency in the United States is pursued through a diverse mix of policies and programs that support and supplement private investments in energy efficiency from consumers and businesses. Examples of these efforts include minimum efficiency standards for appliances and equipment promulgated by the U.S. Department of Energy (DOE), state and local building energy codes, tax credits, a national efficiency labeling program (ENERGY STAR ® ), and efficiency programs that are managed by utilities and other program administrators using utility customer (i.e., ratepayer) funds.
Electricity efficiency programs funded by utility customers are offered in nearly every state, with spending reaching $6.1 billion in 2017, according to the Consortium for Energy Efficiency. Programs target all market segments (residential, commercial, industrial, and agriculture) and include financial incentives, technical assistance, education, and energy audits. Program administrators, primarily electric utilities, respond to multiple policy drivers that vary in scope and stringency. For example, 18 states have adopted binding savings targets, i.e., energy efficiency resource standards (EERS), that require utilities (or third-party program administrators) to meet binding energy savings or minimum spending requirements for a long-term period. In four states (MI, NC, NV, OH), electricity savings from efficiency programs are eligible for compliance with state renewable portfolio standards (RPS) or clean energy standards. Nine states (CA, CT, MA, ME, NH, OR, RI, VT, WA) have enacted legislation that requires utilities to achieve all cost-effective energy savings, while five states (IA, IN, MN, MO, UT) have voluntary (non-binding) savings targets in which utilities propose desired savings levels for review by state regulators. Twelve states (CA, CT, DC, HI, MA, MT, NH, NJ, NY, OH, OR, RI) have introduced a system or public benefits charge as a dedicated funding source for efficiency programs, setting a floor on program spending [1].
Accurate assessments of the cost performance of efficiency programs are an increasingly important policy and regulatory priority given the scope of efficiency programs and declining costs for some supply-side resource alternatives (e.g., wind, solar, and natural gas). The key metric we analyzed in this study was how much it costs to save a kilowatt-hour of electricity, i.e., the cost of saving electricity. Policymakers, utility resource planners, and efficiency program administrators rely on cost performance metrics, such as the cost of saving electricity, to assess energy savings potential, to design and implement programs in a cost-effective manner, and to help ensure electricity system reliability at the most affordable cost as part of resource planning processes.
This study analyzed the cost of saving electricity for efficiency programs implemented between 2009 and 2015 funded by customers of 116 investor-owned utilities and other program administrators in 41 states. In many states, there are significant delays (1-2 years) in reporting actual program-level results for efficiency programs; Lawrence Berkeley National Laboratory (LBNL) continues to track reported spending and savings for electric efficiency programs. It is the most comprehensive effort yet toward quantifying the cost of saving electricity at the program level through efficiency programs funded by customers of investor-owned utilities that serve nearly 70% of U.S. electricity needs [2]. Although other researchers have undertaken efforts to quantify the cost of saved energy [3][4][5][6], these efforts have relied on much smaller datasets of utilities and programs (involving up to 19 states). A major contribution of this study is the development of a first-ever aggregate "cost curve" for electricity efficiency programs, based on the actual efficiency resource during the period. It provides insights into the relative cost performance of various types of efficiency programs and their relative contribution to the efficiency resource at a national level.
This study is organized as follows. We begin by summarizing the data collection process we used to compile the program cost and savings data, describing the program typology we applied to facilitate comparative analysis of similar programs, and discussing the cost performance metrics we developed. Next, we discuss our results for the program administrator cost of saved electricity (PA CSE), both at the national level and by market sector, based on data from 116 program administrators in 41 states. We focus on the results for an aggregate program savings "cost curve" of the actual electricity efficiency resource by the type of program. The cost curve provides insight into the types of efficiency programs that deliver savings at various cost levels and the relative savings contribution of each type of program at a national level. We then present results for the total CSE based on data from 67 program administrators in 27 states (see Section 2.2 for an explanation of the difference between the program administrator CSE and the total CSE). We conclude with a discussion of some of the implications of our CSE study on the evolution of efficiency resource portfolios and potential applications of the efficiency program cost curve.

Data Collection and Analysis Approach
We compiled data for this study primarily from annual demand-side management (DSM) or energy efficiency reports filed with state regulatory agencies by program administrators of electricity efficiency programs funded by customers of U.S. investor-owned utilities in 41 states (see Figure 1). We relied on these annual reports because they typically include data for a portfolio of programs and are publicly available. In cases when particular data were not readily available in annual reports or were ambiguous, we consulted other reports (e.g., from independent evaluators of programs) or solicited information directly from the program administrator or state regulatory staff. At the time of this study, the LBNL Cost of Saving Electricity Database did not include much data on efficiency programs administered by publicly-owned electric utilities and rural electric cooperatives. Tthe oversight entity (e.g., city council, governing board) for U.S. publicly-owned utilities (POUs) and rural electric cooperatives often does not require reporting of program-level data for spending and savings for efficiency programs, which is why they are not included in this study. A small amount of spending and savings for publicly-owned utility territories are embedded in the data for several third-party administrators (e.g., Wisconsin Focus on Energy, Efficiency Vermont) [7].
The dataset in this study comprised 8790 program years. A program year is a year's worth of data for each program in the LBNL Cost of Saving Electricity Database. For example, data covering four years of spending and impacts for a particular program represent four program years. Data fields for each program record (or program year) include the program name, spending information (e.g., budget, actual expenditures, cost breakdown by category), annual and lifetime gross and net energy savings, and participation data (where available) for electricity efficiency programs as reported by 116 program administrators (see Appendix A).

Program Typology
To analyze similar types of efficiency programs, we applied a typology we developed that characterizes programs along several dimensions: market sector, technology, delivery approach, and intervention strategy. Figure 2 provides a partial snapshot of the three tiers in the typology: (1) market sector (blue boxes), (2) simplified program categories (orange boxes), and (3) detailed program categories (gray boxes). The typology includes seven sectors, 27 simplified program categories, and 62 detailed program categories for energy efficiency [8]. (In the residential and commercial sectors, individual building renovation measures, such as windows and roof insulation, are typically not offered as separate programs. Instead, utilities and other program administrators offer multi-measure programs.) We can compare programs that focus on similar market sectors and end-users (e.g., commercial custom rebate and commercial prescriptive rebate) and analyze the differences in program designs (e.g., whole home/direct install vs. whole home/audits).

Cost of Saved Electricity (CSE) as a Metric: Definition and Inputs
The cost of saved electricity (CSE) is an indicator of the cost performance of electricity-saving activities from the economic perspective of the utility. This metric is the cost of activities that avoid the use, and thus the production and delivery, of a kilowatt-hour. The CSE spreads, or "levelizes," the cost of achieving electricity savings over the economic lifetime of the installed measures and treats efficiency program costs as an investment financed with a loan that is repaid over a term equal to the life of the measures (see Appendix B). Researchers initially devised a cost of saved energy metric to assess the costs of efficiency investments compared to the cost of building and operating power plants [9][10][11]. A more comprehensive metric, namely the total cost of saved energy, emerged in the early 1990s to account for both the cost to the utility and the cost to utility customers participating in efficiency programs [12][13][14].
The program administrator CSE accounts for the expenditures in planning, administering, designing, and implementing programs and providing incentives to market allies and end-users to take actions that result in energy savings, as well as the costs of verifying those savings. (We included evaluation, measurement, and verification (EM&V) costs at the portfolio-level and for specific programs (if reported at the program level). We did not include some ancillary costs associated with investments in energy efficiency because they were not reported or included in program administrator annual reports: performance incentives for the program administrator, the time and transaction costs incurred by participants, and tax credits.) The total CSE includes the costs incurred by the program administrator, as well as costs incurred by participants (e.g., the consumer purchase cost of energy-efficient appliances, equipment, or measures net of any incentives paid by the program, such as rebates). We included "net" participant costs to avoid double-counting program incentives such that participant costs are limited to out-of-pocket expenses of the participant. We used a 6% real discount rate as an approximation of the weighted-average cost of capital for an investor-owned electric utility (see Appendix B).
When calculating CSE values, there are choices regarding which annual electricity savings values to use (i.e., gross vs. net savings), as well as the basis for savings estimates (e.g., claimed savings, impact evaluation results). The distinction between the so-called "net" and "gross" savings are important elements of the analysis of the impacts of efficiency programs [15]. Gross savings are defined as the difference in energy consumption with the energy efficiency measures promoted by the program in place versus what the consumption would have been without those measures in place. Gross savings are those associated with the program participants' efficiency actions, regardless of the cause of those actions. The net savings are defined as the difference in energy consumption that is attributable to a particular energy efficiency program. This change in energy use typically includes savings obtained by participants, netting out savings from those customers who are free riders (i.e., who would have installed the measures irrespective of the program) and some consideration of potential spillover effects of the program to non-participants [16].
We used gross energy savings when calculating the CSE primarily because net savings are not universally reported by program administrators. (When net savings are reported, inconsistencies in the definition and estimation of net-to-gross ratios across states add considerably to uncertainties already embedded in the estimates of energy savings; see Billingsley et al. [17] for a more in-depth discussion of our rationale for using gross rather than net savings estimates. When we used net savings data for those program years where we have both gross and net savings, we found that the estimated PA CSE would increase by about 23% to $0.031/kWh.) Program administrators primarily use two methods for estimating electricity savings from efficiency programs: • Claimed savings for a program are typically calculated by multiplying the number of efficiency measures installed (or actions taken) by the ex ante estimates of per-unit savings. These ex ante estimates are often documented in a technical reference manual (TRM) of efficiency measures and actions. A "technical reference manual" is a term of art that describes a document or database of standardized assumptions and ex ante values for determining the savings from well-defined energy efficiency measures installed and operated under defined conditions [18,19]. A TRM may include the methods, formulas, and default assumptions used for estimating energy savings from energy efficiency measures and projects. TRMs are administered and managed at various geographic scales, from regional organizations (e.g., the Regional Technical Forum in the Pacific Northwest) to statewide efforts, by consultants selected by state regulators or program administrators. In some states, individual program administrators maintain less formalized measure lists with deemed savings and measure lifetime estimates for their use. Ex ante estimates are derived using various methods including building energy simulation models, deemed calculation methods, and deemed savings approaches Program administrators also differ widely in assumed baselines, e.g., whether the level of energy performance assumed before installing a measure or taking another efficiency action is based on common practice, a building energy code, or a tiered or dual baseline that changes over the savings lifetime of a measure. Most program administrators also typically have an independent evaluator undertake ex post verification to show that a sample of measures have been installed and are operating properly. • Impact evaluation savings are estimated by measuring the energy use of program participants ex post and comparing this to counterfactual estimates of what this energy use would have been in the absence of the program. Some states and program administrators attempt to incorporate results from impact evaluations to update the deemed savings estimates in their TRM applied to ex ante estimates for future program years.
The savings data in the LBNL database are primarily claimed savings taken from annual reports filed by efficiency program administrators with state regulators. States and program administrators vary widely in the level of rigor they apply in estimating ex ante savings values and the frequency with which they update those assumptions as impact evaluations are completed.
The focus of our analysis is on savings-weighted average CSE values. The savings-weighted averages are calculated using costs and savings for all programs over the average lifetime of savings at each level of analysis (e.g., national, type of program, state, and market sector). The cost term includes all spending, including spending on programs for which no savings can be claimed according to state public utility commission requirements or guidelines (e.g., residential audits). Because the averages are savings-weighted, larger programs have a greater influence on the average CSE than smaller programs.

Program Data Quality, Consistency, and Availability: Issue and Challenges
We highlight several issues related to program data quality that can confound the analysis and compromise the integrity of the results: (1) incomplete or inconsistent data reporting, (2) defining and reporting annual and lifetime savings of efficiency measures, and (3) defining and reporting participant costs. Billingsley et al. [17] and Hoffman et al. [20] include a detailed explanation and discussion of these issues and our approach to addressing them.
We developed procedures for standardizing data across utilities and states to address and mitigate these issues. For example, when a program administrator reported only net savings, we obtained program-level net-to-gross ratios for the same program to convert the values to gross savings. Where average program lifetimes were not available, we imputed values by drawing upon an average value for similar programs. We had to impute the program average measure lifetime for about 59% of the program years using average values from the programs where this information was provided. The total cost data presented unique challenges for data collection and input. For each annual report collected, LBNL researchers ascertained how the program administrator defined the total costs and participant costs and took steps to standardize these values.
We also followed an internal quality assurance and quality control (QAQC) protocol that included flags for aberrant values. Our data entry and QAQC processes helped identify issues that we discussed with program administrators (or regulatory staff). In general, we took all data reported by program administrators as given. The results of LBNL's calculations are therefore highly dependent on values as reported by program administrators.

Results
In this section, we present an overview of the program administrator (PA) cost of saved electricity (CSE) at the national level. Next, we aggregate spending and savings results and depict them in terms of a cost curve that shows the costs of acquiring savings through the most common types of efficiency programs offered by the 116 U.S. program administrators in our sample. We then disaggregate and summarize the PA CSE results for each state. Finally, we present a national overview of the total CSE based on the 27 states that currently report total costs at the program level.

Program Administrator Cost of Saved Electricity: National Results
We included results from program administrators in 41 states that reported impacts and costs at the program level. The PA CSE for the national "portfolio" of all programs and related activities was $0.025/kWh in constant 2016 dollars (see Figure 3).

Figure 3.
Program administrator CSE for electricity efficiency programs by market sector as national savings-weighted averages. * The sample size for the full portfolio included programs for which savings were not claimed but that supported the efficiency activities of the program administrator (e.g., planning, research, and evaluation). The costs for these programs were included in our calculation of the PA CSE at the portfolio and market sector levels. C&I: Commercial and Industrial.
The residential sector provided the lowest cost energy savings during our study period ($0.021/kWh). Residential lighting rebate programs had an average PA CSE of $0.011/kWh and were a key driver of the low CSE values. These programs accounted for 45% of the residential sector's lifetime savings, highlighting the potential for low-cost savings from lighting measures during this period, particularly given that residential lighting accounted for just 15% of residential electricity use in 2009. Appliance and consumer electronics rebate programs had an average CSE of $0.029/kWh and accounted for 10% of the lifetime savings in the residential sector.
Efficiency programs that targeted commercial and industrial (C&I) customers had a savingsweighted average cost of saved electricity of $0.025/kWh. Three types of C&I programs-rebates for custom retrofits; prescriptive rebates for the installation of high-efficiency lighting, HVAC (heating, ventilation, and air conditioning) equipment and controls, refrigeration, and motors; and new construction-accounted for 74% of the C&I sector's annual and lifetime savings. The average CSE values for these three program types were quite attractive, ranging between $0.019/kWh and $0.026/kWh. Programs that specifically targeted small C&I customers contributed 10% of the lifetime electricity savings in the C&I sector with an average CSE of $0.038/kWh.
The PA CSE for low-income programs was about $0.105/kWh. Programs that targeted lowincome households accounted for a modest share of overall savings (2%) and spending (9%). Program administrators typically paid the full cost of measures for these low-income programs and often incurred costs to address issues related to the poor condition of older homes and health and safety issues (e.g., asbestos removal, old wiring) before efficiency measures could be installed. Low-income programs also often had aims beyond energy savings (e.g., lower energy bills, improved health and safety of occupants, and better comfort).

A Multi-Program Cost Curve for Electricity Efficiency
We collected information regarding the efficiency of program spending, savings, and average measure lifetime data at the program level. Using this information, Figure 4 displays an aggregate efficiency program cost curve of the actual, realized electricity efficiency resource during the 2009-2015 period based on program administrators' reporting of spending and savings. This cost curve provides insight into the types of efficiency programs that deliver savings at various cost levels, as well as the relative contribution of each program type to the efficiency resource at a national level. For context, "supply" curves for efficiency measures are a common tool used to estimate the remaining technical, economic, and achievable potential for energy efficiency, typically for a single utility territory, state, or region. Efforts to develop a national supply curve for energy efficiency have also used this approach, portraying the efficiency resource as a composite of measures at a combination of savings potential and cost [21].
We used this concept to create a cost curve for efficiency programs that were actually implemented between 2009 and 2015. Programs are arrayed along the x-axis in order of the ascending cost of the electricity savings (in $/kWh). The y-axis shows the cost. The width of each bar on the xaxis is scaled to represent the lifetime savings of that type of program. The values at the top of each bar show the percentage of total lifetime savings for all programs implemented during the study period for which program administrators claimed savings.
Programs aimed at supporting more aggressive building energy codes were the least-cost efficiency resource, but these programs were only offered in a few states (e.g., CA, MA). Residential lighting and other consumer product rebate programs provided the most lifetime savings nationally (19% of total savings) at the next lowest cost.
Moving up the cost curve, prescriptive rebate programs that exclusively targeted industrial and agricultural customers and prescriptive rebate programs that were mixed (i.e., open to commercial, industrial, and agricultural customers) were the next least-cost efficiency resource. These programs typically offered financial incentives (e.g., rebates) to customers that installed high-efficiency lighting, HVAC equipment, controls, refrigeration, and motors. Industrial/agricultural prescriptive rebate programs accounted for 1.5% of lifetime savings nationally, while C&I prescriptive rebate programs accounted for 15.3% of lifetime savings and C&I custom programs accounted for 16.7% of lifetime savings. As an aggregate, these prescriptive programs had a low PA CSE value of $0.019/kWh.
Programs that offered rebates to C&I customers for custom, often more comprehensive, retrofits had an average PA CSE of ≈$0.022/kWh. Programs that promote more efficient designs in new C&I construction contributed about 6.7% of lifetime savings nationally with an average CSE of $0.023/kWh. Programs that specifically targeted small C&I customers contributed 6.3% of the lifetime electricity savings nationally with an average PA CSE of $0.038/kWh. Programs in the residential sector showed more of a dichotomy between low-cost, high-savings programs that targeted single measures (e.g., compact fluorescent lamps or light-emitting diodes (LEDs)) and higher cost programs that were often aimed at more comprehensive, multi-measure approaches to home energy savings. This range in cost performance is largely a reflection of differences in the program delivery and the measure mix. Whole-home retrofit programs accounted for 2.1% of lifetime savings nationally and typically had a higher cost of savings ($0.069/kWh). Projects were more comprehensive in scope, often including heating and air-conditioning system replacements. In cold climates, air sealing and insulation were common measures. The full cost, not the incremental cost, of these measures was typically used for most cost estimates. These measures also saved heating fuel (often natural gas or fuel oil) in addition to electricity, a benefit that was not accounted for in our CSE metric for electricity efficiency programs.
Behavioral feedback programs rapidly proliferated among program administrators from 2009 to 2015, although they accounted for less than 1% of lifetime savings, primarily because nearly all program administrators during this period assumed that savings from behavioral feedback programs only lasted one year. These programs use mailed and online messages to customers to persuade them to reduce their consumption by comparing their energy use to that of similar households. These "home energy report" messages contained tips on saving energy (e.g., turning down the thermostat in the winter overnight and when not at home) and could serve as gateways to other residential efficiency programs. Such behavioral feedback programs appeared to be among the costlier sources of residential electricity savings ($0.066/kWh) during our study period, given the short assumed lifetime. However, a growing number of evaluations suggest that participants' efficiency behaviors may last longer. Electricity (and/or gas) savings per household tended to be small in behavior-based programs (1%-3%). Thus, program administrators and evaluators have often utilized rigorous experimental designs (e.g., randomized control trials of participants and large sample sizes) to evaluate and estimate the average savings. A meta-analysis [22] of evaluations of the five longestrunning behavior feedback programs recommends using a measure lifetime of 3.9 years. If we had assumed that all behavioral feedback programs had an effective useful lifetime of three years, the savings-weighted average PA CSE for these programs would have been much lower at $0.028/kWh. Given the potential impact on the cost of saved energy estimates, it is incumbent for program evaluators to continue to test and quantify the persistence of behavioral feedback savings as part of rigorous EM&V activities. Figure 5 shows the average program administrator CSE values for all 41 states in our dataset with the dotted red line showing the national savings-weighted average value. The values varied significantly at the state level. Values for 16 states were less than or equal to $0.02/kWh during our study period. These states tended to be concentrated in the Midwest, South, and Intermountain West. Some of these states were relatively new to energy efficiency, were just ramping up their programs with a heavy focus on lighting, or had program design restrictions that limited savings acquisitions (e.g., caps on customer payback periods). Five states had average PA CSE values that exceeded $0.04/kWh. Likely reasons include the following. Four of the states are located in the Northeast (CT, VT, MA, NH) and had relatively high electricity prices, extensive histories in pursuing energy savings, and strong policy commitments (e.g., statutory mandates to acquire all cost-effective energy efficiency or meet specified savings targets). Thus, they tended to have greater market saturation for efficiency measures and have mined more of the lowest cost savings opportunities. The fifth state, Florida, had a significantly higher CSE than the regional average. The state's investor-owned utilities had elevated costs during our study period, which was at least partly because they were required to offer free detailed energy audits to customers (and, as in many states, were not allowed to claim electricity savings for the audits). If we excluded the costs of audit programs, Florida's PA CSE would be 20% lower, or $0.038/kWh, and the PA CSE for the South would be about 6% lower, matching the national average. Figure 6 shows the relationship between CSE values for 2015 by state compared to electricity savings as a percent of 2015 retail electricity sales in the state. The chart provides one way to assess the relative impact of program administrator efforts. State-level PA CSE values tended to increase as states achieved higher electricity savings levels (compared to retail sales), although there was significant variation in this relationship across states. During our study period, 23 states reported program savings that equaled or exceeded 1% of 2015 retail sales. Nine of those states reported savings that exceeded 1.5% of retail sales (CT, AZ, CO, CA, HI, ME, VT, RI, MA), while four states (ME, VT, RI, MA) reported savings that exceeded 2% of retail sales.

The Total Cost of Saved Electricity (Total CSE)
We also report on the total CSE for a subset of program administrators that currently require reporting of total costs at the program level. The sample size for this analysis was 67 program administrators in 27 states for which we compiled 4590 program years of data. The total CSE for 2009-2015 programs in our sample was about $0.05/kWh (Figure 7). The total CSE for programs that targeted residential customers was $0.039/kWh, while the total CSE for programs that focused on low-income customers was $0.145/kWh. The average value for the total CSE for the commercial, industrial (C&I), and agricultural sectors was $0.055/kWh. Figure 7   From a resource investment perspective, the program administrator cost can be regarded as the cost of leveraging investment by participants. To acquire savings across the full portfolio of programs, the program administrator contributed about 54% of total costs, while participants contributed about 46%. (The program administrator cost component of the total CSE in Figure 7 is slightly higher than that reported in Section 3.1. A likely explanation is that for the 27 states that reported total costs, there was heavier representation from states that have pursued energy efficiency for many years. These states tended to offer a more comprehensive set of programs, which often translated into a higher savings-weighted average PA CSE compared to the 41 states in our full dataset.) In Figure 8, we show the total CSE in the four U.S. Census regions. Two distinct total CSE profiles emerged from this regional breakdown: (1) the South and Midwest regions at $0.042 and $0.045/kWh, respectively, where many program administrators were ramping up their efficiency efforts and gaining experience; and (2) the Northeast and West regions at about $0.052 and 0.053/kWh, respectively, where many experienced program administrators had been designing and managing energy efficiency programs for several decades and had higher avoided costs or retail rates, which may have allowed them to justify higher spending on efficiency programs.
As Figure 8 shows, the Midwest region relied more heavily on participant cost contributions than other regions. The region's share of program administrator to total costs was 39% for the study period, and the participant costs to total costs were 61%. By contrast, in the West and Northeast regions, the program administrator costs accounted for 55% of total costs and the participant costs were about 45% of total costs. In the South, the program administrator share was 65% of the total CSE, and the participant cost contribution was about 35% of the total CSE.

Implications of the Cost of Saved Electricity Results on the Evolution of Energy Efficiency Resource Portfolios
In this study, we found that the PA CSE for the national "portfolio" of all programs averaged $0.025/kWh on a savings-weighted basis between 2009 and 2015, while the total CSE averaged $0.05/kWh. We found significant variation in these CSE values by region (the Midwest and South tended to have lower CSE values than the West and Northeast) and by state. The average values for PA CSE for electricity efficiency compared favorably with the levelized cost of energy (LCOE) values for the most common technologies for new U.S. generation facilities today (e.g., wind, utility-scale solar PV, and natural gas-fired combined cycle, where recent LCOE values were ≈$0.03-$0.05/kWh for wind, $0.03-$0.04/kWh for solar PV, and $0.045-$0.065/kWh for natural gas-fired combined-cycle generation) [23,24]. However, the LCOE methods are simplified, may suffer from a lack of comparability and transparency due to differing assumptions and methods, may be too static, and may not account for uncertainties [25]. We also calculated CSE values at the program level for many types of efficiency programs. This more disaggregated analysis can provide insights on key issues that may influence the mix, cost, and performance of energy efficiency resource portfolios going forward. We developed a national "cost curve" for actual electricity efficiency programs during the study period that provides a composite national portrait of the efficiency resource across market sectors (residential and C&I) and program types (see Figure 4).
We drew several major implications from the cost curve. First, residential efficiency portfolios were highly dependent on low-cost savings from rebates for lighting and other residential consumer products. These rebate programs accounted for 19% of lifetime savings for the entire national portfolio. With a PA CSE value of only $0.011/kWh, the programs kept the cost of saved energy down for the efficiency portfolio as a whole. Federal and state energy efficiency standards were raising the energy performance of these residential lighting and consumer products. Over time, these standards may change the baseline against which savings were calculated. (On a per measure basis, the savings differential between compact fluorescent lightbulbs (CFLs) and incandescent bulbs was substantially greater than between CFLs and LEDs. However, technological changes can also enhance lifetime saving, e.g., LED lamps have longer measure lifetimes than incandescent and halogen bulbs and CFLs.) Furthermore, the performance of LEDs continues to improve and unit costs continue to decline. As the market penetration of LEDs increases, program administrators may have reduced opportunities to acquire low-cost lighting savings because an increasing number of consumers may be adopting LED technology irrespective of the efficiency programs, and the replacement cycle for LED lights may be less frequent. Thus, the magnitude of prospective savings may decline for residential lighting programs, which seek to promote next-generation technologies or serve hard-toreach markets.
Second, C&I programs-specifically, rebates for C&I custom retrofits, prescriptive measures, and new construction-delivered nearly half of the national portfolio savings on a lifetime basis. These programs had CSE values ($0.02-$0.025/kWh) that were attractive to program administrators. The bulk of these savings came from large C&I customers. However, in recent years, about a dozen states (mainly in the U.S. South and Midwest) have allowed large C&I customers to opt out of utility efficiency programs. (Criteria for customers that are eligible to opt out of programs vary by state. Many states set a criterion of greater than 1 MW of peak demand. In some jurisdictions, such large C&I customers may choose to opt out of efficiency programs and therefore do not pay for the costs of these programs or receive any of the benefits (e.g., incentives). In other states, large C&I customers are allowed to demonstrate that they have implemented cost-effective measures in their facilities ("self-direction" of their energy efficiency charge), but may still be required to pay a portion of the efficiency charge to support programs designed to meet equity goals (e.g., for low-income households.) This phenomenon often leads to situations where 10%-30% of a utility's load (particularly in the Midwest) was no longer eligible to participate in the efficiency programs offered by the administrator. If this trend continues, it will likely shift reliance for savings in the C&I sector to market sectors dominated by small to mid-size C&I customers. Programs that target small C&I customers tended to have higher PA CSE values in our sample ($0.038/kWh) than programs that targeted large C&I customers, as well as a lower savings potential. Thus, a shrinking C&I market for program administrators may have put upward pressure on CSE values in the C&I sector and the overall portfolio.
Third, the cost curve and our analysis highlighted programs that could potentially put downward pressure on the overall cost of the national efficiency portfolio. Examples included additional evidence that savings from residential behavior programs lasted for 3-4 years (rather than 1 year) and more utilities were developing programs that support the implementation of more stringent state building codes (e.g., code training and enforcement), as well as appliance and equipment standards.

Potential Applications for the Efficiency Cost Curve
Utility planners and policymakers have long struggled with how to model and forecast the potential impacts of energy efficiency in integrated resource planning and transmission planning models [26]. Often, utility planners rely on simplistic or fixed estimates of energy efficiency based on various types of potential studies. Utility system modelers would prefer to aggregate efficiency programs into a "supply" option that assumes that the adoption of energy efficiency changes as the cost to save a kilowatt-hour changes. Thus, energy efficiency supply curves provide information on the marginal cost of increasing amounts of energy efficiency [27].
The efficiency cost curve that we developed in this study (see Figure 4) provides an important element of the empirical basis and data for deriving a supply curve of existing energy efficiency programs as implemented on a national basis. Utility planners would still have to translate our ex post data on efficiency cost curves for future scenarios. However, actual, realized program results allow for benchmarking against efficiency potential studies that rely on engineering estimates of savings. Other researchers have taken the median and inter-quartile values for levelized CSE by market sector from earlier LBNL reports to estimate the first-year CSE and derive a supply curve that matches the cost to actual energy efficiency savings levels [27]. Aggregated, actual efficiency program cost and savings data-rather than fixed, pre-specified estimates of the efficiency potential-offers utility planners an empirical basis to estimate adoption rates for energy efficiency as electricity costs (or avoided costs of supply-side resources) change.

Progress, Challenges, and Future Directions
In recent years, we have witnessed a continued expansion in reliance on energy efficiency programs as a core electricity resource. Program-level reporting of efficiency costs and impacts is increasing. More program administrators are reporting information on customer cost contributions, which allowed us to calculate the total CSE.
At the same time, we found that many program administrators did not provide a complete picture of the impacts or costs of efficiency investments at the program level. For example, program average measure lifetimes are essential for calculating the CSE. However, only 27% of program administrators reported measure lifetime, lifetime savings, or both. This data limitation means that we had to impute the program average measure lifetime for over half of the program years based on average values from programs for which program administrators reported this information. Public utility commissions may wish to consider requiring program administrators to report information on participant costs and to improve the consistency of estimated lifetimes of installed measures located in similar climate regions across states.
Additional information and analysis gaps remain that, if addressed, would enhance our understanding of energy efficiency as a resource. For example, there is increasing interest by utilities, regional grid operators, and regulators in the time-varying impacts of energy efficiency measures [28] and programs [29] and a need to develop metrics and reporting for the cost of reducing peak electricity demand. Frick et al. [30] undertook an initial effort to assess the impact of electricity efficiency measures in reducing peak demand in nine states. Similarly, while LBNL's earlier efforts focused on investor-owned utilities, Schwartz et al. [31] recently completed an initial study on the cost of saving electricity for publicly owned utilities. Additionally, a new study by Schiller et al. updated an earlier LBNL report [17] that included the cost of saving energy for gas utility programs. (Results from the new study, Cost of Saving Natural Gas Through Efficiency Programs Funded by Utility Customers: 2012-2017, is available at https://pubs.naruc.org/pub/7D3531DF-155D-0A36-31E2-75BC84292F7E.) Trends in the cost performance for programs on which utilities are most reliant for electricity savings, and comparing cost performance trends of efficiency and supply-side resources, are also important areas of future research. An examination of such trends could help utilities and regional grid operators better anticipate changes in the future resource mix.
Going forward, efficiency cost curves based on existing programs are a helpful but not yet sufficient element for deriving cost curves for future efficiency programs. While cost curves derived from current programs provide a useful starting place, more analysis and information are necessary. Potential areas of exploration include analysis of the impacts, interactions, and feedback among complementary energy efficiency strategies. For example, minimum efficiency standards for equipment, appliances, and lighting capture some of the remaining potential for efficiency programs, impacting projections of cost curves for future efficiency programs.
Finally, in studies that analyze climate mitigation policy, the potential role (e.g., impacts and cost) of energy efficiency programs is a key area of interest. Efficiency is often highlighted as a foundational element of the climate mitigation strategy of a state, region, or country. However, these studies do not explicitly model utility-customer-funded energy efficiency programs, codes, and standards. Instead, the studies simply assume reductions in end-use energy intensities based on forecasted adoption rates of various energy-efficient technologies over time. Addressing such information gaps and improving modeling practices will further advance energy efficiency as a cost-effective resource and inform sound decision-making to meet energy needs reliably and at least cost and risk.