Energy and Greenhouse Gas Savings for LEED-Certiﬁed U.S. Ofﬁce Buildings

: In this work, we present results from the largest study of measured, whole-building energy performance for commercial LEED-certiﬁed buildings, using 2016 energy use data that were obtained for 4417 commercial ofﬁce buildings (114 million m 2 ) from municipal energy benchmarking disclosures for 10 major U.S. cities. The properties included 551 buildings (31 million m 2 ) that we identiﬁed as LEED-certiﬁed. Annual energy use and greenhouse gas (GHG) emission were compared between LEED and non-LEED ofﬁces on a city-by-city basis and in aggregate. In aggregate, LEED ofﬁces demonstrated 11% site energy savings but only 7% savings in source energy and GHG emission. LEED ofﬁces saved 26% in non-electric energy but demonstrated no signiﬁcant savings in electric energy. LEED savings in GHG and source energy increased to 10% when compared with newer, non-LEED ofﬁces. We also compared the measured energy savings for individual buildings with their projected savings, as determined by LEED points awarded for energy optimization. This analysis uncovered minimal correlation, i.e., an R 2 < 1% for New Construction (NC) and Core and Shell (CS), and 8% for Existing Euildings (EB). The total measured site energy savings for LEED-NC and LEED-CS was 11% lower than projected while the total measured source energy savings for LEED-EB was 81% lower than projected. Only LEED ofﬁces certiﬁed at the gold level demonstrated statistically signiﬁcant savings in source energy and greenhouse gas emissions as compared with non-LEED ofﬁces.


Introduction
One of the most important problems facing humans today is that of climate change, driven by anthropogenic greenhouse gas (GHG) emission, which is due in large part to the burning of fossil fuels. U.S. buildings are responsible for 40% of U.S. primary energy consumption and 33% of its energy-related GHG emissions [1]. In addition to CO 2 , the burning of fossil fuels releases other pollutants, including CH 4 , N 2 O, SO 2 , NO X , and particulate matter, all of which have negative environmental consequences, particularly when released from thousands of buildings in urban centers [2]. When buildings switch from natural gas or other fossil fuels to electric energy, the accompanying pollutants are shifted to the electric power sector, which may or may not reduce total GHG emission (depending on the details of the electric grid) but does improve air quality by moving emissions sources out of population centers. Moreover, power plants do a better job of reducing pollutants (other than CO 2 ) than do individual boilers and furnaces.
Energy efficiency is a cornerstone of any strategy for reducing GHG emissions, and green building programs in turn are a major strategy for reducing energy use in buildings [3]. Founded in 1998, the U.S. Green Building Council's (USGBC) Leadership in Energy and Environmental Design (LEED) program is the most popular green building certification program in the U.S. [4]. Many governmental and other types of organizations have mandated that all new buildings and major renovations achieve the LEED Silver that mandate building owners disclose their annual energy and water use to local city governments [25]. In many cases, these data are being made public. These data provide the opportunity to compare the energy use of LEED buildings with that of non-LEED buildings for the same geographic region, property type, time frame, and climate, largely avoiding the selection bias associated with voluntary data submission.
We previously used 2011 municipal energy benchmarking data to study the performance of 21 LEED office buildings in New York City (NYC) [23] and 2015 data to study the performance of 113 LEED offices, multifamily housings, and K-12 Schools in Chicago [24]. We found no evidence for source energy or GHG savings by LEED buildings in either study.
We obtained 2016 energy data from some 28,500 commercial properties (465 million m 2 ) in 10 major cities spanning the continental U.S. We identified 861 of these properties/buildings as LEED-certified in systems that address whole building energy use.
In this paper, we focus on the subset of buildings that correspond to the office property type. This subset includes 4417 properties, with floor area totaling 110 million m 2 . Of these, 551 were identified as LEED-certified, totaling 31 million m 2 . This constitutes the largest study of measured energy performance for LEED-certified buildings ever published.
For each office building in the benchmarking data, we extract five metrics associated with its annual energy consumption: site energy, source energy, electric energy, non-electric energy, and (energy-related) greenhouse gas emission. Each of these metrics, adjusted for building size, are then compared between LEED-certified offices and other offices in the same cities to understand measured savings associated with LEED-certification.
We also obtain, for most of the LEED buildings, the number of points awarded during certification for energy optimization and, from these, we determine the energy savings projected by their respective design teams. We then compare these projected savings with the actual measured savings for each individual LEED building. Finally, we investigate the role of building age and look for differences in energy savings associated with the level of LEED certification.

Benchmarking Data
Building benchmarking data for nine U.S. cities for 2016 were downloaded from their respective web sites. Data for the tenth city (Denver) were provided at our request on the condition that individual building identities were kept confidential. Benchmarking data are organized for these municipalities by the U.S. EPA's Energy Star Portfolio Manager [26]. In general, disclosed data included property address, primary property type, floor area (in ft 2 ), site energy use intensity or EUI (in kBtu/ft 2 ), source EUI (in kBtu/ft 2 ), total GHG (in metric tonne CO 2 ), year built, and Energy Star score. Three cities (Boston, Chicago, and Seattle) also provided detailed fuel information. Source EUI information was not included for Boston, but we were able to calculate it from information provided. Portland did not provide the year built. In some cases, data for multiple buildings were combined into a single property. In most cases, data were reported for buildings with floor area A ≥ 4650 m 2 , although some smaller buildings were included. Municipal buildings were included for some cities but not for others. Portland and Minneapolis did not include data for multifamily housing (MFH). Data for 28,478 properties remained after eliminating any records that were missing either floor area or site energy. These data included 90 different property types, with 4417 identified as office, financial office, or office with data center. We collectively refer to these three property types as office buildings, or offices as is also done by the EPA's Portfolio Manager in calculating their Energy Star scores. (Only the city of Portland classifies properties as "office with data center"-and there are only 11 such properties. We have no reason to believe that these properties are significantly different from properties classified as office or financial office in other cities. ) We determined that total GHG figures reported for Seattle properties were significantly lower than those calculated by Portfolio Manager, which uses regional emission factors determined by eGRID sub-regions [27]. For consistency, we replaced these GHG figures with ones calculated using the same methodology employed by Portfolio Manager for other cities.
Data were cleaned to remove properties with floor area 465 m 2 or less, resulting in elimination of 57 properties (including 7 offices). For many eliminated buildings, their floor area was so small as to suggest data entry errors, a hypothesis supported by accompanying site and source EUI that were orders of magnitude above normal. Properties that were eliminated either contained erroneous data or were so small as to have no impact on subsequent analysis. We also eliminated 34 properties (including 6 offices) for which the source-to-site energy ratio was greater than 3.25. It should be noted that the highest understandable source/site ratio is 3.14, which corresponds to an all-electric building. The largest ratio found was 13.7 for which we had no explanation. Nearly 2000 properties had ratios between 3.14 and 3.25, which we attributed to round-off errors. The cut-off of 3.25 was arbitrarily chosen so as to eliminate the worst offenders while keeping most buildings in the dataset. The third cleaning process eliminated 29 properties (including 5 offices) whose total GHG fields were blank. A final filter was applied on a city-by-city basis to eliminate offices with incredibly high or low site EUI. Approximately 5% of the properties were thus eliminated whose log(site_EUI) were more than two standard deviations above or below the mean log(site_EUI) for offices in the same city. The final cleaned and filtered office dataset contained 4168 office properties, totaling 110 million m 2 .

LEED Building Identification
LEED buildings were not identified in the benchmarking data. To identify LEEDcertified buildings we downloaded the LEED project database from the USGBC web site [6]. LEED project records were matched with benchmarking data records to identify any benchmarked buildings that were certified in LEED systems that address whole building energy use which, for these offices, was limited to New Construction (NC), Core & Shell (CS) and Existing Buildings (EB).This matching process was similar to that used in our previous Chicago study [24]. Here, however, we added a computerized prescreening process in which geocoding [28] was used to determine GPS coordinates for each benchmarked building and each potential LEED project record, and then potential matches were determined with a computer program based on proximity of GPS coordinates. The resulting list of potential matches was then manually checked as described in [24]. Any building with a LEED certification date before 1 July 2016 was considered certified for this study. This process identified 861 LEED-certified buildings in the benchmarking data, corresponding to 42 different property types. Of these buildings, 551 were identified as offices. For 534 of these LEED offices, we were able to determine the number of points awarded for energy efficiency (EAc1) from LEED scorecards posted on the Green Building Information Gateway [29]. In some cases, particularly for LEED buildings certified under v1 and v2, the posted EAc1 values were inconsistent with posted total energy and atmosphere (EA) values. In these cases, we determined consistent EAc1 values by working backwards from EA, combined with posted point values for other EA categories.

Site, Source, Electric, and Non-Electric Energy
Portfolio Manager calculates site energy use intensity (EUI) from 12 months of utility data (total energy purchased) combined with the building floor area. Source EUI are subsequently calculated by combining utility (i.e., fuel) data with fuel-dependent, national average site-to-source energy conversion factors [30]. In 2016, the EPA values for the site-tosource factors for electricity and natural gas were 3.14 and 1.05, respectively. Most buildings use only these two sources, so for such buildings, the site EUI and source EUI may be combined to calculate electric EUI and non-electric (i.e., natural gas) EUI. We used this method to calculate electric EUI and non-electric EUI for all buildings, including those that use other fuels, such as district hot or cold water. When tested on Chicago buildings, we found that this method produced no more than a 3% error in total electric or non-electric energy (see Supplemental Materials).

Comparing Energy Metrics between Building Sets
In this study we consider five metrics related to building energy use: site EUI, source EUI, electric EUI, non-electric EUI, and GHG intensity. Many previous building studies compare either the mean or median EUI between two building sets. While such metrics are readily analyzed using accessible statistical routines, neither is related to the total energy consumed (or total GHG emission) by the set of buildings. As has been argued previously, the appropriate metric is the gross EUI, equal to the total energy divided by total floor area for N buildings [12,15]. This is mathematically equivalent to the area-weighted mean EUI for the building set [14] and is the metric used by the U.S. Energy Information Administration (EIA) in reporting energy use for subsets of U.S. commercial buildings as determined by its CBECS [9]. It is also the method utilized in the municipal benchmarking data when EUI values are reported for properties containing more than one building.
The energy performance of the set of LEED buildings is judged by comparing their area-weighted mean EUI with that for a set of similar non-LEED buildings. In our earlier studies, we modified a 2-sample t-test for weighted averaging and used it to determine the statistical significance of differences in area-weighted means [12,23,24]. While the modification is plausible, it lacks a supporting mathematical framework. Here, we use the more modern method of permutation testing, using resampling to generate a permutation distribution for the difference statistic with its corresponding (2-sided) p-value [31]. The details of each of these methods are provided in the Supplementary Materials.
These comparisons between LEED and non-LEED are readily made on a city-by-city basis. However, when used to analyze aggregate data spanning multiple cities, this approach can have an unwanted result of comparing LEED buildings dominated by one city with non-LEED buildings dominated by another. Consider the outcome, for instance, if Los Angeles dominated the area-weighted non-LEED average while NYC dominated the LEED average. Buildings in NYC typically use more energy than those in LA. Even if LEED buildings out-perform non-LEED ones in both cities, the aggregate result would show that non-LEED out-perform LEED buildings. To avoid this outcome, we define, for each of the j = 1 to N buildings, δe j to be the difference between the EUI (e j = E j /A j ) of the building and the area-weighted mean EUI for similar non-LEED buildings in the same city (e j and δe j represent any of the five intensity metrics). For each city, the area-weighted mean delta is equal to the difference in the area-weighted mean EUI for LEED and non-LEED buildings. The aggregate area-weighted mean, however, avoids the abovementioned problem and, when multiplied by total LEED floor area, yields the total energy or GHG emissions saved by these LEED buildings. In the rest of this paper, it is understood that all mean intensity metrics are calculated using area weighting.

Imputing Design Energy Savings from EAc1 Scores
Buildings achieve LEED-certification at four different levels-Certified, Silver, Gold and Platinum-depending on the total number of points awarded in dozens of categories chosen to minimize the building's environmental footprint and improve occupant health and satisfaction [4]. The first LEED system introduced was New Construction (NC). A second LEED system, Core & Shell (CS) was introduced that focuses on building envelope and core building systems, targeting newly-constructed buildings before their tenants are known. A third system, Existing Buildings, Operation and Maintenance (EBOM), was introduced to make LEED certification accessible to existing buildings-the vast majority of the commercial building stock. Since 2005, LEED has undergone several revisions, and additional certification systems have been added. All LEED-certified offices in this study were certified under some version of NC, CS, or EB, with the vast majority corresponding to the latter.
The largest single point category is for energy optimization, recorded on the LEED scorecard as the parameter EAc1 in the energy and atmosphere (EA) category. The number of possible energy optimization points and criteria for awarding them vary with LEED system and version. We were able to obtain EAc1 parameters for 807 of the 861 LEED buildings from the Green Building Information Gateway web site [29]. We found that 324 of these (including 75 offices) were for buildings certified under NC, CS, or other LEED systems, for which EAc1 points are directly related to a site energy savings ratio, r site , projected by their respective design teams, and is given here: where e design and e base are the design team's simulated annual site EUI for the adopted LEED building design and a code-compliant (and presumably less energy-efficient) "baseline" design. The imputed projected relative site energy savings for these buildings ranged from 0 to 48%. We did not have access to projected base EUI for each project, but a reasonable estimate of the baseline EUI is the gross site EUI for non-LEED buildings of the same type (office) in the same city. This assumption along with r site values imputed from EAc1 scores and Equation (1) allowed us to calculate the projected site EUI savings (e design − e base ) for each of these LEED buildings. The remaining 483 LEED buildings for which we were able to obtain EAc1 parameters (including 459 offices) were certified under an EB LEED system. For offices, EAc1 points are awarded based on the Energy Star score reported for the building at the time of certification. (For other space types that are not eligible for an Energy Star score, the methodology is more complicated.) EAc1 parameters for these buildings were translated into "design" Energy Star scores, i.e., Energy Star scores submitted as part of LEED certification. An Energy Star score, under reasonable assumptions, was shown to be related to a source energy savings ratio, where e is the building source EUI andê is the median source EUI for the national distribution of this building type, adjusted for various EPA-identified operating parameters [32]. We did not have access to the operating parameters that would allow us to calculateê for each building; instead, we assumed thatê is equal to the gross source EUI for non-LEED buildings of the same type (office) in the same city. With these assumptions, we used Equation (2) to calculate projected source EUI savings for each of these 459 LEED offices. We elaborate on this methodology in the Supplemental Materials. Figure 1 displays the mean site EUI for the 10 cities and in aggregate, broken out by electric (blue) and non-electric (brown) energy. The scale on the left axis is in SI units (MJ/m 2 ) while that on the right axis is in kBtu/ft 2 units used in the U.S. The error bars represent the bootstrap standard deviations of the (weighted) means. It should be noted that all standard errors reported in this paper correspond to one sigma. The aggregate mean site EUI of (850 ± 10) MJ/m 2 is in agreement with the gross office site EUI, as found in the 2012 CBECS. In aggregate, electric contributes 72% to site EUI-also consistent with 2012 CBECS results. Offices in NYC have higher site EUI and use relatively less electric energy. Offices in West Coast cities (Seattle and LA) have the lowest site EUI and use relatively more electric energy. These results are not surprising given the various climates and regional electric mix. It should also be noted that there are differences in data shared by the various cities. In particular, federal buildings of significant importance to Washington, DC, are not covered by these data,. Moreover, disclosures for some cities include municipal buildings while others do not. Finally, compliance rates vary between cities. use relatively more electric energy. These results are not surprising given the various climates and regional electric mix. It should also be noted that there are differences in data shared by the various cities. In particular, federal buildings of significant importance to Washington, DC, are not covered by these data,. Moreover, disclosures for some cities include municipal buildings while others do not. Finally, compliance rates vary between cities.

Comparing LEED with Non-LEED Offices
The basic statistics for non-LEED offices in the 10 cities and in aggregate are summarized in Table 1. Similarly, statistics for LEED offices are summarized in Table 2. Both tables list the numbers of buildings, their total floor area (in millions of square meters), and their (area-weighted) mean site EUI, electric EUI, non-electric EUI, source EUI, and green-house gas intensity (GHGI). The four energy metrics are expressed in MJ/m 2 and GHGI in kg/m 2 (CO2 equivalent). For all five metrics, standard deviations of the (weighted) means (SDm) were calculated using bootstrap with 10,000 replicates.

Comparing LEED with Non-LEED Offices
The basic statistics for non-LEED offices in the 10 cities and in aggregate are summarized in Table 1. Similarly, statistics for LEED offices are summarized in Table 2. Both tables list the numbers of buildings, their total floor area (in millions of square meters), and their (area-weighted) mean site EUI, electric EUI, non-electric EUI, source EUI, and green-house gas intensity (GHGI). The four energy metrics are expressed in MJ/m 2 and GHGI in kg/m 2 (CO 2 equivalent). For all five metrics, standard deviations of the (weighted) means (SDm) were calculated using bootstrap with 10,000 replicates. Table 1. Area-weighted mean intensity metrics for non-LEED offices. SDm is the standard deviation of the weighted mean calculated using the bootstrap method with 10,000 replicates. Four energy intensities are in MJ/m 2 and GHG intensity in kg/m 2 .  Figure 2 compares mean site EUI for LEED offices (green) with those of non-LEED offices (red) by city and in aggregate. Figure 2 clearly shows that, adjusting for floor area, LEED offices, on average, have lower site EUI than non-LEED offices in each city and in aggregate. The error bars in Figure 2 suggest that the differences between LEED and non-LEED Site EUI are resolvable (i.e., statistically-significant).  The LEED site EUI savings (the differences between LEED and non-LEED in Figure 2) are graphed in Figure 3. For individual cities, these results are identical to those displayed in Figure 2, while in aggregate they are slightly different (see the earlier discussion in Section 2.4). Here, the site EUI savings may be readily compared with the standard error in the delta (the error bars in Figure 3). In each case, the t-value is the ratio of the height of the y-value divided by the standard error (half-length of error bar). For Boston, for instance, the mean site EUI savings is 130 MJ/m 2 and the std. error in this mean delta is 80 MJ/m 2 . The resulting ratio or t-value is 1.6. The two-sided p-value obtained for this delta using permutation testing is 0.1163 (see Supplemental Materials). In aggregate, LEED offices save 95 MJ/m 2 , which represents an 11% savings from the mean site EUI for non-LEED offices in aggregate. We have high confidence in this result; the p-value associated with this difference is p < 10 −3 . The LEED site EUI savings (the differences between LEED and non-LEED in Figure 2) are graphed in Figure 3. For individual cities, these results are identical to those displayed in Figure 2, while in aggregate they are slightly different (see the earlier discussion in Section 2.4). Here, the site EUI savings may be readily compared with the standard error in the delta (the error bars in Figure 3). In each case, the t-value is the ratio of the height of the y-value divided by the standard error (half-length of error bar). For Boston, for instance, the mean site EUI savings is 130 MJ/m 2 and the std. error in this mean delta is 80 MJ/m 2 . The resulting ratio or t-value is 1.6. The two-sided p-value obtained for this delta using permutation testing is 0.1163 (see Supplemental Materials). In aggregate, LEED offices save 95 MJ/m 2 , which represents an 11% savings from the mean site EUI for non-LEED offices in aggregate. We have high confidence in this result; the p-value associated with this difference is p < 10 −3 .

Non-LEED
height of the y-value divided by the standard error (half-length of error bar). For Boston, for instance, the mean site EUI savings is 130 MJ/m 2 and the std. error in this mean delta is 80 MJ/m 2 . The resulting ratio or t-value is 1.6. The two-sided p-value obtained for this delta using permutation testing is 0.1163 (see Supplemental Materials). In aggregate, LEED offices save 95 MJ/m 2 , which represents an 11% savings from the mean site EUI for non-LEED offices in aggregate. We have high confidence in this result; the p-value associated with this difference is p < 10 −3 .  Table 3 provides a yet more detailed way to view these results, along with those for the other four intensity metrics. LEED savings represent the area-weighted mean δe for LEED buildings divided by the area-weighted mean metric for non-LEED offices in the same city. Here, p-values are calculated for the differences using permutation testing with 10,000 replicates. As was found in Figure 3, Boston LEED offices demonstrate a 14% savings in site EUI (as compared with non-LEED offices in Boston) but the p-value for this difference is 0.117. In aggregate, however, the LEED savings in site energy is 11% with a p-value less than 0.001. At the individual city level, LEED offices in Chicago, Denver, Minneapolis, and Washington demonstrate site energy savings near or above the 95% confidence level (p ≤ 0.05). LEED offices in other cities demonstrate site energy savings but with less statistical confidence. Table 3. LEED savings in the five intensity metrics relative to the area weighted means for non-LEED offices in the same cities. The p-values are 2-sided, calculated using permutation testing with 10,000 replicates.  Table 3 for individual cities are derivable from the LEED and non-LEED city means listed in Tables 1 and 2. In aggregate, this is not the case. The discrepancy is especially evident for GHGI. Tables 1 and 2 show that, in aggregate, LEED offices have a higher mean GHGI than non-LEED offices do. Yet, in each individual city, LEED offices have lower mean GHGI than non-LEED offices do. Table 3 shows that LEED offices save source energy and GHG emission relative to non-LEED offices in each city and in aggregate. Except for Washington (and possibly Denver), the uncertainties in these savings at the city level are so large that the savings are not statistically significant. In aggregate, however, LEED offices save 7% in both source EUI and GHG intensity, and these savings are statistically significant. Table 3 also shows that, in aggregate, LEED offices save 26% in non-electric EUI but very little (4%) in electric EUI; moreover, this value is not statistically significant at the 95% confidence level (p = 0.113).

Savings listed in
We also compared non-LEED offices to LEED ones by level of certification. These results are summarized in Table 4. This table is similar in format to Tables 1-3. Table 4 shows that LEED Silver offices demonstrate no statistically significant savings relative to non-LEED offices. LEED Gold offices demonstrate statistically significant savings in most metrics: 12% in site EUI, 9% in source EUI, 27% in non-electric EUI, and 8% in GHG intensity. Platinum offices demonstrate the greatest savings in site and non-electric EUI, but they do not out-perform Gold offices in key metrics of source EUI or GHG intensity savings. Only Gold offices demonstrate evidence for modest (6%) savings in electric EUI.

Impact of Building Age
It is commonly believed that newer buildings use less energy than do older buildings, owing to their more efficient technology and better construction practices. Some studies, however, have suggested the opposite [24,33]. In a previous study of Chicago buildings, we found that LEED buildings were generally newer than other buildings. Energy performance of Chicago LEED buildings fared better when compared to other newer buildings because the newer buildings tended to have higher source EUI than older buildings [24].
To test this idea, we performed linear regressions of the five intensity metrics on the variable YearBuilt for all non-LEED offices. Each regression is of the form with e being the intensity metric (site EUI, etc.). Portland buildings were not included in these analyses as Portland did not disclose the year built in their data. Year built was not provided for a handful of other buildings in the dataset; any such building was eliminated before analysis. A few other buildings for which YearBuilt < 1800 were also eliminated as outliers. The results of these regressions for the 3483 remaining non-LEED offices are listed in Table 5. The low p-values for the first three metrics indicate that year built is a significant predictor for these metrics. The high p-values for source EUI and GHG intensity suggest the opposite for these two metrics. The relatively low R 2 (≤6.1%) indicates that only a small portion of these metrics is "explained" by this variable. The trends that emerge here agree with our observations. Newer buildings are, on average, achieving lower site EUI, in part by fuel switching, i.e., using less non-electric energy but relatively more electric energy. These changes, however, do not necessarily lead to lower average source EUI or GHG emission. While these regressions were not weighted, area-weighted regressions produce similar results, as did regressions on all offices (LEED and non-LEED). Given that year built is a significant predictor of three of the five energy metrics, it is of interest to compare the year built for LEED offices with non-LEED offices in our data. We calculated the mean year built for both LEED and non-LEED offices and found that in each of the nine cities (omitting Portland), LEED offices are, on average, newer than non-LEED offices. In aggregate, we find the mean year built for LEED and non-LEED offices to be 1978 and 1952, respectively. Hence, the comparison of LEED offices with all non-LEED offices (Table 3) is, in part, a comparison of newer (LEED) offices with older (i.e., non-LEED) offices. It should be noted that while LEED offices tend to be newer than non-LEED offices, the differences are not as large as one might imagine. The vast majority of LEED offices were certified using one of the existing buildings (EB) systems. In some cases, renovations of very old buildings qualify for a LEED-NC classification. Here, the earliest year built for an office certified under NC is 1898. In total, 12 offices certified under NC or CS have YearBuilt < 1950.
In order to compare LEED offices with non-LEED offices of similar vintage, we took a city-by-city approach and began eliminating the oldest non-LEED offices until the mean year built for non-LEED offices matched that for LEED offices in the same city. We then used our earlier methodology to calculate LEED savings in each of the five intensity metrics relative to these newer, non-LEED offices in each city and in aggregate. These results are summarized in Table 6, which is analogous to Table 3, except that now the base set for each city is a newer set of non-LEED offices. Additional statistics for newer offices, similar to Tables 1 and 2, are provided in the Supplemental Materials. Table 6. LEED savings in the five intensity metrics relative to the area weighted means for newer, non-LEED offices in the same cities. N and A are for the subset of newer non-LEED offices whose mean year built matches that for LEED in the same city. The p-values are 2-sided, calculated using permutation testing with 10,000 replicates. Comparing Tables 3 and 6, we see that, except for non-electric energy, LEED offices demonstrate even more savings when compared with newer, non-LEED offices in the same cities. In particular, when compared with newer non-LEED offices, LEED offices demonstrate 12% savings in site energy, 8% savings in electric energy, 10% in source energy, and 10% in greenhouse gas emission. Savings in individual cities vary as do the statistical significance of these savings. However, in aggregate, all results are statistically significant. For non-electric energy, Table 6 demonstrates lower LEED savings (21%), reflecting the trend that newer offices, whether LEED or not, tend to use lower non-electric energy than do older offices.

Comparing Measured with Predicted LEED Savings
Roughly a quarter of the 861 LEED buildings identified in the benchmarking data were certified under a LEED system such as NC or CS, for which points for energy optimization were awarded on the basis of projected site energy savings. We were able to obtain the EAc1 parameters for 75 such offices from their LEED scorecards posted on the Green Information Building Gateway [29]. Using the method described in Section 2.5, these were used to impute projected site EUI savings for each of these offices. The projected site EUI savings for 72 of these were then compared with the measured savings based on mean EUI for non-LEED offices in each city (summarized in Table 1). Figure 4 is a graph of the measured site EUI savings (δSiteEUI j ) versus the projected site EUI savings for each of these LEED buildings. If the measured savings agree with the projected savings, the graph is expected to be a straight line of slope unity, shown as the dashed green line in the figure. Instead, Figure 4 demonstrates negligible correlation between the projected and measured site EUI savings. A linear least squares fit to the data in Figure 4 yields a line with slope −0.16 with an R 2 of 0.0025. The p-value is 0.67, confirming that design projections are not a significant predictor of measured savings. The total measured site energy savings for these 74 LEED offices is 11% below their total predicted site energy savings.

Comparing Measured with Predicted LEED Savings
Roughly a quarter of the 861 LEED buildings identified in the benchmarking data were certified under a LEED system such as NC or CS, for which points for energy optimization were awarded on the basis of projected site energy savings. We were able to obtain the EAc1 parameters for 75 such offices from their LEED scorecards posted on the Green Information Building Gateway [29]. Using the method described in Section 2.5, these were used to impute projected site EUI savings for each of these offices. The projected site EUI savings for 72 of these were then compared with the measured savings based on mean EUI for non-LEED offices in each city (summarized in Table 1). Figure 4 is a graph of the measured site EUI savings (δSiteEUIj) versus the projected site EUI savings for each of these LEED buildings. If the measured savings agree with the projected savings, the graph is expected to be a straight line of slope unity, shown as the dashed green line in the figure. Instead, Figure 4 demonstrates negligible correlation between the projected and measured site EUI savings. A linear least squares fit to the data in Figure 4 yields a line with slope −0.16 with an R 2 of 0.0025. The p-value is 0.67, confirming that design projections are not a significant predictor of measured savings. The total measured site energy savings for these 74 LEED offices is 11% below their total predicted site energy savings. The vast majority of the LEED offices in our study were certified under some version of existing buildings (EB). For these cases, the EAc1 points are derived from building Energy Star scores at the time of certification. From these, we impute source EUI savings as described in Section 2.5. Figure 5 is a graph of the measured 2016 source EUI savings vs. the imputed design source EUI savings for these 454 offices, for which EAc1 parameters were obtained. The symbols indicate the LEED version corresponding to the certification. The majority of these offices were certified under v2009 (red triangles). If the measured savings agreed with the predicted savings, we would expect all the points to fall on a line through the origin with slope unity, indicated by the dashed green line in the graph. Instead, we see extensive scatter with little correlation (R 2 = 8%). The blue line in the graph is the actual trend line for the data. For a vast majority of the buildings, the The vast majority of the LEED offices in our study were certified under some version of existing buildings (EB). For these cases, the EAc1 points are derived from building Energy Star scores at the time of certification. From these, we impute source EUI savings as described in Section 2.5. Figure 5 is a graph of the measured 2016 source EUI savings vs. the imputed design source EUI savings for these 454 offices, for which EAc1 parameters were obtained. The symbols indicate the LEED version corresponding to the certification. The majority of these offices were certified under v2009 (red triangles). If the measured savings agreed with the predicted savings, we would expect all the points to fall on a line through the origin with slope unity, indicated by the dashed green line in the graph. Instead, we see extensive scatter with little correlation (R 2 = 8%). The blue line in the graph is the actual trend line for the data. For a vast majority of the buildings, the measured savings fall well below the expected savings. The total measured source energy savings for these offices is just 19% of the total predicted source energy savings.
Energies 2021, 14, x FOR PEER REVIEW 14 of 20 measured savings fall well below the expected savings. The total measured source energy savings for these offices is just 19% of the total predicted source energy savings.

Discussion
We previously reported the LEED office performance for NYC (2011 data) [23] and Chicago (2015 data) [24]. The 2016 results reported here for Chicago are consistent with those reported for 2015 for both LEED and non-LEED offices. In Chicago, LEED offices demonstrate 10% savings in site energy (12% in 2015) but no significant savings in source energy or GHG emission (see Table 3).
NYC offices have lower energy use in 2016 than in 2011, and LEED offices are performing slightly better, relative to non-LEED offices. The reduction for NYC offices from 2011 to 2016 is 11% in site EUI and 16% in source EUI. It is likely that this is partly attributed to NYC's aggressive programs to benchmark and reduce building energy use. Other factors such as weather and economic activity may also contribute to these differences. In 2011, NYC LEED offices demonstrated neither site nor source energy savings relative to non-LEED offices. For 2016, LEED offices demonstrate 8% savings in site energy but no source energy savings (see Table 3). This may be related to the evolution of LEED standards, as the vast majority of NYC offices here were certified under LEED version 2009, whereas in 2011 they were mostly LEED v1 and v2.
LEED office energy savings in the eight other cities surpass those found for Chicago and NYC, on average, demonstrating positive savings in both source energy and GHG emissions. In aggregate, savings in both of these metrics is 7%. While well below the projected 25-30% energy savings asserted for LEED buildings [2,10] these aggregate savings are nonetheless positive and statistically significant.
To better understand the gap between predicted and measured energy savings for LEED buildings, we looked for a correlation between the two on a building-by-building basis. Our expectation was that LEED offices that earned the most points for energy optimization would be those that actually saved the most energy, either site or source depending on the method for awarding these points. This was not the case. Instead, we found little or no connection between predicted and measured energy savings (see Figures 4 and 5). It should be noted that the absence of correlation in Figure 4 does not

Discussion
We previously reported the LEED office performance for NYC (2011 data) [23] and Chicago (2015 data) [24]. The 2016 results reported here for Chicago are consistent with those reported for 2015 for both LEED and non-LEED offices. In Chicago, LEED offices demonstrate 10% savings in site energy (12% in 2015) but no significant savings in source energy or GHG emission (see Table 3).
NYC offices have lower energy use in 2016 than in 2011, and LEED offices are performing slightly better, relative to non-LEED offices. The reduction for NYC offices from 2011 to 2016 is 11% in site EUI and 16% in source EUI. It is likely that this is partly attributed to NYC's aggressive programs to benchmark and reduce building energy use. Other factors such as weather and economic activity may also contribute to these differences. In 2011, NYC LEED offices demonstrated neither site nor source energy savings relative to non-LEED offices. For 2016, LEED offices demonstrate 8% savings in site energy but no source energy savings (see Table 3). This may be related to the evolution of LEED standards, as the vast majority of NYC offices here were certified under LEED version 2009, whereas in 2011 they were mostly LEED v1 and v2.
LEED office energy savings in the eight other cities surpass those found for Chicago and NYC, on average, demonstrating positive savings in both source energy and GHG emissions. In aggregate, savings in both of these metrics is 7%. While well below the projected 25-30% energy savings asserted for LEED buildings [2,10] these aggregate savings are nonetheless positive and statistically significant.
To better understand the gap between predicted and measured energy savings for LEED buildings, we looked for a correlation between the two on a building-by-building basis. Our expectation was that LEED offices that earned the most points for energy optimization would be those that actually saved the most energy, either site or source depending on the method for awarding these points. This was not the case. Instead, we found little or no connection between predicted and measured energy savings (see Figures 4 and 5). It should be noted that the absence of correlation in Figure 4 does not depend on whether the site EUI savings are calculated relative to all non-LEED offices or only newer, non-LEED offices. Low correlation between predicted and measured energy savings was first reported by NBI in 2008. Their study focused on LEED-NC v2 [10]. At that time, there were roughly 600 U.S. LEED-certified buildings, with total floor area of 7.2 million m 2 . A decade later with two LEED versions farther along and 40-50 times the number of certified buildings and floor area, this performance gap remains.
For LEED EB systems, the problem is with the Energy Star score, which is the basis for awarding points for energy optimization. One of us has extensively studied the science behind these scores and found it to be highly problematic [32,34]. Energy Star scores are half based on measured energy use and half based on adjustments associated with user-supplied operational parameters. The first part is reproducible, but the second part is not, is highly flawed, and is easily gamed by simply adjusting the reported building operating parameters.
It should be noted that our imputed site EUI and source EUI savings based on LEED points awarded for energy optimization (EAc1), while based on reasonable assumptions, are not calculated using methodology identical to that employed by design teams when these points were awarded. We simply do not have access to the predicted baseline site EUI used by the design teams for the NC or CS LEED systems, nor do we have access to the operating parameters used in calculating the Energy Star scores behind EAc1 points for offices certified under EB systems. It is possible that someone with access to this confidential information could use it to demonstrate there is some sense in which these buildings were expected to yield the projected energy savings, but we find there to be little utility in this if it does not result in a significantly lower measured energy use and GHG emission-namely of the kind that is necessary to address climate change and that so many municipalities have pledged to achieve.
Numbers provided in Table 3 can be readily combined to estimate total savings by LEED offices. For instance, the aggregate reduction in annual GHG emissions for LEED offices is (7%) × (77 kg/m 2 ) = 5.4 kg/m 2 . This savings multiplied by LEED office aggregate floor area yields a total savings for 2016 of 170,000 metric tonne of CO 2 , as compared with non-LEED offices of the same floor area in the same cities.
MacNaughton et al. estimated the CO 2 emissions savings for all LEED-certified buildings in the U.S. and five other countries in 2000-2016 to be 33 Mtonne CO 2 [2]. Their calculation is based on the assumptions that each LEED building annually achieves the energy savings projected by its design team and that these savings occurred uniformly in all fuels. We criticized these assumptions as being inconsistent with the measured energy savings for LEED buildings, and based on our 2015 Chicago data, we offered that these savings are zero or even negative [7]. Here, our data show that LEED office savings, in aggregate, are greater than those reported for Chicago but significantly lower than projected by design teams. The estimated 33 Mtonne savings above is equivalent to an annual GHGI savings of 22 kg/m 2 /year for LEED buildings. This figure is nearly four times the 5.4 kg/m 2 /year savings that we report here for U.S. LEED offices in 2016. Tables 3 and 4 show that LEED offices are achieving much higher savings in nonelectric energy, i.e., 26% for all LEED, 16% for Silver, 27% for Gold, and 42% for Platinum. It is interesting that these measured savings in non-electric energy roughly track design projections for total energy savings [2] (Figure 2). Non-electric energy is responsible for 30% of the energy-related GHG emissions for commercial buildings. This, combined with our observed 26% reduction in aggregate non-electric energy for LEED offices (see Table 3), implies an 8% reduction in GHG emission, close to the 7% reduction observed for all LEED offices.
It can be argued that it is unfair to compare LEED offices with all other offices in this study. This view is supported by several facts. As noted, LEED offices are, on average, 26 years newer than other offices. It is also the case (see Tables 1 and 2) that LEED offices are, on average, 2-3 times larger than non-LEED offices. (When the total floor area is divided by the number of offices, it can be seen that the average LEED office in this study is 56,000 m 2 while the average non-LEED office is 22,000 m 2 .) Others have shown that LEED (and other green-labeled) buildings typically represent more high-end, more desirable commercial space [22]. These observations suggest that it would be fairer to compare LEED offices with only larger, newer, high-end non-LEED offices. It should be noted, however, that the EPA's Energy Star office model regression finds that SourceEUI increases with building size, but only up to a floor area of 9300 m 2 . For larger buildings, they make no additional adjustment [35].
While there is merit in the above argument, we find the justification for comparing LEED with all non-LEED to be more compelling. Six of the ten cities in this study (Boston, Minneapolis, NYC, Portland, Seattle, and Washington) are signatories of the Carbon Neutral City Alliance and have pledged to reduce GHG emissions by 80-100% by 2050 [36]. Chicago has adopted a more aggressive climate action plan in aiming to become 100% carbon neutral by 2040 [37]. These commitments are to reduce GHG relative to existing levels (i.e., all existing buildings). The comparison of LEED building energy and GHG savings relative to other newer buildings is not relevant to these goals. Fair or not, a successful strategy for addressing climate change will require buildings whose GHG emissions are greatly reduced over those of existing buildings.
The movement to switch from natural gas (and other fossil fuels) to electric energy for buildings is not unique to LEED. This trend is found in our regression on the year built, as seen previously in our 2015 Chicago study [24], and is evident in CBECS [9]. Indeed, there are those who advocate the immediate electrification of buildings as the best route to carbon neutrality [38]. The reasoning is clear: Heating a building with natural gas (or another fossil fuel) locks in the associated GHG emissions for the lifetime of the heating system. Choosing to heat with electricity (typically utilizing heat pumps) instead provides the building a pathway to zero emissions, benefitting entirely from ongoing efforts to make the electric grid 100% carbon-free. However, in the short term, building electrification can and often does result in little or no savings in GHG emission. Moreover, depending on the carbon content of the regional electric grid, building electrification can result in increased (indirect) GHG emission as compared with the alternate employment of efficient natural gas heating. This, for instance, would be the outcome of using resistive electric instead of natural gas heat in most parts of the U.S. Depending on the pace with which the electric power sector lowers its carbon, it is possible for well-intentioned building electrification to result in greater GHG emission over the lifetime of the HVAC system. Some project a rapid transition to a carbon-free grid [39]. Indeed, advocates of municipal climate action commitments must implicitly believe a rapid transition to a carbon-free electric grid as the goals of 80-100% carbon reduction by 2040 or 2050 are not possible otherwise. Aspirational legislation, however, does not guarantee scientific outcome.
In 2018, the U.S. electric power sector derived 63% of its primary energy from fossil fuels and was responsible for 70% of the energy-related GHG emissions of U.S. commercial buildings [40]. The U.S. power sector's carbon footprint has decreased by 25% over the last 15 years, owing largely to the replacement of old, inefficient coal plants with new, more efficient natural gas plants. Expanded renewables have also lowered GHG emission but have played a minority role. While this replacement of coal with gas is expected to continue, the future benefit will be reduced as natural gas plants also replace retired, carbon-free nuclear plants. Expansion of renewables continues, but absent major advances in storage technology is unlikely to result in a rapid decline of fossil fuel use. In its 2019 report on energy outlook, the Energy Information Administration projects that in 2050 fossil fuels will provide 56% of U.S. grid electricity, despite a projected 230% increase in renewable electricity [41].
Given the uncertainties in predicting carbon content of future electricity, a sound strategy for new construction and major renovation would be to adopt an HVAC system expected to minimize the integrated GHG emission (both direct and indirect) over the lifetime of this system. This may or may not be an all-electric system, depending on many details including the current and foreseeable fuel mix for the regional electric grid.
Some of the limitations of this study require further comment. The benchmarking data used in this study are assembled using the EPA's Portfolio Manager. In most cases, energy data are submitted directly by utilities. Building parameters, such as floor area, space type, operating parameters, etc., are submitted by the building owner or their representative. Portfolio Manager incorporates various cross-checking means that prompt the user, should the entered data stand out as anomalous. In New York City, a relatively small number of consultants manage the data entry for a large fraction of the properties, reducing the sources of error. In the case of LEED buildings, we confirmed that floor areas in the benchmarking data are consistent with those recorded in the LEED project database. For Chicago and NYC, we have results from earlier studies, which provide a measure of consistency. Nevertheless, the validity of the data remains an untested assumption.
On average, source energy is a better measure of a building's energy footprint than site energy is. However, source energy does not account for local or even regional variation in grid efficiency or renewable grid generation. This is particularly concerning for building owners in the Pacific Northwest where hydroelectricity dominates the grid. The disparity, however, is not as large as one might guess, owing to regional electric connectivity and the role of natural gas peaking plants. When the electric load decreases even in a grid dominated by renewable energy, the result is reduced fossil fuel use at an inefficient peaking plant somewhere in the regional grid (see chapter 18 of Ref. [32]). Still, the usefulness of building source energy is decreasing, with the expanded use of renewable energy in the U.S. electric grid. To remain useful, the definition of source energy must evolve to account for regional differences in the electric grid.
Greenhouse gas figures calculated by the portfolio manager (and used in this study), unlike source energy, do take into account regional variation in the electric grid. They do not, however, reflect local differences within an eGRID region [27]. Our decision to change the GHG figures for Seattle buildings resulted in significant increases in GHGI for all Seattle buildings but impacted both LEED and non-LEED buildings equally; this had little impact on our conclusion regarding GHGI savings for LEED buildings in Seattle (Table 3 or Table 6). In addition, since Seattle contains only 7% of the LEED floor area in this study, this had minimal impact on the aggregate GHGI savings for LEED.

Summary and Conclusions
Utilizing 2016 municipal building energy benchmarking data, we compared the energy performance of about 550 LEED-certified office buildings (31 million m 2 ) in 10 U.S. cities with the performance of about 3600 other office buildings (79 million m 2 ) in these same cities for the same time period. This is the largest such study to date with respect to LEED building numbers or floor area. While LEED buildings continue to demonstrate wide variability in energy performance, we nonetheless found that LEED-certified U.S. office buildings, on average, are achieving statistically significant source energy savings and reductions in greenhouse gas emissions. In aggregate, we found that these office buildings save 11% in site energy and 7% in both source energy and GHG emissions, as compared with non-LEED office buildings in their same cities. On average, U.S. LEED office buildings are achieving 26% savings in non-electric fuels (mostly natural gas) while demonstrating no significant savings in electric energy. The total energy savings and reduction in GHG emissions are considerably lower than the 25-30% savings projected for LEED. Broken down by the level of certification, we found that LEED Silver offices achieve no significant savings while Platinum offices show no more savings than do Gold. Only Gold offices demonstrate statistically significant savings.
On average, LEED offices were 26 years newer than non-LEED offices. When compared with newer non-LEED offices, in aggregate LEED the savings were 12%, 10%, 8%, and 21% in site, source, electric, and non-electric energy, respectively, and 10% in GHG emission.
For individual buildings, we compared the projected energy savings (derived from LEED points awarded for energy optimization) with the measured energy savings, and we found little or no correlation between them. The data suggest that energy savings for LEED offices have little connection with LEED-certification points awarded for energy optimization. The total source energy savings for office buildings certified under EB (the vast majority of LEED offices) is less than one fifth the total savings implied by their LEED points earned for energy optimization. Unless corrected, this reward system is unlikely to produce buildings with significant energy savings.
Finally, we showed that LEED offices rely more heavily on electric energy than do other offices. On the one hand, this positions LEED buildings to achieve maximum benefit from future reduction in the carbon content of the U.S. electric grid. On the other hand, as the U.S. electric grid is presently heavily dependent on fossil fuels, LEED offices demonstrate little GHG savings today relative to other offices. Only time will tell whether future savings in GHG emission will justify the choice to rely heavily on electric energy.  Data Availability Statement: Data analyzed for this paper were obtained from a variety of public and private sources. Non-confidential data will be made available upon request.