Evaluating the Sustainability of New Construction Projects over Time by Examining the Evolution of the LEED Rating System

Abstract

In 1998, the U.S. Green Building Council (USGBC) developed the Leadership in Energy and Environmental Design (LEED) program to provide a standard form of evaluation for sustainability in building design and construction. Since its inception, LEED has undergone seven significant revisions, wherein the expectations needed to achieve the desired certification level were clarified and updated. The reasons for these updates are varied and include the recognition of new technologies and materials, the application of more stringent energy standards, and the recognition of differences in building types. However, the perception within the architecture, engineering, and construction (AEC) industry is that higher certification levels are more challenging to obtain in each subsequent version of LEED. For example, projects receiving a Gold certification under LEED 2.1 may only qualify for a Silver certification under LEED 3.0. The goals of this paper are to review changes in LEED over time and to empirically test this perception. Direct comparisons of the text of the credit requirements were performed between LEED versions (v) 2.1, v2.2, and v3.0 on a credit-by-credit basis. The comparison revealed ten different categories of changes between versions. From this comparison, conversion matrices were developed to allow a project scorecard from an older version of LEED to be converted to a newer version. To address uncertainty resulting from changing the level of detail in the information submitted on project scorecards, both strict and interpretative versions of these matrices were developed. These matrices were then applied to a sample of LEED-certified building projects, drawn using a stratified random sampling procedure from the publicly available USGBC database. The strata were separated first by LEED version (e.g., v2.1, v2.2, and v3.0) and then by certification level (e.g., Certified, Silver, Gold, Platinum). After converting the project scorecards from this sample, qualitative and correlational analyses were performed to test the hypothesis that LEED scores increase over time. The results show that in both strict and interpretive transformations, LEED scores show a slight to moderate increase in points over time.

1. Introduction

Since the U.S. Green Building Council’s (USGBC) Leadership in Energy and Environmental Design (LEED^®) pilot was introduced in 1998, the market for green buildings has grown and is expected to continue to grow. This can be attributed to the recognized business benefits, decreased operating costs, improvements to occupant health and wellbeing, and other social benefits [1]. While a common perception may be that delivering a project certified by a sustainability rating system, such as LEED will produce a building that is more sustainable, there is evidence that certified sustainable buildings do not always perform as expected [2,3]. The difference between the expected performance of a building, as predicted with energy modeling tools, and the “real” performance during operation has prompted changes in LEED’s certification process. Regardless of this limitation, in the twenty years since its introduction, LEED has become the most widely used and recognized credit-based building certification system for sustainability in the United States [4,5] and research on this rating system has continued to grow [6]. The LEED rating system awards certification based on the number of points obtained by fulfilling requirements across multiple categories, including sustainable sites (SS), water efficiency (WE), energy and atmosphere (EA), materials and resources (MR), indoor environmental quality (EQ), and innovation and design process (ID). In order of increasing point requirements, the certification levels are Certified, Silver, Gold, and Platinum. However, the LEED rating system is not static. Since its pilot with Version 1.0 in 1998, LEED has undergone seven revisions and split into 26 different rating systems, based on specific building sectors or project types.

This support for more sustainable design and construction practices is international. BREEAM^TM is commonly accepted as the earliest certification system and is often used as a source of comparison for other rating systems [7]. Since its inception in 1990, over 560,000 buildings have been certified under BREEAM^TM [7]. An estimated 600 other sustainability rating systems [8] have sprouted up across the world to support the building sustainability movement. Some of the most common alternative rating systems include, but are not limited to CASBEE in Japan, DGNB in Germany, Green Star in Australia and New Zealand, and ITACA in Italy, WELL Building in Greece, and Green Globes in the U.S. and Canada. In addition, LEED has developed a more expansive global presence, with adoption across 160 countries [7].

Recognizing the similar mission of achieving sustainability in the built environment, many authors have compared the sustainability outcomes of various rating systems. This has typically been done by evaluating the rating tool (i.e., scorecard) and its associated information. For example, Doan et al. [7] compared LEED, BREEAM, CASBEE and Green Star and identified indoor environmental quality (IEQ), energy, and materials as the main focus of these green rating systems. Furthermore, they noted that the mandatory credits have tended to increase over time to meet the evolving understanding of sustainability [7]. In a similar analysis, Illankoon et al. [9] compared eight rating systems and determined that Energy, IEQ, and water are the most considered key criteria. In a comparison of yet another subset of rating systems, Bernardi et al. [10] identified that a number of these tools focused on the following schemes: energy performance, solid waste management, and water. With the exception of CASBEE, a focus on energy is prevalent across rating systems. [7,9,10,11]. This array of comparison-focused research indicates that no rating system can successfully consider all aspects of sustainability [8].

Authors have also attempted to standardize points across rating systems to improve their comparisons. Mattoni et al. [11] compared five rating systems and determined that six focal areas existed across these rating systems, including site, water, energy, comfort and safety, materials, and outdoor quality. They further provided a normalized weighting scale across these categories to show where each rating system focused their credits. It was found that across all rating systems, energy and comfort and safety received the most emphasis [11]. Of relevance, they focused LEED v4 for homes and noted that this rating system focused approximately 38% of its credits in energy, 18% on comfort and safety, and 15% on site considerations. LEED also places a higher emphasis on water than other rating systems [11]. However, it is important to recognize that these weightings may not be consistent across different LEED certification areas (e.g., new construction).

Specific to LEED, such as that by Pushkar [12,13], has also compared across a single rating system. In both [12] and [13] Pushkar compared LEED certification earned under LEED version (v) 3.0 and LEED v4.0 in two different countries. In their work, they removed uncontrollable impacts (e.g., different building types, sizes) through a statistical analysis to better understand how credits were pursued under each version in each location. They showed that certain regions earned certain credits more often than other regions. With a focus on LEED Building Design & Construction: New Construction (LEED-NC), Pushkar showed that office buildings are more likely than education spaces to earn the Optimize Energy Performance credit 7 (EAc7) [14].

LEED, among other rating systems, have been challenged due to the perceived cost and impact on sustainability. According to Todd et al. [15], the LEED rating system attempts to balance two goals—one to provide a standardized form of environmental assessment and another to engage and transform the market. To transform the market, certain credits are made intentionally more challenging in a way that rewards advanced or innovative sustainable design efforts. Yet, this constant evolution is often perceived as having a higher cost, which often deters from its adoption. AlAwam et al. [16] showed that the investment in sustainability certainly can exponentiate as the highest LEED rating is achieved (i.e., platinum), but, at lower levels, the premium for LEED is a reasonable. Furthermore, the long-term benefit is even greater when the performance gap between design and reality is reduced. For example, it has been shown that LEED’s Existing Building Operation and Management can successfully achieve this gap reduction [17].

While Ref. [18] showed that the performance of some types of buildings does improve with LEED certification, sustainability is often considered a moving target, and each update to the certification process can “raise the bar” on sustainability. The perception from the industry is that, for a given level of certification, the requirements become more stringent over time [2]. In other words, a building awarded LEED Silver under the most current version would have superior performance (i.e., be more sustainable) to a similar building awarded LEED Silver under an earlier version. However, differences in versions of these ratings make it difficult to empirically determine how effective an integrated design and delivery process is in improving building sustainability. To address the need to compare the levels of sustainable performance across LEED versions, this paper presents a procedure for converting credits achieved from earlier to later versions of LEED. This system was then applied to a representative sample of certified projects from the USGBC’s public database, and a correlational analysis and an ANOVA were performed to compare the converted LEED scores across LEED versions. In addition to addressing the initial need for creating a common basis for comparing scores, this research also identifies new questions regarding whether the changes in LEED requirements and certification levels in each version are increasing sustainability over time.

2. Background

Since its pilot in 1998, LEED has gone from one rating system with 57 credits and 70 points to 26 different rating systems (e.g., LEED for New Construction, LEED for Health Care), each with their own credits and points. While the credits are similar across rating systems, the points attributed to the credit may vary considerably. The implication is that projects earning the same certification, whichever the rating system, are roughly equal in their sustainability. However, as these rating systems are updated to different versions, the credit requirements change, with the perception that the new version is more demanding [2]. The reasons for these changes can include the introduction of new technologies or materials (e.g., higher efficiency equipment), the updating of design standards (e.g., ASHRAE 90.1), and the rebalancing of sustainable priorities based on building types. The various rating systems, versions, credits, points, and certifications can make it difficult to make direct comparisons of projects using different systems or even the same system but different versions.

For many years, researchers and practitioners have theorized a link between more sustainable project outcomes and the use of integrated design and delivery processes. No current process or method exists for mapping the score of a LEED project certified with an older version to the most recent version. A few attempts have been made in specific categories, such as in the EA credits, as shown in Rastogi et al.’s [4] comparison between LEED v3.0 and v4.0. However, no studies were found that holistically compared different LEED versions or addressed how a single project might score under a newer version. The underlying assumption is that the certification levels, such as Silver or Gold, create the appropriate basis for comparison. However, there have been substantial changes in how points are awarded, especially in the EA category, and having a consistent way to evaluate the sustainability performance of projects is necessary to understand how to compare project sustainability outcomes when considering other aspects of the design and construction process, such as delivery methods or specific design practices.

Despite the lack of a comparative system for mapping or comparing versions of LEED, a few studies have used the rating system for understanding sustainability in the context of project delivery. In one of the earliest studies considering sustainable performance, Korkmaz et al. [18] collected empirical data for completed construction projects. They analyzed the effect of delivery and procurement decisions, as well as emerging green practices, in the achievement of sustainable outcomes. Although they noted the challenges with the change to LEED-NC during the data collection, no data conversion was used. Instead, the authors used a percentage of points achieved within a given system as the basis for analysis. They highlighted the need to focus on energy performance as a key subset of LEED credits. Building on this approach, Gultekin et al. [19] developed more detailed data for in-process indicators related to the pursuit of high-performance buildings, such as early use of energy simulation, experience with high-performance green buildings, and owner requirements for hiring a LEED or green design consultant. The authors used the percentage achievement of LEED points to conceptualize a “level of green,” as well as weighting the energy efficiency and indoor environmental quality credits in assessing the high-performance green index. The authors found that greater design experience with high-performance green buildings, early use of simulation tools, and increased communication were important for achieving sustainable outcomes.

While these studies were able to leverage the existing systems to compare sustainability outcomes across projects, the use of the percentage of points achieved is a compromise. Korkmaz et al.’s [18] study was quite early in the adoption and implementation of LEED, with fewer systems and versions to consider within the data. Similarly, Gultekin et al. [19] constrained the data collection to a subset of project types and versions to limit variation in LEED point submissions specifically to avoid the complications that they bring in analyzing a consistent approach to sustainable outcomes. Thus far, all of the literature in the project delivery space ignores the shift in emphasis introduced across versions, added granularity within certain points or topics, and the changes in how some points are assessed in favor of analytical simplicity. As new versions and systems have emerged, this problem has only compounded.

3. Research Methods

In this research, LEED scorecards from LEED v2.1 (released in 2002) [20] and v2.2 (released in 2005) [21] were converted to the LEED v3.0 (released in 2009) [22] rating system. Versions 1.0, 2.0, and 4.0 were excluded from the study because, at the time of this research, there was an insufficient quantity of projects among those versions to provide meaningful analysis. To convert LEED scorecards, point requirements were reviewed credit-by-credit, and a direct comparison of the credit text was performed between LEED versions (i.e., v2.1, v2.2, and v3.0). This comparison enabled the development of multiple conversion matrices that allowed a project scorecard from an older version of LEED (e.g., v2.1) to be converted to a new version (e.g., v3.0). Following the development of conversion matrices, the matrices were then applied to a sample of LEED-certified building projects to better understand the impacts of each conversion process. A random sample was drawn from the publicly available USGBC project database using a two-level stratification. These processes are described in greater detail in the sections that follow. The merits of the conversion process are then discussed as recommendations for anyone seeking to evaluate how a project may be certified if it were constructed and certified under a later version of LEED.

3.1. Development of Conversion Matrices

Converting scores between LEED versions can be done by either taking a forward pass that aligns older credits with current standards or a backward pass to align newer credits with older requirements. Because LEED versions build on one another and updates begin with previous standards, the researchers developed a system that converted scores from older versions of LEED to the newer v3.0. This included a direct conversion from v2.1 to v3.0 (Path A), a two-step conversion from v2.1 to v2.2 to v3.0 (Path B), and a conversion from v2.2 to v3.0 (Path C), as shown in Figure 1. In theory, Path A and B should provide the same outcome. This was tested and is discussed in the results that follow. By taking a forward pass, future extrapolation can occur as more project data is collected for later versions, such as v4.0.

In any version of LEED, prerequisites must be completed prior to earning points in a category. This research fundamentally assumed that all prerequisites would always be earned, even if that prerequisite has evolved over time. With this assumption in place, the thresholds and points required for each credit were then analyzed and a conversion performed.

To convert scorecards, point requirements were reviewed, credit-by-credit, and a direct comparison of the credit text was performed between LEED versions (i.e., Path A, Path B, and Path C). Commonalities were identified among the types of changes that occurred; thus, these changes were categorized and are described in the paragraphs that follow and are depicted in

Table 1. Summary of categorized changes between consecutive version of LEED.

Change Category		v2.1 to v2.2	v2.2 to v3.0	v2.1 to v3.0
1.	Added a requirement 1	1	0	1
2.	Increased the strictness of the requirement 1	6	3	4
3.	Complete requirement change 1	3	1	3
4.	Addition of an option, with an added requirement or increased strictness 1	5	2	9
5.	Either addition of requirement or increase in the strictness of requirement within multiple possible paths 1	0	2	0
6.	Addition of an option with no other changes 2	6	3	6
7.	Decrease in the threshold of requirement 2	8	2	9
8.	Addition of an option with decreasing requirements or decreasing thresholds 2	1	0	1
9.	Minor Changes 3	19	20	16
10.	No Change 3	3	18	2
11.	Converted to a requirement	0	1	1

Credit considered: ¹ More challenging to earn; ² Easier to earn; ³ The same difficulty to earn.

. During the conversion process, the text was reviewed to determine if the project would still earn the credit under the new requirements, would only receive partial points if the requirement threshold were altered, would earn more points, or would achieve no points. For some categories, quantifying changes in requirements between LEED versions was not always possible. For this reason, two separate conversion matrices were developed: strict and interpretive. The interpretive approach allowed for partial points to be earned when some incremental change occurred. In contrast, the strict approach eliminated the fuzziness from the data and assumed an all-or-nothing approach to changes.

3.2. Strict Versus Interpretive Approaches to Conversion

In the strict approach, if the credit requirements were altered in the newer LEED version such that (1) an additional requirement was added to achieve the point, (2) the requirement threshold was altered in a way that made it more difficult to achieve, or (3) the requirement completely changed, then a project would not receive any points for that credit in the transformation process to the newer version. However, if the credit requirement became more lenient or stayed the same, then the project would continue to earn points in the newer version under this strict approach. By pursuing this strict approach, the potential bias of the researcher was reduced by limiting the points to those that could be confirmed as either exact matches across the versions or more lenient changes in the new versions. However, as a result, the strict interpretation was more conservative regarding the interpretation of points across versions of the requirements.

The interpretive approach, on the other hand, acknowledged the different types of changes in point requirements between LEED versions and used an expected value (EV) calculation to determine whether a project scorecard would earn more, less, or the same number of points. While performance thresholds can be inferred from the points on the project scorecard, exact values are not known; therefore, the identification of the EV provided a means of accounting for this uncertainty in the conversion process. In defining the quantifiable impact, partial points could be awarded in the transformation to determine how close the project was to earning certification in a newer version of LEED. In applying the interpretive approach, several assumptions were needed. These include:

• When earning a credit, the project met the credit requirements, but did not exceed the credit’s required thresholds.
• When earning a credit with multiple paths (or options) available to earn the credit, the project was equally likely to pursue each path.
• When additional requirements were added, all requirements had equal weights.
• When a threshold was changed, the amount of effort to reach the new threshold was assumed to be linear.

3.3. Categorization of Credit Changes

The credits were categorized in accordance with the types of changes that occurred as the LEED requirement evolved. A process was then established for awarding points within each category of change that occurred. The following bullets detail this process for the strict and interpretative approaches and provide examples of how points would be awarded under each.

3.3.1. Added a Requirement

As LEED evolved, additional requirements were sometimes added in later versions. For example, for EQ Credit 1: Carbon Dioxide (CO2) Monitoring the requirement in v2.1 was the installation of “CO2 Monitoring system that provides feedback on space performance in a form that affords operational adjustments” [20]. In v2.2, this requirement was held, and the requirement was added to “configure all monitoring equipment to generate an alarm when the conditions vary by 10% or more from setpoint, via either a building automation system alarm to the building operator or via a visual or audible alert to the building occupants” [21]. Under the strict approach and based on the assumption that the project did not exceed the requirements, it can be assumed that the project did not meet all requirements under the new version as an alert system was not previously required and, thus, the project would not have earned the credit. In the interpretative approach, when an additional criterion is added in a later version to a single existing criterion, 50% of the credit requirements would have been met, and the project would receive an EV = 0.5. Similarly, when a single criterion is added to two existing criteria in an updated LEED version, the project would have an EV = 66.6% (assuming the project earned that credit), which correlates to the percent of requirements that the project would have certainly fulfilled in the newer version. In this example, the addition of a second requirement (alert system) to a single existing requirement (CO2 monitoring system) occurred. Thus, 50% of the credit requirements were met in the new version, and the EV = 0.5.

3.3.2. Increased Strictness of Requirement

As sustainability knowledge and technology have evolved, the strictness of the requirement was sometimes increased within a credit. For example, in SS Credit 6.2: Stormwater Design, the stormwater runoff required to be captured and treated increased from 80 to 90% between v2.1 and v2.2. Under the strict approach, no points are earned as we assumed the minimum threshold was all that was performed. Thus, only 80% runoff is assumed to be captured and, in the updated version, this is insufficient to earn credits, whereas under the interpretive approach and aligned with assumption four, we assumed that the effort is linear (e.g., the effort required to achieve that extra 10% reduction is assumed to be the same as the initial 80%). Therefore, the EV in this category was calculated as the minimum quantity required to be completed in the older version divided by the minimum quantity required to be completed in the newer version and EV = 80/90 = 0.89.

3.3.3. Complete Requirements Change

At times, the requirement for a credit underwent a drastic change that is no longer comparable to the previous version. When this happens, we assume that past actions would not earn the credit. For example, EQ Credit 7.2 substantially changed its requirements from v2.1 to v2.2. This credit was called Thermal Comfort: Permanent Monitoring system in v2.1 and required “permanent temperature and humidity monitoring systems.” In v2.2, this credit was called Thermal Comfort: Verification and required a “thermal comfort survey within a period of six to 18 months after occupancy.” For this credit, the requirements changed completely, and the installation of monitoring systems would not earn points in the later version. Thus, no points were earned when applying either the strict or interpretive approach.

3.3.4. Addition of Option along with an Added Requirement or Increased Strictness

When an option was added in a newer version of the LEED standard, there was increased flexibility for the project to achieve that credit. This was because a second path was created. However, that option was not available for the project in the earlier version, and therefore is assumed that path was never pursued. We can consider this situation as a single path and evaluate only the path that continued to exist to determine its EV in accordance with the principles outlined in Section 3.3.1 (Added Requirements) or Section 3.3.2 (Increased Strictness of Requirement). For example, in v2.1 for the SS Credit 2: Development Density & Community Connectivity, the requirement to meet development density focused on ensuring that the project was situated within an area with a minimum density of 60,000 SF/acre. In v2.2, this requirement was updated to require that the project be situated within an area with a minimum density of 60,000 SF/acre and that the project was situated on a previously renovated/developed site. An option was added in v2.2 as well. This option allowed for projects to pursue a path that focused on community connectivity in which the project must be located near basic services to earn this credit instead. Under the strict approach, we conclude that while a new path was added, this path can be disregarded as it did not previously exist. Based on the assumption that the project did not exceed the requirements (see Assumption #1), within the strict approach, it can be assumed that the project did not meet all requirements under the new version because the project was not previously required to be on a previously renovated site. Therefore, the project would not have earned the credit. Under the interpretive approach, we also conclude that while a new path was added, this path can be disregarded as it did not previously exist (see Assumption #1). However, in this example, the change increased the strictness of the requirement. Since a requirement was added, there were now two requirements (minimum density and previously developed site). Aligned with Section 3.3.1 (Added Requirement) description, each of these requirements was assumed to be equal weighting (0.5 points per requirement). The project would have achieved 0.5 points total for achieving the minimum density but would not have earned the extra 0.5 points from developing on a previously developed site.

3.3.5. Either Addition of Requirement or Increase in the Strictness of Requirement within Multiple Possible Paths

Assumption #2 states that, when multiple paths existed, each path was equally likely to be taken for a project (i.e., per path EV = 1/n where n is the number of paths). Therefore, if one path is altered as described in Section 3.3.1 (Added Requirements) or Section 3.3.2 (Increased Strictness of Requirement), the EV of that path is adjusted accordingly, but the EV of the other two paths remains the same. For example, in v2.2 for SS Credit 2: Development Density & Community Connectivity two paths exist. In v2.2, in the first path, a project could earn credits if it were situated within an area with a minimum density of 60,000 SF/acre and that the project was situated on a previously renovated/developed site. In the second path of v2.2, the requirement was to construct on a previously developed site and within ½ mile of the residential area of density 10/units per acre and within 0.5 miles of 10 basic services. These basic services could all be included in the project. The first path remained the same in v3.0; however, one component of the second path was altered, and the 10 basic services requirements were adjusted to indicate that at least 8 must be operational and only two could be planned as part of the project for mixed-use projects. Under a strict approach, because we do not know which path was taken, we assume no points were earned to be conservative in point estimation. However, under the interpretive approach, each path is worth 1/n points, where n = 2. When a path does not change requirements, the value of that path stays the same (Path 1 = 0.5 × 1). In path two, there are multiple and statements, and only one had a reduced threshold. Therefore, this path is split into three components (see principles associated with Section 3.3.2, Increased Strictness of Requirement), where two components remained unchanged (i.e., multiplied by 1) and the third component was reduced by what certainly was achieved (aligned with Assumption #4). In this case, the newer version allows only two facilities to be planned as part of the project; thus, the conservative approach was to assume that projects that pursued this credit previously only planned to include basic services in their project and that none were previously existing. Therefore, only 2/10 of possible services would be considered as completed under the previous version of LEED. This leaves the following EV for Path 2: (Path 2 = 0.5 × ((0.3 × 1) + (0.3 × 1) + (0.3 × 2/10)) = 0.33. Adding paths 1 and 2 together, the EV = 0.5 + 0.33 = 0.83.

3.3.6. Decrease in Strictness of a Requirement

At times, no options were added, but the newer versions of LEED decreased the strictness required to earn that credit. When this happened, past actions would still earn the credit in the newer version and EV = 1.0. For example, in SS Credit 4.1: Alternative Transportation–Public Transportation Access, two possible paths existed in both v2.1 and v2.2. In v2.1, the first path requires projects to be located within “1/2 miles of a commuter rail, light rail, or subway station” [20]. In the second path, the project could be located within “¼” mile of two or more public or campus bus lines.” [21]. In v2.2 [22], the first path created greater flexibility by allowing the rail stations to be planned and funded. (“Locate project within 1/2 mile of an existing, or planned and funded, commuter rail, light rail or subway station.”) The second path remained the same. This allowed greater flexibility to achieve this credit. For both the strict and interpretive approaches, one point would be awarded as the requirements for this credit were decreased in strictness and previous actions would still have earned the credit.

3.3.7. Decrease in the Threshold of a Requirement

At times the newer versions of LEED had a decrease in the threshold required to earn that credit. When this happened, past actions would still earn the credit in the newer version, and additional points could be earned beyond the required threshold (similar to the principles asserted in Section 3.3.2, Increase in Strictness of Requirement). For example, in v2.1 for MR Credit 6: Rapidly Renewable Materials, the use of rapidly renewable building materials and products were required to be 5% of the total value of all building materials. In v2.2, this threshold decreased to 2.5% of the total of all building materials. In the strict approach, the previous actions would have continued to earn credits, and thus, 1 point was awarded. However, for the interpretive approach, when a project performed actions that went above the required, it could earn additional points. In this case, projects certified at 5% threshold in v2.1 were double the requirements for v2.2. Aligned with Assumption #1, the project was assumed to complete the minimum requirements. Aligned with Assumption #4, we also assumed that the effort is linear. Therefore, the EV in this category was calculated as the minimum quantity required to be completed in the older version divided by the minimum quantity required in the newer version, and EV = 5/2.5 = 2.

3.3.8. Addition of an Option

In some cases, an option was added as a way to achieve the credit, but the existing options did not change. In these cases, the project would continue to earn points because past activities still meet the requirements. For example, in SS Credit 4.3 Alternative Transportation Low Emitting and Fuel-Efficient Vehicles, two options existed to achieve the credit in v2.1. These included: (1) providing alternative fuel vehicles for 3% of the building’s occupants, as well as preferred parking for these vehicles, and (2) installing alternative-fuel refueling stations for 3% of total vehicle parking capacity. In v2.2, these two options remained, but the option to provide preferred parking for low-emitting and fuel-efficient vehicles for 5% of the total vehicle parking capacity was added. For both the strict and interpretive approaches, one point would be awarded as the requirements for this credit as the existing paths did not change, and thus, the project would earn credit for what had been done in the past.

3.3.9. Minor Changes

Minor changes such as clarity in word choices or adding a single component may be included in later versions. However, these changes were deemed unsubstantial and did not impact the ability to earn points. For example, in v2.1, for SS Credit 4.2, Alternative Transportation: Bicycle Storage & Changing Rooms, the requirement stated that bike storage must be available for “5 or more of regular building occupants.” In v2.2, this terminology changed to “5% or more of all building users (measured at peak periods).” Under both the strict and interpretive approaches, this change was determined to not substantially impact the project’s ability to earn points; thus, points continued to be awarded in the later version.

3.3.10. No Change

If there are no changes between versions, or if there were minor terminology changes that were for the purposes of clarification, the credit will always be awarded in later versions, and EV = 1.0. It is worth noting that metric units were included in v3.0 but this change was considered irrelevant for certification purposes.

3.3.11. Converted to Prerequisite

In one case, a credit was converted into a requirement. Fundamentally, we assumed all requirements were always met. In this case, no points were awarded in newer versions because these points no longer existed once the credit became a requirement. This occurred in the WE Credit 3.1 Water Use Reduction: 20% reduction between v2.2 and v3.0, where the requirement to “Employ strategies that in aggregate use 20% less water than the water use baseline calculated for the building” was moved from being an earned credit to a pre-requisite.

3.3.12. Other

Lastly, several credits had circumstances that altered the mathematical calculations of the points awarded in the expected value calculations. Each of these cases is described below.

• For the EA Credit 1: Optimize Energy Performance, the ASHRAE/IESNA standard was updated in each version and the number of available points changed between v2.2 and v3.0. This required the creation of a scale to compare the energy savings and the awarded points across the subsequent versions. Appendix A provides more details on this change, where points decrease while the threshold rises.
• In EA Credit 2: On-site Renewable Energy, the available points for the credit changed between v2.2 and v3.0. In this case, the points were updated based on the required thresholds for each new point value.
• In Materials and Resources Credit 1.1 Building Reuse–Maintain Existing Walls, Floors, and Roof, the threshold required to earn the credit was changed between v2.2 and v3.0. In v2.2, one point could be earned by maintaining at least 75% of the building’s existing structure and envelope. In v3.0, maintaining this same 75% would earn two points. In v2.2, this was the minimum threshold. However, in v3.0, the minimum threshold earning that one point was 55%. Following Assumption #1, we assume that projects complete the minimum required. Because the new points available for performing 75% (the previous minimum required) were now two, two points would be awarded for this credit in both the strict and interpretive approach.
• The available points in ID Credit 1: Innovation in Design increased between v2.2 and v3.0 from four points to five points. In the strict version, no additional points were earned and a credit that earned 3 points in this category would always earn 3 points regardless of version. This is done because what was considered innovative in v2.1 may not have been considered innovative by the time v3.0 came out. Because of the flexibility of this credit and the lack of a standardized approach, in the interpretative approach the researchers determined that each point earned would be worth 5/4 = 1.25 points in v3.0. Thus, a project that earned 3 points in v2.1 or v2.2 would then earn 3.75 points in v3.0.
• In v3.0, the Regional Priority credit was introduced. Because earlier versions did not have this credit, it was excluded from any transformation and subsequent analysis.

Table 1 summarizes how many times each change occurred between versions. Appendix B provide a complete list of credit names and Appendix C details each change category by credit. As can be seen in Table 1, 30% of the credits became more difficult to achieve in the update to v2.2 (rows 1–5), 30% of the credits became easier to achieve (rows 6–8), and 40% of the credits were equally difficult to achieve (rows 9–10). In the update from v2.2 to v3.0, 20% of the credits became more difficult to achieve, 15% became easier to achieve, and 65% of the credits were equally difficult to achieve. The number of credits that became easier to achieve in the transformation from v2.1 to v3.0 was approximately 33%. Another 33% became easier to achieve and the remaining 35% did not experience significant changes. Finally, in v3.0, one credit became a requirement and thus was excluded from the analysis.

3.4. Sampling Procedure

To better understand how this conversion methodology impacts project certifications, a stratified sampling approach was used to collect a representative sample of project scorecards from the USGBC database. To identify the number of projects, the population was limited to LEED-NC projects and used a two-level stratified sampling approach. The strata are defined by LEED version (I.e., v2.1, v2.2, and v3.0) and certification level (I.e., Certified, Silver, Gold, Platinum) to ensure that the random sample adequately represents the population of projects in the database.

Table 2. Population and sample size of projects registered under LEED Building Design & Construction: New Construction in the U.S.

LEED Version	v2.1		v2.2		v3.0
Certification Level	Population	Sample Size	Population	Sample Size	Population	Sample Size
Certified	383	17	774	8	1290	20
Silver	455	21	1940	21	2913	45
Gold	370	17	2548	27	2210	34
Platinum	70	3	352	4	418	6
Total	1278	58	5614	60	6831	105

shows the total population of each category when the data was extracted from the LEED project directory website (usgbc.org/projects) on 28 January 2020. Then, the data were randomly sampled in accordance with the number of projects needed to achieve a 99% confidence interval of the mean score.

3.5. Converting the Sample

In both the strict and interpretative approaches, the conversion matrices were applied to each project’s scorecard to transform the credits earned into the later LEED version (e.g., credits earned in v2.1 were converted to equivalent credits that would be earned in v.2.2 and v3.0). At times, the points available for the credit were increased in later versions. This can be exemplified in the SS Credit 2 Development Density & Community Connectivity credit, where the available points started at one in v2.2 and increased to five in v3.0. This credit fell in the change category either addition of the requirement or an increase in the strictness of the requirement within multiple possible paths, and earned 0.83 after transformation. These converted points were multiplied by the available points to create the finalized scores. This was done through the following calculation: 0.83 × 5 points = 4.15 points. This process was followed for all credits. The final points for each project were then summed and rounded to the nearest whole number. This number was then used to assign a new certification value based on the version it was converted to (see

Table 3. LEED Certification point comparison for v2.1, v2.2, v3.0.

	Point Range by Version
Certification Level	v2.1	v2.2	v3.0
Certified	26–32	26–32	40–49
Silver	33–38	33–38	50–59
Gold	39–51	39–51	60–79
Platinum	52–69	52–69	80–110

). The results are discussed in the section that follows.

4. Results

Changes in Certification Level

The following section presents the results of converting the scorecards on the stratified sample along Paths A, B, and C (Figure 1) to better understand the impacts of the conversion process. After applying the conversion, the certification level that each sampled project earned often decreased by at least one certification level, sometimes more than one.

Table 4. Summary of post-conversion certification level changes for projects in the stratified sample.

	Interpretative			Strict
Percent of Projects:	Path A	Path B	Path C	Path A	Path B	Path C
Stayed in Same Certification Level	22%	12%	7%	0%	0%	0%
Dropped 1 Certification Level	59%	67%	72%	53%	45%	42%
Dropped 2 Certification Levels	19%	21%	22%	43%	52%	55%
Dropped 3 Certification Levels	0%	0%	0%	3%	3%	3%

depicts these certification changes. Most prominently displayed through this information is that 59% of the sampled project dropped one certification level from the direct conversion from v2.1 to v3.0 (Path A), 67% of the sampled projects dropped one certification level in the two-step conversion from v2.1 to v2.2 to v3.0 (Path B), and 72% of the sampled projects dropped one certification level between v2.2 and v3.0 (Path C) across the interpretative approach. In the strict approach, more projects dropped a greater number of certification levels (e.g., two certification levels), with no projects staying in the same category in which the project was originally certified. These results suggest that the same certification level becomes more difficult to obtain in more recent versions of LEED.

• Correlational Analysis

To test the hypothesis that LEED scores increase over time, a bivariate Pearson correlational analysis was performed between the converted point totals and time. Time was represented numerically by the number of days between the date of the earliest LEED certification in the sampled projects and the date of each project’s LEED certification in the USGBC database. The results of this analysis are summarized in

Table 5. Correlation table of time and LEED point totals from various transformation methodologies.

	1.	2.	3.	4.	5.
1. Time	1
2. Points, Paths A and C, Interpretive	0.341 **	1
3. Points, Paths A and C, Strict	0.489 **	0.968 **	1
4. Points, Path B, Interpretive	0.370 **	0.993 **	0.968 **	1
5. Points, Path B, Strict	0.518 **	0.954 **	0.994 **	0.965 **	1

Notes: * p ≤ 0.05; ** p ≤ 0.01; *** p ≤ 0.001.

. Time had a significant (p < 0.01) positive correlation with the converted point totals. The strict transformations had stronger correlations with time (0.489 for Paths A and C, 0.518 for Path B), than the interpretive transformations (0.341 for Paths A and C, 0.370 for Path B). The correlations of point totals with time were similar in magnitude and directionality, regardless of transformation path. The results of this analysis suggest that a small to moderate increase in LEED points is occurring over time.

b.

Analysis of Variance in Converted Points by LEED Version

A one-way analysis of variance (ANOVA) was performed to compare the effect of LEED version on the converted points obtained by the sampled projects. When using a 1-step conversion to v3.0 (i.e., Paths A and C), a one-way ANOVA revealed a statistically significant difference in converted points between at least two versions of LEED for the interpretive (F(2, 220) = 26.78, p = 0.000) and strict (F(2, 220) = 71.02, p = 0.000) methodologies. A visual comparison of these differences is shown in Figure 2. A Tukey’s HSD test for multiple comparisons for the interpretive methodology found that the mean converted points was significantly different between projects converted on Path A and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−14.85, −6.99]) and projects converted in Path C and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−12.77, −4.99]). The mean converted points for the strict methodology were also significantly different between projects converted on path A and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−20.92, −13.40]) and projects converted in path C and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−17.32, −9.88]).

When using a 2-step conversion to v3.0 (i.e., Path B), a one-way ANOVA revealed that there was a statistically significant difference in converted points between at least two versions of LEED for the interpretive (F(2, 220) = 32.81, p = 0.000) and strict (F(2, 220) = 86.33, p = 0.000) methodologies. A visual comparison of these differences is shown in Figure 3. Similar to the 1-step results, a Tukey’s HSD test for multiple comparisons for the interpretive methodology found that the mean converted points were significantly different between projects converted on Path B and project originally certified in v3.0 (p = 0.000, 95% C.I. [−16.07, −8.39]) and projects converted in Path C and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−12.68, −5.08])., Comparisons for the strict methodology found that the mean converted points were significantly different between Path B and project originally certified in v3.0 (p = 0.000, 95% C.I. [−22.53, −15.24]) and Path C and projects originally certified in v3.0 (p = 0.000, 95% C.I. [−17.20, −9.99]).

The results of the ANOVA and posthoc tests show that the mean converted points are not significantly different between LEED versions 2.1 and 2.2. However, there is a significant difference between projects certified and converted from both earlier LEED versions (v2.1 and v2.2) and projects certified in v3.0, with v3.0 having the higher mean.

5. Discussion

These findings have both academic and industry implications. For academics, this study provides a methodology for evaluating the sustainability performance of building projects, so that research can better investigate the link between integrated design and construction practices. For practitioners, the study provides evidence that clarifies a common perception that newer LEED versions have “raised the bar” on sustainability as it evolved from v2.2 to v3.0.

5.1. Implications of Point Conversion Methods

In this study, two different approaches were considered to convert a project’s LEED scorecard from its originally certified version of LEED to a later version, specifically v3.0. The interpretative method of conversion allowed for projects to achieve partial points towards their certification based on the change category to which it was assigned. While the LEED rating system does not typically allow for partial points, the effort towards a credit that became more difficult to achieve in later versions (see Table 2) should be considered during the conversion process, as some progress was made towards the sustainability goal, even if the target itself changed. The strict approach took an all-or-nothing perspective for awarding points. Because partial effort was not rewarded, the strict transformation approach had a more substantial negative impact on the level of certification earned than the interpretive approach.

Value can be obtained from both the strict and interpretive approaches to converting project score cards to a later version of LEED, but it depends on the question that the user is seeking to answer. The strict approach can answer the question: How many credits would this project have definitively earned under a newer version of LEED? This approach to transforming credits is inherently conservative and provides a worst-case answer. Thus, in general, the strict approach is most appropriate in determining a lower boundary on the converted certification level. The interpretative approach, on the other hand, attempts to account for some uncertainty—specifically, the lack of detailed information available in the LEED scorecard—to predict the level of certification that a project may have received if it were certified under a more recent version of LEED. The interpretative approach can answer the question: What certification level is this formerly certified project most likely to achieve in the latest version of LEED? This question and the interpretative methodology provide some recognition of the pursuit of sustainability and provide value in determining how sustainable former projects were relative to the newest LEED thresholds.

Two different paths for converting scorecards from v2.1 to v3.0 were also considered, a direct, 1-step conversion across multiple versions of LEED (see Path A in Figure 1) and a longitudinal, 2-step conversion across multiple versions of LEED (i.e., Path B). Path A had less degradation across projects than the 2-step approach (Path B), as can be seen in Table 4 from the higher total percentage of projects that had no certification changes or only a single level decrease in certification. This is because credits converted in the direct approach (Path A) were not negatively impacted when the credit became more difficult to achieve in the next version and easier to achieve in the version after. Rather, this direct conversion skipped the negative impacts and directly considered the thresholds in the latest version. In contrast, the longitudinal transformation was negatively impacted by the quantification of credits that became more difficult to achieve in an intermediate version of LEED (e.g., v2.2) because this approach could not necessary re-award points if the credit requirements were subsequently loosened in a later version (e.g., v3.0). The longitudinal transformation considered in Path B likely overweighed the negative impact of the increased strictness in v2.2 and underweighted the increased flexibility given by v3.0.

Thus, the recommended approach is to follow a direct transformation approach (e.g., Path A, Path C) to determine the most realistic certification level and point score a project would obtain if it were certified under a newer version of LEED. However, this approach can be time intensive. If, however, the company or researcher is continually updating older versions of LEED to the latest version, a longitudinal approach would be easier to pursue because a direct transformation matrix can be converted each time an updated version is released and that transformation matrices can be multiplied by each other over time. However, in doing so, projects may experience a greater drop in recognized points due to compounding changes and data loss across multiple transformations; likely more than is realistic as observed in the two-step path presented. The longitudinal conversion (Path B) is therefore a overly conservative in updating LEED scorecards within the interpretative approach.

5.2. Industry Impact

In addition to the value of the method for converting among LEED versions, the results provide some evidence that supports the perception that point scores are increasing over time. These results were consistent for both the strict and interpretive paths. The increase in achieved points in V3.0 suggests that the industry is improving at designing, constructing, and certifying more sustainable buildings. One potential explanation for this is that, in addition to the growing interest and encouragement of pursuing sustainable design and construction, our industry is maturing in its capacity to design more sustainable buildings. As designers become more familiar with emerging technologies, and gain familiarity with more sustainable design practices, such as energy modeling, the standard for project design and some of the common components or materials have become more sustainable. Thus, the minimum bar for design on all projects has likely been improving and designers can pursue more ambitious sustainability goals, potentially targeting a higher certification level than would have been feasible a decade ago. It is important to note that the despite the improvements observed in V3.0, the differences between V2.1 and V2.2 were not significant.

In addition to observing improvement in sustainability across projects through the growth in achieved points, the results also show a shift in certifications levels from older to newer LEED versions. This means practitioners can expect more challenging sustainability goals in each subsequent version of the LEED rating systems. Essentially, in addition to refining how projects can earn points across the different credits on the scorecard, the USGBC has also been “ratcheting up” the minimum requirements to achieve the Certified, Silver, Gold, and Platinum recognition. Some of these are apparent, through the addition of required points within certain areas, while others may be less obvious, such as transitioning to newer and more stringent standards as the baseline against which performance is analyzed, such as the ASHRAE 90.1 envelope performance requirements. While the comparison across versions is not direct in all cases, the triangulation of findings across both the increase in points and the shift in certification both point to the long-term impacts that LEED has had on building sustainability, through the improvements in their rating systems over time.

5.3. Limitations

In developing and executing the conversion process, the researchers acknowledge several limitations. First, the change in points earned had natural decay. This could be because the transformation could more easily evaluate how much stricter a credit became and the likelihood that a credit would continue to get points when considering this increased strictness. The conversion process often could not account for the benefit realized from the 21 credits across two versions that got easier to achieve unless the threshold could be quantified. Yet, the possibility remains that projects could have earned more points if it were possible to quantify how far a project progressed towards a credit that became easier to achieve over time. Assumptions #1–4 contributed to this limitation. In addition, Regional Priority Credits were introduced to acknowledge that sustainability is location specific. By eliminating that consideration from the common scoring system, its value is diminished. Future research could consider project data available from LEED to retroactively apply Regional Priority Credits to projects using earlier versions. Lastly, the energy conversions were done using a conservative adjustment. However, a sensitivity analysis of energy reduction across ASHRAE 90.1 versions could be done to determine a more accurate impact of energy rating on the whole of the project’s rating under the common scoring system to determine if its weighting is appropriate.

6. Conclusions

Sustainability rating systems for building projects are constantly evolving, and there is a perception among industry professionals that the rating requirements become more stringent with every new version. The purpose of this research was to investigate whether there was evidence to support or contradict this perception. A credit-by-credit review of LEED v2.1, v2.2, and v3.0 was performed to identify and categorize the types of changes that occurred between versions. A total of ten unique changes were identified, ranging from the addition of requirements to increasing or decreasing strictness of existing requirements, to the addition of alternate routes to the same number of points. Once the changes were identified, the researchers developed a conversion methodology to translate scorecards from older to newer versions of LEED, which considered the impact of multiple approaches to dealing with the changes (i.e., strict and interpretive) and multiple conversions (i.e., 1-step and 2-step). The conversion process was tested on a stratified sample of project scorecards drawn from the USGBC database. The results indicated that a strict conversion produced the most conservative estimate of LEED ratings, when moving from older to more recent versions of LEED and represented the lower rating boundary. The interpretive approach recognized that some uncertainty exists in fulfillment of credit requirements that is not captured in the scorecard data. Using expected value calculations, the interpretive approach provides a less conservative, but likely more realistic prediction of what projects certified under an older version of LEED may have obtained in the more recent versions. After the conversion, most sampled projects decreased by one or two certification levels, either from Silver to Certified or Gold to Silver, providing evidence that higher certifications levels are more challenging to obtain in new versions of LEED. Regardless of the approach, converted LEED scores were found to be positively correlated with time, providing evidence that sustainability of buildings increased from 2006 to 2017. An ANOVA of converted LEED scores by version revealed no statistically significant difference in mean score between converted scores from v2.1 and v2.2. However, the mean scores of v3.0 were significantly higher than those originally certified in v2.1 and v2.2.

In addition to providing the direct results of the analysis across versions, the method for converting across LEED versions offers a contribution to the academic community that will allow further research into factors contributing to sustainable project outcomes. The rich pool of data offered through the database of projects housed in USGBC records provides an opportunity for further empirical research into the project delivery, integrated design, analytical methods, systems, and a broad array of other topics that potentially influence sustainable facility design and construction. By providing a method for comparison of projects across versions, the ability to use data from the larger pool greatly expands the value and opportunities for empirical analyses using these data. In future work, each credit can be critically examined to determine how credits have evolved alongside the definition of sustainability itself and how those changes impacted the overall sustainability of LEED certified buildings. Another opportunity for future work includes a comparison of LEED individual credit changes against other rating systems to serve as a verification for the trajectory of sustainable buildings.