Tools are often presented with a table of contents where the different captions represent different hierarchical levels. Different tools use different number and naming of these levels, but at least four hierarchical levels can often be found. The names proposed in this framework (definitions and descriptions of these levels are found in Section 4.2 Content) are:

In order to communicate environmental information of various kinds, many assessment tools transform the complex information as presented above to a condensed result: as a final label or a few aggregated values. The prerequisite for aggregation is a hierarchical structure (Figure 1).

The graphical method is inspired by Sutrisna and Barrett [36], who demonstrate how “Rich Picture Diagrams” can be applied to modeling case studies to capture the richness of the information in construction projects. The Rich Picture Diagrams are used to “improve presentation and enable thorough cross-case analysis by providing a holistic view” of the construction project storylines [36]. In this framework, this type of hierarchical diagram (as in Figure 1) provides an overview of the structure and its components, which improve the understanding of one or more BEATs (see Figure 2). The diagrams in Figure 2 illustrate how LEED, CSH and EcoEffect are structured, and how they aggregate the large number of building environmental issues addressed into one, two, or three final results. The outdoor environment in EcoEffect is not presented. Differences in hierarchy, aggregation and number of issues available are shown. However, each issue can contain a number of parameters, which are not included in the diagram.

The next part of the framework explores the content of the tools in more detail. The framework proposes (Table 1) that the comparison starts with the results and continues with the labels as the assessment result for a building is often presented as a label. The number of labels and the method of giving scores, including the scoring intervals, are also of interest. This is followed by a closer study of the categories (sometimes “areas” or “aspects”). From examination of several tools the following general set of categories has been chosen:

The great majority of assessed issues could easily be subsumed under the first six headlines, if not there is a possibility to use the last ‘Else’ caption. As the categories can overlap each other there is a risk that one issue can fit into two categories. When the issues have been categorised they are further compared and categorised depending on the kind of aspects they assess:

Organising the categories and issues as in Table 2 presents a comprehensive overview. If an issue assess several aspects it can be categorised as several types of issues.

The last aspects of “content” to compare are the used parameters, demanded input data, and intrinsic data included in the BEAT. It is often difficult to measure issues and environmental impacts directly. For this reason, different kinds of approximations are sought out, which are often called “indicators.” To measure an indicator, one or several parameters are needed. A parameter is here being defined as a property or unit that can be measured or observed. Each issue therefore has one or several parameters to assess the building. Other data included in the tools that may influence the results are, for example, built-in background data, upstream data (data associated with the production of materials and products (e.g., production of raw, auxiliary and operating materials), downstream data (data associated with the disposal of materials and products (e.g., waste treatment)) and defined and/or calculated life expectancy. There is always a risk that different input data are used and interpreted in different ways, e.g., there are several ways of measuring the floor area of a building. It can be one of the reasons why the same building can be given different results by different assessors.

The methodology used for aggregation and weighting and what weights different issues and categories have, is analyzed separately in the framework to highlight the importance of these issues. How the BEAT aggregates and weights issues and categories influences the assessment result. To produce a conclusive result and an environmental label, aggregation is necessary. A way to compensate for varying significance between issues is to vary the number of scores that can be obtained. This indirectly means a weighting. A good result on an issue where 4 points are available will contribute twice as much to the final result as an issue where only 2 points are available. Another way to compensate for variations in significance is to assign different weights to issues and/or categories. The weighting can be based on e.g., opinions, cost or damage. Opinions can be taken from environmental experts, professionals or laymen, and the weighting figures can be calculated from many answers, for instance, mean values or consensus values reached through a discussion process. There are various multi-criteria decision-making tools used in Life Cycle Assessment [38].

The final part of the framework covers the functional, spatial, and temporal boundaries, as well as assessed impacts included. To make a precise comparison of buildings, they need to offer the same services and function. That is why assessment developers have devised specific versions of tools for different kinds of buildings, e.g., types of use such as houses, schools, offices, etc. To improve comparability of building performance, the European working group on “Sustainability of Construction Work” has suggested that the term “functional equivalent” [39,40] should be used for stating functional properties of a building. This includes requirements on space, indoor climate, air quality, light, silence, etc.

The spatial boundary specifying the part of the physical building that is included in the assessment has also to be defined. The boundary between the building and its surroundings is not always clear, as a building interacts with the infrastructure, e.g., grids of energy support, telecommunications, sewage, roads, etc., and the ecosystem. The spatial extension of the assessed objects may range from building elements to neighborhoods.

There are also temporal boundaries related to the life cycle stages of the building components and the life cycle phases of the building process which has to be examined. Finally, the framework for analysis of BEATs highlights which environmental impacts and other impacts a tool considers.

Some of the main findings from Figure 2 are depicted in Table 3.

In LEED, the aggregation is an addition of the indirectly weighted scores (called points in the tool). In the Code for Sustainable Homes, the scores (called credits) are weighted into points. EcoEffect gives three final results: one for impacts on the external environment, one for impacts on the indoor internal environment, and one for the outdoor internal environment. A further analysis of the outdoor assessment is not included in this comparison. The external impacts include two categories (Emissions and Waste and Natural Resources), which are weighted. These categories together contain 9 issues, which are first normalized, then weighted. The 2 categories and 9 issues in the indoor environment are weighted on both category level and issue level.

The overview shows that LEED has a flat structure that is easy to understand, but the uneven distribution of scores needs explanation. CSH and especially EcoEffect look much more complex because of the weighting and normalization which adds several levels to the diagram.

The number of components (results, categories and issues) in the tools is fewer in CSH and EcoEffect than in LEED. Still CSH and EcoEffect look more complex. It was also more difficult to draw their diagrams.

In LEED, indirect weighting is performed by assigning a different maximum amount of scores to each issue, and in some cases by having several issues dealing with the same environmental aspect (e.g., one issue for 10% reused material plus one issue for 20% reused material). This means that all points have the same “environmental” value regarding contribution to the final result. It also means that the points are tradable, with the exception of a few mandatory aspects that have emphasized importance. Often there are optional ways to obtain a point. Normally 1–4 points are available per issue, except for two Energy indicators and two Sustainable Site indicators, where more points can be gained (19 for Optimization of Energy Performance and 7 for On-Site Renewable Energy). This aggregation system is easily understood, but has been criticized because, in LEED v2.2, the basis for the scores assigned was not explained and the environmental meaning of the final score was hazy [28]. Since then, the indirect weighting has been adjusted. The point value of the credit should now accurately reflect its potential to either mitigate the negative environmental impacts of a building, or promote positive impacts [44]. The weighting is based on both US EPA’s (United States Environmental Protection Agency) TRACI (Tool for the Reduction and Assessment of Chemical and Other Environmental Impacts) environmental impact categories and weightings devised under the auspices of NIST (National Institute of Standards and Technology), which compare the impact categories to each other and assign a relative importance to each. A tool to analyze how each LEED credit interacts with the list of impacts has then been used by the LEED Steering Committee to distribute the points in LEED 2009 [44].

In CSH, some credits are mandatory while most are tradable. Most issues assessed give a maximum of 4 credits, except the issues Dwelling Emission Rate and Environmental Impact of Materials, which can give up to 15 credits. Since the categories are weighted, every credit obtained in one category has a different value than a credit obtained in another category. The weighting factor is derived from a survey of international “experts” and a consultation with industry representatives. The CSH Technical Guide 2009 [27] only states that the weights are set “relative to their importance.” The weightings and environmental ranking of the materials in the Green Guide to Specification [43] “are based on Life Cycle Assessments (LCA), using BRE’s Environmental Profiles Methodology 2008” [45].

The basis for the assessment and weighting in EcoEffect is a life cycle approach. Equivalents for the external environmental issues (called impact categories) are first calculated. They are then normalized, i.e., divided by the corresponding value per capita in the country, in the end showing a percentage. For internal impact a scale with four steps (0–3, where low scores mean a good result) is applied, punishing ill health and discomfort. Recent standards or practices found in codes are used as reference values and given the same score. The scores for each issue are then weighted both on the issue level and on the category level. EcoEffect weights issues and categories according to their potential impact (damage) on people. The weighting system is based on a disability/discomfort scale developed as an extension of the DALY (Disability Adjusted Life Years) system [23]. It considers both short intense impacts and long mild impacts on humans. For each issue and impact category, weights have been established by estimating the potential harm the endpoint problems within each category might cause people [2]. For external impacts, the assessment is based on the environmental impact from the total amount of energy and materials used per square meter or per user, i.e., the number of users for which the building is designed. This favors efficient use of space, which is interesting from an environmental point of view as the positive effect of improved environmental building performance is counteracted by constructing larger buildings [46].

The significance of the weighting of different categories is one aspect of weighting that can be studied further. In many tools are the energy aspects considered most important [21]. Approximately one-third of the assessment scores in LEED and CSH are devoted to energy, and approximately one-fifth in EcoEffect. Together, the two most important categories in the tools represent more than 50% of the total available scores. This means that the lowest label in LEED and CSH can be achieved by meeting mandatory requirements and scoring high in two categories; Energy and Sustainable Site. [To receive the lowest label, LEED v3. demands a total result of 36% (40 points) and CSH a total score of 35% (36 credits)].

The framework for analysis is a method to enable analysis, understanding and comparison of BEATs, and, to some extent, it is also a proposal for a uniform terminology. This will aid structured analyses of BEATs which increase the transparency and thereby also the understanding of the tools. This will facilitate a reflective and critical discussion regarding their content and structure.

Using different BEATs may influence building design, building process, environmental performance and, the bottom line, environmental impact. That is why it is important to have an ongoing discussion regarding BEATs and their applicability and environmental relevance. We believe that the framework can play a role in this respect.

The differences between BEATs are unclear and give users the possibility of selecting a system which favors their own building or construction and manufacturing methods. The differences continue to cause confusion in the sector and on the market. According to [49], there seem to be strong incentives to make use of ambiguity rather than clarity in relation to “green building.” The interpretative flexibility of “green,” “environmental” or “sustainable” building with a plurality of meanings allows competing ideologies and special interests to gain rhetorical prominence and to influence decision makers’ views of what should be prioritized. This framework makes it easier to identify how BEATs are structured, what they contain, and where any differences are located.

The hierarchical diagram presents a new way to visualize the structure for a BEAT. It also increases the transparency of the tools which is important for the credibility of labeling systems. The assessment tool should “openly display how data are processed and the objectives ruling the processes,” according to [50]. Making the diagram takes time if the tools are complex, but it also saves time as it clarifies the content of the tool manuals which often contain hundreds of pages. The layout of current BEATs is relatively constant over time, which means that the mapping process seldom has to be repeated.

The framework facilitates classification of tools by a deepened and more thorough analysis of the tools. Berardi [21] presents three different types of assessment tools proposed by Hastings and Wall [51]: cumulative energy demand systems, life cycle analysis systems, and total quality assessment systems. This comparison of tools shows that there are large differences even between tools belonging to the same group, i.e., total quality assessment systems. This framework may therefore aid the work of further tool classification.

The framework identifies tool differences, not their consequences, when applied on certain buildings. If this is the question, it has to be investigated separately for these buildings. One reason why consequences of tool differences are difficult to assess is due to the use of tradable points. The tradable points make it possible for buildings with large differences to be assessed and get the same label.

From an environmental point of view, there is a risk that issues not included in a tool will not be considered at all. One such issue is embedded hazardous substances, which is left out in the three tested tools. If that issue is not considered, the sector will probably not put the same emphasis on a decreased use of hazardous substances. The statement “We measure what we care about and we care about what we measure” [52] highlights the importance of including all aspects that are important from an environmental point of view and also of not including impacts that are less important. Further investigations are needed in order to understand why certain issues are assessed in one system and not in another. Underlying conditions for including or excluding a certain issue are for example: the cost of the assessment, the context in which the tool is developed, local building code, perceived environmental issues, intelligibility, the possibility for clients to influence [37]. Other reasons could be cultural differences, measurability [53], available data, extensive documentation, specific/expensive equipment, time-consuming calculations, local climate and geographical conditions.

In a similar way, the significance of different environmental aspects can vary regionally. LEED has regional points as a way of emphasizing regional environmental aspects. Water saving is of higher priority in regions where water is a scarce resource. Regional aspects can always be added as extra criteria separated from the BEAT to highlight regional environmental aspects. Other features, such as regenerative design and resilience, as well as features apart from environmental aspects, can also be included in the tools. The inclusion of quality aspects and social aspects in CSH reflects this. However, it may blur what the final result represents.

In the framework, a new distinction is made between issues denoted as Procedures, Features and Performance. The different tools use these distinctive types to a varying extent. Many tools assess designed but not-yet-built buildings with demand for verification later on. At the design stage, the only available data that can be assessed are the results of design activities such as drawings, calculations, and descriptions, and they can be handled in different ways. Procedure issues can be used in this phase as one way to deal with this difficulty of assessing a building before it is built and to improve the likelihood of achieving a high environmental quality of the completed building. However, the assessment result also depends on how the procedure is evaluated and the result interpreted. Performance issues can represent the real environmental impact more direct. Aggregating a mixture of Procedures, Features and Performances can be questioned because it gives an ambiguous result. There is also a larger risk of double counting when issues are of more than one type and thereby overlap each other.

The weighting tends not to be something that is highlighted in the tools or in other comparisons of environmental assessment, even though the weighting dominates the final valuation of an assessed building. According to Ding [15], there is at present neither a consensus-based approach nor a satisfactory method to guide the assignment of weightings. Finding a systematic and relevant way to weight and aggregate the assessment results is a major problem in all BEATs [48]. Weighting can be made at different levels, i.e., between categories, issues or parameters, reflecting their relative significance.

The framework addresses aggregation and weighting as important things to analyze when comparing BEATs. If e.g., impacts of material production (Materials and Waste) contribute significantly to the total environmental impact of a building, it should be taken into account to the same extent. The weighting of the category should therefore correspond to the environmental importance of the materials in some way. However, this is not the fact in the three assessed methods. A simplification for comparing environmental impact of building materials is to consider the materials embodied energy [54] in relation to the energy use for operation. Research indicates that the embodied energy in materials represents 23–42% of the total energy use during a 50-year life cycle [54,55,56,57,58,59,60]. Other studies show that the energy use during occupancy is larger than the embodied energy and accounts for approximately 80% of a building’s total energy use [61,62,63]. In low-energy buildings, the relative impact from materials becomes larger [64,65,66]. From these studies, it follows that the relative impact from “Material and Waste” increases with decreasing energy use and also with decreasing life span. However, in the tools, the Material and Waste category is weighted rather low in all three tools. It makes up to less than 15% of the scores in LEED and CSH. In EcoEffect this relationship depends on the environmental impact of the materials used. The fact that embodied energy and environmental impacts from materials are not always accurately represented in variations of parameters, issues and scores in BEATs has been demonstrated in another study [67].