Next Article in Journal
Towards Credible City Branding Practices: How Do Iran’s Largest Cities Face Ecological Modernization?
Next Article in Special Issue
Failure Impact Assessment for Large-Scale Landslides Located Near Human Settlement: Case Study in Southern Taiwan
Previous Article in Journal
Identifying Farm Pond Habitat Suitability for the Common Moorhen (Gallinula chloropus): A Conservation-Perspective Approach
Previous Article in Special Issue
Flowchart on Choosing Optimal Method of Observing Transverse Dispersion Coefficient for Solute Transport in Open Channel Flow
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Sacrificial Pseudoreplication in LEED Cross-Certification Strategy Assessment: Sampling Structures

Department of Civil Engineering, Ariel University, Ariel 40700, Israel
Sustainability 2018, 10(5), 1353; https://doi.org/10.3390/su10051353
Submission received: 3 April 2018 / Revised: 20 April 2018 / Accepted: 25 April 2018 / Published: 27 April 2018

Abstract

:
The study aims to suggest sampling structures to avoid sacrificial pseudoreplication in the evaluation of Leadership in Energy and Environmental Design (LEED)-certified projects. The sampling includes two structures that exclude sacrificial pseudoreplication and one structure that leads to sacrificial pseudoreplication: (i) The state is the sampling frame in which LEED projects are treated as primary sampling units; (ii) The US is the sampling frame, the state is the primary sampling unit in which LEED projects are treated as evaluation units; and (iii) The US is the sampling frame in which LEED projects are pooled from different states and treated as primary sampling units. The three sampling structures are applied to the evaluation of the Silver-to-Gold cross-certification performances of LEEDv3 for new construction and LEEDv3 for existing buildings. The same cross-certification strategy was revealed if either structure (i) or structure (ii) was applied, while it was poorly estimated and misinterpreted if structure (iii) was applied, i.e., sacrificial pseudoreplication had occurred.

1. Introduction

We first start with statistical terminology necessary for explaining the study (Table 1).
“Pseudoreplication is a serious type of statistical error that is unfortunately common in all the sciences. It was originally defined in the context of manipulative experiments but can also occur in observational studies” [3]. Leadership in Energy and Environmental Design (LEED)-certified projects data analysis is an observational study [4]. Therefore, LEED project data analysis should be considered regarding the problem of pseudoreplication.
The term pseudoreplication was coined by Hurlbert [5] and defined as “… a particular combination of experimental design (or sampling) and statistical analysis which is inappropriate for testing the hypothesis of interest”. Pseudoreplication typically occurs when the number of observations or the number of data points are treated inappropriately as independent replicates [5,6,7]. Pseudoreplication contains several types: simple, temporal, and sacrificial. One of the most common types is sacrificial pseudoreplication [6]. This study focuses on the sacrificial pseudoreplication that arises when the evaluation units from different primary sampling units are pooled to obtain “[but not universal] … exaggeration of both the strength of the evidence for a true difference between treatments and of the precision with which any difference that does exist has been estimated” [2]. Recently, when Wu et al. [8] and Wu et al. [9] analyzed the LEED project data, they utilized pooling procedures that achieved extremely low P values. An evaluation of LEED project data and different sampling structures (without and with pooling) is discussed below.
A collection of Certified, Silver, Gold, and Platinum LEED-certified projects in the US Green Building Council (USGBC) website [10] is a unique database that has been used by researchers to perform retrospective analyses of the LEED-certified building certification, category, and cross-certification performances [4,8,9,11,12]. Such analyses produce feedback that can be a useful information for experts who oversee further improvements in the seven LEED categories such as sustainable sites (SS), water efficiency (WE), energy and atmosphere (EA), material and resources (MR), indoor environmental quality (EQ), innovation in design (ID), and regional priority (RP).
In three recent cases [4,8,9], a single-unit design structure of the analysis according to the term originally suggested by Hurlberd [7] (p. 652) was used. In these studies, all LEED projects were treated as statistically independent units using a non-parametric Wilcoxon–Mann–Whitney (WMW) test [13]. The WMW test was chosen, because LEED data are related to an ordinal measurement scale.
Wu et al. [8] pooled all LEED projects within the US to determine cross-certification performances of LEED-NCv2.2-certified projects in 2007–2015. Sample sizes for both Silver and Gold LEED projects were n1 = 1798 and n2 = 2469, respectively [8] (p. 174, Table 8). Based on the absolute effect size (mean credit increase, [MCI]) and statistical difference (p-values) of the pooled data, the authors concluded that at least three categories, EA (MSI = 2.72 and p = 0.000), EQ (MCI = 1.50 and p = 0.000), and SS (MCI = 1.12 and p = 0.000), were propelling categories for moving the projects from Silver to Gold certification [8] (p. 173, Table 6).
Wu et al. [9] recently carried out pooling of all LEED projects within the US to determine cross-certification performances of LEED-NCv3 projects in 2007–2015. Sample sizes for both Silver and Gold LEED-certified projects were n1 = 1310 and n2 = 1201, respectively [9] (p. 375, Table 9). Based on MCI and p-values of the pooled data, the authors concluded that at least three categories, EA (MCI = 5.46 and p = 0.000), SS (MCI = 2.57 and p = 0.000), and WE (MCI = 0.84 and p = 0.000), were propelling categories in order to move from Silver to Gold certification [9] (p. 375, Table 7).
In both studies [8,9], large sample sizes were used to determine the statistical difference via p-values. This can lead to statistical error, i.e., fallacy of large sample size (Hurlbert and Lombardi 2009). Hurlbert and Lombardi [14] allegorically noted: “it lurks quietly in the darkness, waiting for researchers to pass by who are too focused on obtaining adequate sample sizes. If sample sizes are too large, one may be ‘in danger’ of getting very low p-values and establishing the sign and magnitude of even small effects with too much confidence.”
In this respect, Pushkar and Verbitsky [4] more recently performed pooling of all LEED projects within individual states to determine cross-certification performances of Silver-Gold LEED-NCv3 certificated projects in 10 US states (CA, n1 = 42, n2 = 58; FL, n1 = 31, n2 = 11; GA, n1 = 11, n2 = 9; IL, n1 = 13, n2 = 19; MA, n1 = 19, n2 = 14; NY, n1 = 14, n2 = 11; OH, n1 = 23, n2 = 8; TX, n1 = 27, n2 = 9; VA, n1 = 38, n2 = 11; WA, n1 = 10, n2 = 11) in 2016. It should be noted that because LEED data is ordinal data, the median credit increase (MdCI) in the Pushkar and Verbitsky [4] was used instead of the mean credit increase (MCI) that was suggested in the Wu et al. [9] study. In particular, the EA category was propelled in six of 10 states (CA [MdCI = 10.0 and p = 0.01); FL [MdCI = 5.0 and p = 0.016]; IL [MdCI = 7.0 and p = 0.004]; MA [MdCI = 2.5 and p = 0.001]; VA [MdCI = 5.0 and p = 0.001]; and WA [MdCI = 8 and p = 0.001]; SS was propelling category in two of 10 states (WA [MdCI = 4.5 and p = 0.001]; OH [MdCI = 4.5 and p = 0.001]), and the WE was a propelling category in one of 10 states (CA [MdCI = 2.0 and p = 0.013) [4]. Thus, in the Pushkar and Verbitsky [4] study, results of EA, SS, and WE were somewhat contradictory to the results revealed in the Wu et al. [9] study.
One possible reason for such contradictory results is that the different US states have adopted different green and energy codes along with their different versions [15,16]. For example, one of the main energy regulations, ASHRAE 90.1 Standard (Energy Standard for Buildings Except Low-Rise Residential Buildings), is not under national regulation. Hence, states are willing to adopt any version of the ASHRAE 90.1; in this way, different ASHRAE 90.1 versions are adopted in different US states [15]. However, these different ASHRAE 90.1 versions that are accepted in different US states can have different influences on one of the highly weighted LEED categories, EA [4]. In addition, ASHRAE 90.1 [17] version has usually moved dynamically through the years in the same state. For example, in CA: 2011—ASHRAE 90.1-2007, 2014—ASHRAE 90.1-2010, and 2017—ASHRAE 90.1-2013; and in OH: 2011—ASHRAE 90.1-2004, 2014—ASHRAE 90.1-2007, and 2017—ASHRAE 90.1-2010 [17]. Thus, an adopted ASHRAE 90.1 version can be changed one time in three years in the same state, while typical duration of the building construction is approximately two years. For this reason, to minimize the evaluation of LEED projects in the same state with different ASHRAE 90.1 versions, we propose that the time collection of the LEED projects must not exceed one year. Therefore, due to both the spatial and temporal influences of ASHRAE 90.1 energy code on EA category, LEED projects certified in different years cannot be pooled in one US group as suggested by Wu et al. [8] and Wu et al. [9].
In the context of statistical terminology, a single-unit design allows three methods of analysis: (i) If a sampling frame contains primary sampling units, and the primary sampling unit does not contain evaluation units, then primary sampling units are treated as statistically independent units; (ii) If a sampling frame contains primary sampling units, and a primary sampling unit contains evaluation units, then evaluation units must be averaged within each primary sampling unit, and then primary sampling units are treated as statistically independent units. However, (iii) If a sampling frame contains primary sampling units that contain evaluation units, and evaluation units are pooled from two or more primary sampling units and treated as primary sampling units, it will lead to the problem of sacrificial pseudoreplication [5]. Ignoring sampling structure of the collected data leads to sacrificial pseudoreplication and can lead to “artificially inflated degrees of freedom, giving the illusion of having a more powerful test than the data support” [1].
In our case, pooling LEED projects from different states of the US and treating them as primary sampling units can lead to erroneous conclusions about cross-certification strategies [4]. Thus, based on the Kozlov and Hurlbert [18] study, to avoid erroneous conclusions about cross-certification strategies, the following logical structure should be accepted: any two LEED projects in the same state share more similar “green building policy” conditions compared to any two LEED projects from the different states. It is suggested that states in the US are “natural” groups or clusters. If a single-design structure in the present study is used, then the primary sampling units are only used in the inferential statistical analysis.
The goal of the present study is to correct the statistical malaise over evaluating and interpreting the LEED certification process. Correcting statistical analysis included two possible evaluation structures: (i) individual state sampling frame analysis and (ii) the US sampling frame analysis. These two structures were contrasted to a third evaluation structure: (iii) sacrificial pseudoreplication analysis.
Application of the three evaluation levels was demonstrated on a case study of LEED-NCv3 and LEED-EBv3 projects certified in four US states in 2016. For both certification schemes, only the most popular Silver and Gold certifications [4] were considered. As a result, the cross-certification analyses were performed so as to reveal which LEED categories are Silver-to-Gold propelling. This will serve as valuable information for understanding the strategies employed by green building practitioners in moving projects toward higher certification level.

2. Materials and Methods

2.1. Design of the Study

Only Silver and Gold projects of LEED-NCv3 and LEED-EBv3 in the US in 2016 were considered for the analysis, which was performed in the following three consequent steps:
  • In this analysis, each of the three structures was evaluated using either approximate or exact WMW tests: (i) individual state sampling frame analysis used an approximate WMW test, (ii) the US sampling frame analysis used an exact WMW test, and (iii) sacrificial pseudoreplication analysis used an approximate WMW test. To perform an approximate WMW test [11], the balanced sample sizes for Silver (n1) and for Gold (n2) projects were (n1 = n2 ≥ 9). For states with suitable sample sizes for LEED projects, an approximate WMW test was used. Then, LEED projects from each state were randomly selected. To perform an exact WMW test, minimum sample size was (n1 = n2 = 4) [19] (p. 19). To obtain primary sampling units, the median Silver and Gold project in each state was computed and collected.
  • USGBC scorecards of the randomly selected projects were retrieved from the USGBC website new construction building directory [20] and existing building directory [21]. Then, information on the retrieved projects regarding the awarded points in the six main categories (SS, WE, EA, MR, EQ, and ID) was accumulated. Eventually the RP points were also accumulated and redistributed among the five relevant main categories.
  • For each of the three structures, (i) individual state sampling frame analysis (the state is sampling frame in which LEED projects are treated as primary sampling units), (ii) the US sampling frame analysis (the US is the sampling frame, the state is the primary sampling unit in which LEED projects were treated as evaluation units), and (iii) sacrificial pseudoreplication analysis (the US is the sampling frame in which LEED projects were pooled from different states and treated as primary sampling units), robust statistical analysis for comparison between the randomly selected Silver- and Gold-certified projects in ordinal scale was performed.

2.2. Statistical Analysis

LEED data are presented in ordinal scales. Based on that fact, the following nonparametric tests, Cliff’s δ [22] and WMW test [13], were used to compare the two unpaired groups. The data are presented as the median ± interquartile range (IQR, 25th–75th percentile).
Cliff’s δ was used to measure the substantive significance (effect size) between two unpaired groups. Cliff’s δ [22] (p. 495) is expressed as
δ = #(x1 > x2) − #(x1 < x2)/(n1n2)
in which x1 and x2 are scores within group 1 and group 2, respectively; n1 and n2 are the sizes of the sample groups, group 1 and group 2; and # indicates the number of times.
Cliff’s δ ranges between −1 and + 1; positive (+) values indicate that group 1 is larger than group 2, 0 indicates equality or overlap, and negative (−) values indicate that group 2 is larger than group 1 [22].
The effect size is considered (i) negligible if |δ| < 0.147, (ii) small if 0.147 ≤ |δ| < 0.33, (iii) medium if 0.33 ≤ |δ| < 0.474, or (iv) large if |δ| ≥ 0.474 [23]. According to Cohen [24] (p. 156), “a medium effect is visible to the naked eye of a careful observer. A small effect is noticeably smaller than medium but not so small as to be trivial. A large effect is the same distance above the medium as small is below it.” It should be noted that the effect size is not “iron-clad criteria” [25] but is only a general rule of thumb that might be followed in the absence of knowledge of the area [26].
WMW test. The WMW test was used to determine statistical difference (p-value) between two unpaired groups. It should be noted that WMW tests could be applied in two forms: approximate form or extract form [27]. If sample sizes were n1 = n2 ≥ 9, then an approximate WMW test was used [28] (p. 56). Mann and Whitney [13] (p. 50) noted that when sample sizes achieve n1 = n2 = 8, then “at this point the distribution is almost normal”. If sample sizes were n1 = n2 = 4, an exact WMW test was used [19] (p. 19). In both tests, a two-tailed p-value was applied. Only balanced sample size cases were studied.
Neo-Fisherian significance assessments. In the current study, for standard types of significance assessment, the hybrid of the Paleo–Fisherian and Neyman–Pearsonian paradigms [i.e., null hypothesis significance tests (NHST)] are replaced by a neo-Fisherian assessment, as recommended by Hurlbert and Lombardi [14,29]. The neo-Fisherian paradigm (1) does not fix α, (2) does not describe p-values as ‘significant’ or ‘nonsignificant’, (3) does not accept null hypotheses based on high p-values but only suspends judgment, (4) interprets significance tests according to “three-valued logic”, and (5) presents effect size information in conjunction with significance tests. Analyses conducted under this paradigm are termed neo-Fisherian significance assessments (NFSAs) [14]. NFSAs are used to interpret the signs and magnitudes of the statistical effects [14]. Based on NFSAs, precise P-values were evaluated and shown according to a three-valued logic as follows: “it seems to be positive” (i.e., there seems to be a difference between group 1 and group 2), “it seems to be negative” (i.e., there does not seem to be a difference between group 1 and group 2), and “judgment is suspended” regarding the difference between group 1 and group 2 [14,29].

3. Results

The number of the selected states with the total number of LEED projects and number of randomly selected LEED projects is presented in Table 2. Ordinal font size numbers show the total number of LEED’s projects. Numbers in parentheses display randomly selected LEED’s projects to perform the balanced non-parametric statistical analysis. The resulting state number with Silver- and Gold-certified projects was n1states = n2states = 4 and included the states of California (CA), Washington (WA), Virginia (VA), and Texas (TX) (Table 2). It was sufficient to perform an exact WMW test in the (ii) US sampling frame analysis [19]. The number of Silver- and Gold-selected projects n1projects = n2projects was different for each of the four states, but it was sufficient to perform an approximate WMW test in (i) individual state sampling frame analysis and (iii) sacrificial pseudoreplication analysis.
Cross-certification performances, namely, from Silver to Gold certification level in both the LEED-NCv3 and LEED-EBv3 in the four states, CA, VA, WA, and TX, were statistically evaluated under (i) individual state sampling frame analysis (an approximate WMW test), (ii) the US sampling frame analysis (an exact WMW test), and (iii) sacrificial pseudoreplication (an approximate WMW test).

3.1. LEED-NC 2009 Certified Projects

The individual state sampling frame analysis. Strategies of each of the four states toward moving from Silver to Gold for LEED-NCv3 are presented in Table 3. According to these strategies, the EA is the most accepted propelling category, which was involved in all four states. These results confirm the popularity of the EA propelling category in cross-certification performance of LEED-NCv3-certified projects, which was revealed by other studies [4,9]. In particular, Pushkar and Verbitsky [4], who analyzed LEED-NCv3 certified projects in 2016, revealed that EA was the most propelling category in Florida (FL), Illinois (IL), and Massachusetts (MA), while EQ was the most propelling category in New York (NY).
SS, EQ, and ID are moderately accepted propelling categories (Table 3); these were involved in two of the four states. Again, the SS and EQ popularity for Silver–Gold moving was revealed early by Wu et al. [9], who outlined SS as one of the main propelling categories and by Pushkar and Verbitsky [4], who named both SS and EQ as the moderate propelling categories. ID was also reported as a somewhat moderate propelling category for Silver–Gold cross-certification performance [9].
According to the present research, WE and MR were not involved in any of the four states in Silver–Gold performance (Table 3). These two categories were noted as the worst-performing categories in LEED-NCv3 cross-certification performance [4,9]. As was explained by Wu et al. [9], there is a difficulty in accessing MR points due to a low possibility of reducing construction material consumption.
The US sampling frame analysis. Total tendency in the strategy employed by the four states applying the US sampling frame analysis toward moving from Silver to Gold for LEED-NCv3 is presented in Table 4. According to the revealed strategies, the EA and EQ were the most accepted propelling categories, SS and ID were the moderately accepted propelling categories, and WE and MR were not accepted propelling categories at all. In general, the results of the US sampling frame analysis (Table 4) match the results of the individual state sampling frame analysis (Table 3). The only exception is the EQ category. In the individual state sampling frame analysis (Table 3), EQ was grouped with the moderately accepted propelling categories (SS, EQ, and ID), and in the US sampling frame analysis (Table 4), it was grouped with the most accepted propelling categories (EA and EQ).
The sacrificial pseudoreplication analysis. Total tendency in strategy employed by the four states on the basis of the analysis when projects from different states were pooled in one US sampling frame for LEED-NCv3 is presented in Table 5. According to these revealed strategies, the SS, EA, EQ, and ID were the most accepted propelling categories, while WE and MR were not accepted propelling categories at all. In general, the results revealed that when projects from different states were pooled in one US sampling frame (Table 5), the results evaluated in the individual state sampling frame analysis were different (Table 3). The differently evaluated results were for SS, EQ, and ID categories: in the individual state sampling frame analysis (Table 3), these three categories were grouped under the moderately accepted propelling categories (SS, EQ, and ID), and in the analysis where different states were pooled in one US sampling frame (Table 5), these categories were grouped under the most accepted propelling categories (SS, EA, EQ, and ID).

3.2. LEED-EB 2009 Certified Projects

The individual state sampling frame analysis. Strategies of each of the four states toward moving from Silver to Gold for LEED-EB 2009 are presented in Table 6. According to the results, all six categories were propelling: SS and EQ were the most accepted, and WE, EA, MR, and IO were moderately accepted propelling categories. Thus, in contrast to unemployed WE and MR in a case of new projects certified under LEED-NCv3 (Table 3), these categories were active in a case of renovated buildings certified under LEED-EBv3 (Table 6). It can be suggested that site and building restrictions are weaker in new building design and construction than in the renovation of existing buildings. Therefore, in selecting preferred strategy design, a team has more flexibility in new buildings than in renovated buildings. As a result, in new building certification, difficult points such as MR points were not considered as a preferred Silver-Gold cross-certification strategy (Table 3), while in renovated buildings, all possible strategies were employed (Table 6).
The US sampling frame analysis. Total tendency in strategy employed by the four states applying the US sampling frame analysis toward moving from Silver to Gold for LEED-EBv3 is presented in Table 7. According to the revealed strategies, the SS and EQ were the most accepted propelling categories; WE, EA, MR, and ID were the moderately accepted propelling categories. The US sampling frame analysis results (Table 7) were the same as the results evaluated with the individual state sampling frame analysis (Table 6).
It is interesting that EQ was one of the most propelling categories in both new buildings (certified under LEED-NCv3) and renovated buildings (certified under LEED-EBv3). However, an additional most propelling category was different in these schemes: EA—in LEED-NCv3, and SS—in LEED-EBv3. It should be noted that SS and EA have the same number of achievable points, 26 pt and 35 pt, respectively, in both LEED-NCv3 and LEED-EBv3 [30,31]. However, there are differences in some inherent credits between LEED-NCv3 and LEED-EBv3 [30,31]. For example, the SS category in LEED-NC 2009 [30] includes four separate rigid sub-credits for SSc4 Alternative Transportation (12 pt) (SSc4.1 Public Transportation Access, SSc4.2 Bicycle Storage & Changing Rooms, SSc4.3 Low-Emitting & Fuel-Efficient Vehicles, and SSc4.4 Parking Capacity). However, the SS category in LEED-EB 2009 [31] includes the one flexible SSc4 Alternative Commuting Transportation credit (15 pt), which covers the issue of reduction in regular conventional commuting trips (when individual conventional automobiles are replaced with any other human-powered conveyances, for example, mass transit, walking, or bicycles). SSc4 credit is more flexible and more point-accounted in LEED-EB 2009 [31], but it is more rigid and less point-accounted in LEED-NC 2009 [30]. This is possibly the reason why SS was one of the most propelling categories in LEED-EB 2009 [31] but not in LEED-NC 2009 [30].
The sacrificial pseudoreplication analysis. Total tendency in strategy employed by the four states on basis of the analysis when projects from different states were pooled in one US sampling frame for LEED-EBv3 is presented in Table 8. According to the revealed strategies, the SS, EA, MR, EQ, and IO were the most accepted propelling categories, while WE was the moderately propelling category. The results of different pooled states in one US sampling frame (Table 4c) were not reflected in the results evaluated with the individual state sampling frame analysis, in which only SS and EQ were the most propelling categories (Table 6).
It is interesting that the most propelling categories in both new buildings (the SS, EA, EQ, and ID) and renovated buildings (the SS, EA, MR, EQ, and IO) were almost the same. There was only exception: MR category. MR was not revealed as a propelled category in LEED-NCv3 certified projects (Table 5), while it was revealed as a propelled category in LEED-EBv3 certified projects (Table 8). However, both of these results should be considered as false, due to deliberately applied wrong analysis that could lead to sacrificial pseudoreplication. Perhaps, this is why these results that are closer to the results reported by Wu et al. [8], who concluded that SS, EA, and EQ were revealed as propelled categories from Silver to Gold for LEED-NCv2.2 certified projects. It can be suggested that such similarity in results was revealed because the same method for statistical evaluation was used here, in the sacrificial pseudoreplication analysis sub-section, and in the study of Wu et al. [8].

4. Conclusions

Silver to Gold cross-certification in both the LEED-NCv3 and LEED-EBv3 projects in the four states (CA, VA, WA, and TX) was statistically evaluated applying (i) the individual state sampling frame analysis, (ii) the US sampling frame analysis, and (iii) the sacrificial pseudoreplication analysis. The following was concluded:
  • The individual state sampling frame analysis. Different cross-certification strategies were revealed in LEED-NCv3 and LEED-EBv3-certified projects. For newly constructed projects certified under LEED-NCv3, four of the six categories were employed, in which EA was the most popular propelling category; SS, EQ, and ID were the intermediately popular propelling categories; and WE and MR were completely unpopular categories. For renovated projects certified under LEED-EBv3, all categories were employed, in which SS and EQ were the most popular propelling categories; WE and EA were the intermediately popular propelling categories; and MR and IO were the least popular propelling categories. Thus, in new versions of LEED-NC and LEED-EB, experts should encourage building practitioners to also focus on currently less popular categories toward more equal high achievements in all of the five main environmental categories.
  • The US sampling frame analysis. For both LEED-NCv3- and LEED-EBv3-certified projects, the cross-certification strategy revealed in (ii) the US sampling frame analysis was the same as the strategy revealed in (i) the individual state sampling frame analysis. This means that the US sampling frame (a median LEED project in each individual state) analysis can be recommended as an appropriate test for revealing total tendency in the LEED strategy of both new and renewed projects.
  • The sacrificial pseudoreplication analysis. For both the LEED-NCv3- and LEED-EBv3-certified projects, the cross-certification strategy revealed that when projects from different states were pooled in one US sampling frame (the sacrificial pseudoreplication analysis), the strategy revealed on the individual state sampling frame analysis was different. This means that the sacrificial pseudoreplication analysis cannot be recommended as an appropriate test for revealing total tendency in LEED category strategy of both new and renewed projects.

5. Implications of This Study

This study outlines the importance of correct applied statistical methods based on the individual state sampling frame analysis as opposed to the incorrect one based on the US sampling frame through revealed building practice in certification under LEED-NCv3 and LEED-EBv3. The results of this individual state-prevailing trend can help LEED researchers to further correct “a serious type of statistical error” [2] in current evaluations of LEED certified projects. As a result, more realistic feedback from LEED certification can help LEED experts in further versions of rating schemes to move towards more sustainable building.

Funding

This research received no external funding.

Acknowledgments

The author is grateful to Oleg Verbitsky for his help with the statistical analysis and to three anonymous reviewers for their helpful comments.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Picquelle, S.J.; Mier, K.L. A practical guide to statistical methods for comparing means from two-stage sampling. Fish. Res. 2011, 107, 1–13. [Google Scholar] [CrossRef]
  2. Hurlbert, S.H. The ancient black art and transdisciplinary extent of pseudoreplication. J. Comp. Psychol. 2009, 123, 434–443. [Google Scholar] [CrossRef] [PubMed]
  3. Hurlbert, S.H.; Lombardi, C.M. Research methodology: Experimental design sampling design, statistical analysis. In Encylopedia of Animal Behavior; Bekoff, M.M., Ed.; Greenwood Press: London, UK, 2004; Volume 2, pp. 755–762. [Google Scholar]
  4. Pushkar, S.; Verbitsky, O. LEED-NCv3 silver and gold certified projects in the US: An observational study. J. Green Build. 2018, 13. in press. [Google Scholar]
  5. Hurlbert, S.H. Pseudoreplication and the Design of Ecological Field Experiments. Ecol. Monogr. 1984, 54, 187–211. [Google Scholar] [CrossRef]
  6. Hurlbert, S.H. On misinterpretations of pseudoreplication and related matters: A reply to Oksanen. Oikos 2004, 104, 591–597. [Google Scholar] [CrossRef]
  7. Hurlbert, S.H. Pseudofactorialism, response structures and collective responsibility. Austral Ecol. 2013, 38, 646–663. [Google Scholar] [CrossRef]
  8. Wu, P.; Mao, C.; Wang, J.; Song, Y.Z.; Wang, X.Y. A decade review of the credits obtained by LEED v2.2 certified green building projects. Build. Environ. 2016, 102, 167–178. [Google Scholar] [CrossRef]
  9. Wu, P.; Song, Y.; Shou, W.; Chi, H.; Chong, H.Y.; Sutrisna, M. A comprehensive analysis of the credits obtained by LEED 2009 certified green buildings. Renew. Sustain. Energy Rev. 2017, 68 (Pt 1), 370–379. [Google Scholar] [CrossRef]
  10. USGBC Website. Available online: http://www.usgbc.org/projects/ (accessed on 21 October 2017).
  11. Fuerst, F. Building momentum: An analysis of investment trends in LEED and Energy Star-certified properties. J. Retail Leis. Prop. 2009, 8, 285–297. [Google Scholar] [CrossRef]
  12. Ma, J.; Cheng, J.C.P. Data-driven study on the achievement of LEED credits using percentage of average score and association rule analysis. Build. Environ. 2016, 98, 121–132. [Google Scholar] [CrossRef]
  13. Mann, H.B.; Whitney, D.R. On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 1947, 18, 50–60. [Google Scholar] [CrossRef]
  14. Hurlbert, S.H.; Lombardi, C.M. Final collapse of the Neyman-Pearson decision theoretic framework and rise of the neoFisherian. Ann. Zool. Fenn. 2009, 46, 311–349. [Google Scholar] [CrossRef]
  15. Sun, X.; Brown, M.A.; Cox, M.; Jackson, R. Mandating better buildings: A global review of building codes and prospects for improvement in the United States. Wiley Interdiscip. Rev. Energy Environ. 2016, 5, 188–215. [Google Scholar] [CrossRef]
  16. Hsieh, L.-H.C.; Noonan, D. Strategic Behavior in Certifying Green Buildings: An Inquiry of the Non-building Performance Value. Environ. Manag. 2017, 60, 231–242. [Google Scholar] [CrossRef] [PubMed]
  17. ASHRAE 90.1. Commercial Adoption, States that Have Adopted a Statewide Commercial Energy Code that Meets or Exceeds an ASHRAE 90.1 Standard, Commercial Code Status. Available online: http://bcapcodes.org/code-status/commercial/ (accessed on 15 September 2017).
  18. Kozlov, M.; Hurlbert, S.H. Pseudoreplication, chatter, and the international nature of science: A response to D.V.Tatarnikov. J. Fundam. Biol. (Moscow) 2006, 67, 145–152. [Google Scholar]
  19. Fay, M.P.; Proschan, M.A. Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat. Surv. 2010, 4, 1–39. [Google Scholar] [CrossRef] [PubMed]
  20. USGBC. LEED for New Construction Projects Directory. Available online: http://www.usgbc.org/projects/new-construction (accessed on 10 October 2017).
  21. USGBC. LEED for Existing Building Projects Directory. Available online: https://www.usgbc.org/projects/existing-buildings (accessed on 10 October 2017).
  22. Cliff, N. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol. Bull. 1993, 114, 494–509. [Google Scholar] [CrossRef]
  23. Zhang, F.; Mockus, A.; Keivanloo, I.; Zou, Y. Towards building a universal defect prediction model with rank transformed predictors. Empir. Softw. Eng. 2016, 21, 2107–2145. [Google Scholar] [CrossRef]
  24. Cohen, J. A power primer. Psychol. Bull. 1992, 112, 155–159. [Google Scholar] [CrossRef] [PubMed]
  25. Durlak, J.A. How to select, calculate, and interpret effect sizes. J. Pediatr. Psychol. 2009, 34, 917–928. [Google Scholar] [CrossRef] [PubMed]
  26. Volker, M.A. Reporting effect size estimates in school psychology research. Psychol. Schools 2006, 43, 653–672. [Google Scholar] [CrossRef]
  27. Fagerland, M.W.; Sandvik, L. The Wilcoxon–Mann–Whitney test under scrutiny. Stat. Med. 2009, 28, 1487–1497. [Google Scholar] [CrossRef] [PubMed]
  28. Marx, A.; Backes, C.; Meese, E.; Lenhof, H.P.; Keller, A. EDISON-WMW: Exact dynamic programing solution of the Wilcoxon–Mann–Whitney test. GPB 2016, 14, 55–61. [Google Scholar] [CrossRef] [PubMed]
  29. Hurlbert, S.H.; Lombardi, C.M. Lopsided reasoning on lopsided tests and multiple comparisons. Aust. N. Z. J. Stat. 2012, 54, 23–42. [Google Scholar] [CrossRef]
  30. LEED-NC 2009. LEED for New Construction and Major Renovations; U.S. Green Building Council: Washington, DC, USA, 2009. [Google Scholar]
  31. LEED-EB 2009. LEED for Existing Buildings: Operations and Maintenance; U.S. Green Building Council: Washington, DC, USA, 2009. [Google Scholar]
Table 1. Statistical terminology.
Table 1. Statistical terminology.
TerminologyReference
The sampling frame is a collection of primary sampling units “accessible for sampling in the population of interest”[1]
The primary sampling unit is statistically independent of other primary sampling units within the same sampling frame[1]
Evaluation units are nested in the primary sampling unit and are statistically dependent of those primary sampling unit[1]
Simple pseudoreplication occurs when multiple measurements are performed on individual primary sampling units and treated statistically as if each represents a separate primary sampling unit[2]
Temporal pseudoreplication occurs when performing on individual primary sampling units multiple measurements, which are performed over time and treated statistically as if each represents a different primary sampling unit[2]
Sacrificial pseudoreplication occurs when the evaluation units from different primary sampling units are pooled and each evaluation unit is treated as a primary sampling unit[2]
Table 2. The distribution of LEED-NCv3 and LEED-EBv3 Silver- and Gold-certified projects within four US states in 2016.
Table 2. The distribution of LEED-NCv3 and LEED-EBv3 Silver- and Gold-certified projects within four US states in 2016.
StateLEED-NCv3: Sample Size (n)LEED-EBv3: Sample Size (n)
SilverGoldSilverGold
California (CA)42 (12)58 (12)24 (12)81 (12)
Virginia (VA)38 (11)11 (11)12 (12)12 (12)
Washington (WA)10 (10)11 (10)11 (9)9 (9)
Texas (TX)27 (9)9 (9)12 (10)10 (10)
Table 3. The individual state-sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
Table 3. The individual state-sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
LEED CategoryStateSample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SSCA1218.5 ± 5.522.5 ± 5.5−0.3470.157suspended
VA1118.0 ± 4.821.0 ± 6.8−0.2730.293negative
WA1018.5 ± 2.023.0 ± 3.0−0.8500.001positive
TX915.0 ± 7.717.0 ± 7.3−0.1980.507negative
WECA125.0 ± 1.05.0 ± 2.0−0.1880.451negative
VA116.0 ± 3.57.0 ± 3.5−0.0500.869negative
WA105.0 ± 2.04.5 ± 1.00.0600.850negative
TX96.0 ± 2.76.0 ± 3.5−0.2470.401negative
EACA129.0 ± 4.517.5 ± 10.0−0.7570.002positive
VA1110.0 ± 3.815.0 ± 6.5−0.7850.002positive
WA108.5 ± 5.016.0 ± 2.0−0.9300.0005positive
TX912.0 ± 10.314.0 ± 5.0−0.2960.310negative
MRCA125.0 ± 1.05.5 ± 2.0−0.0970.707negative
VA117.0 ± 1.87.0 ± 2.00.0250.948negative
WA105.0 ± 1.05.0 ± 2.00.1800.519negative
TX96.0 ± 3.07.0 ± 1.5−0.0740.825negative
EQCA129.5 ± 3.510.5 ± 3.5−0.0140.977negative
VA119.0 ± 3.810.0 ± 3.0−0.1240.646negative
WA109.5 ± 2.010.0 ± 2.0−0.4300.112suspended
TX99.0 ± 1.012.0 ± 1.8−0.6540.022positive
IDCA123.5 ± 3.05.0 ± 1.0−0.3960.105suspended
VA114.0 ± 3.05.0 ± 1.8−0.3550.167suspended
WA105.0 ± 2.04.5 ± 2.00.0800.791negative
TX94.0 ± 2.35.0 ± 1.3−0.3210.268negative
Table 4. The US sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
Table 4. The US sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
LEED CategorySample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SS418.3 ± 2.021.8 ± 3.8−0.6250.171suspended
WE45.5 ± 1.05.5 ± 1.80.0001.000negative
EA49.5 ± 2.315.5 ± 2.3−1.0000.029positive
MR45.5 ± 1.56.3 ± 1.8−0.2500.742negative
EQ49.3 ± 0.510.3 ± 1.3−1.0000.029positive
ID44.0 ± 0.85.0 ± 0.3−0.6880.142suspended
Table 5. The sacrificial pseudoreplication analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
Table 5. The sacrificial pseudoreplication analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
LEED CategorySample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SS4218.0 ± 5.021.5 ± 7.0−0.3750.003positive
WE425.0 ± 3.05.0 ± 2.0−0.1030.420negative
EA429.5 ± 5.016.0 ± 5.0−0.7090.00000002positive
MR426.0 ± 2.06.0 ± 2.00.0350.788negative
EQ429.0 ± 3.010.5 ± 3.0−0.2740.031positive
ID424.0 ± 2.05.0 ± 2.0−0.2960.020positive
Table 6. The individual state-sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-EBv3 in four US states.
Table 6. The individual state-sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-EBv3 in four US states.
LEED CategoryStateSample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SSCA1212.0 ± 6.515.0 ± 5.5−0.4240.083suspended
VA127.5 ± 6.014.0 ± 6.0−0.6600.007positive
WA911.0 ± 3.818.0 ± 5.0−0.9880.0005positive
TX1011.0 ± 2.016.0 ± 5.0−0.5900.028positive
WECA126.0 ± 4.59.0 ± 3.0−0.3680.133suspended
VA129.0 ± 7.08.5 ± 2.00.2990.225negative
WA97.0 ± 2.58.0 ± 1.5−0.3700.199suspended
TX106.0 ± 2.08.0 ± 3.0−0.6900.010positive
EACA1218.5 ± 7.522.5 ± 4.0−0.6040.013positive
VA1219.0 ± 5.022.0 ± 5.5−0.4790.049positive
WA919.0 ± 9.818.0 ± 5.00.1730.566negative
TX1019.0 ± 6.021.0 ± 7.0−0.3200.241negative
MRCA123.5 ± 1.54.5 ± 1.0−0.3470.156suspended
VA123.0 ± 2.04.0 ± 1.0−0.4440.068suspended
WA95.0 ± 1.36.0 ± 2.0−0.3090.288negative
TX103.0 ± 2.03.5 ± 1.0−0.2400.383negative
EQCA127.5 ± 2.08.0 ± 1.50.0560.840negative
VA126.5 ± 3.08.5 ± 2.0−0.5900.015positive
WA96.0 ± 1.38.0 ± 3.0−0.4200.145suspended
TX107.0 ± 2.08.5 ± 2.0−0.5900.028positive
IOCA126.0 ± 0.56.0 ± 0.5−0.0210.953negative
VA125.0 ± 1.56.0 ± 0.5−0.5830.016positive
WA94.0 ± 1.36.0 ± 2.0−0.3210.267negative
TX106.0 ± 1.06.0 ± 1.0−0.0300.939negative
Table 7. The US sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-EBv3 in four US states.
Table 7. The US sampling frame analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-EBv3 in four US states.
LEED CategorySample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SS411.0 ± 2.315.5 ± 2.5−1.0000.029positive
WE46.5 ± 2.08.3 ± 0.8−0.5630.286suspended
EA419.0 ± 0.321.5 ± 2.8−0.5000.314suspended
MR43.3 ± 1.34.3 ± 1.5−0.5630.257suspended
EQ46.8 ± 1.08.3 ± 0.5−1.0000.029positive
IO45.5 ± 1.56.0 ± 0.0−0.5000.429suspended
Table 8. The sacrificial pseudoreplication analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
Table 8. The sacrificial pseudoreplication analysis: median ± interquartile range (IQR, 25th–75th percentile) of six categories of LEED-NCv3 in four US states.
LEED CategorySample Size; n1 = n2Certified ProjectsCliff’s, δp-ValueNFSAs
SilverGold
SS4311.0 ± 5.816.0 ± 5.8−0.6390.0000003positive
WE437.0 ± 4.08.0 ± 2.0−0.2330.063suspended
EA4319.0 ± 6.022.0 ± 5.8−0.3020.016positive
MR433.0 ± 1.84.0 ± 1.8−0.2990.017positive
EQ437.0 ± 2.08.0 ± 2.0−0.3810.002positive
IO435.0 ± 2.05.0 ± 2.0−0.2460.050positive

Share and Cite

MDPI and ACS Style

Pushkar, S. Sacrificial Pseudoreplication in LEED Cross-Certification Strategy Assessment: Sampling Structures. Sustainability 2018, 10, 1353. https://doi.org/10.3390/su10051353

AMA Style

Pushkar S. Sacrificial Pseudoreplication in LEED Cross-Certification Strategy Assessment: Sampling Structures. Sustainability. 2018; 10(5):1353. https://doi.org/10.3390/su10051353

Chicago/Turabian Style

Pushkar, Svetlana. 2018. "Sacrificial Pseudoreplication in LEED Cross-Certification Strategy Assessment: Sampling Structures" Sustainability 10, no. 5: 1353. https://doi.org/10.3390/su10051353

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop