# Quality Control of “As Built” BIM Datasets Using the ISO 19157 Framework and a Multiple Hypothesis Testing Method Based on Proportions

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. BIM Data Quality Elements

- Completeness of data: This category (DQ_Completeness) refers to the presence and absence of objects, their attributes, and relationships. Lack of completeness is important when working with data that reflect reality. For example, a door or window cannot be missing in BIM data. In this case, two DQEs can be considered:
- ○
- Commission: The presence of excess data within the BIM Data. This means that some objects appearing in the BIM data do not exist in the real world.
- ○
- Omission: The absence of certain data within the BIM data. This means that some objects not included in the BIM model exist in the real world.

- Metric accuracy: This category name does not appear in ISO 19157, where it appears instead as positional accuracy (DQ_PositionalAccuracy). Our current proposal is broad, however, and allows the scheme developed in ISO 19157 to be generalized. In this case, the following DQEs are proposed:
- ○
- Absolute positional accuracy: The precise location in the geographical space of buildings and civil works is fundamental. We believe that BIMs should be understood as fully integrated with geographic information and geoservices (e.g., spatial data infrastructures, virtual balloons, etc.). This means that absolute positional accuracy is a critical aspect, and a coordinate reference system and projection is required, if necessary. For example, absolute positional accuracy will be a requirement to properly integrate a BIM model with its cadastral plot and place it correctly in virtual balloons.
- ○
- Relative positional accuracy: This DQE means that the BIM data must accurately collect the relative positions between objects or parts of real-world objects (e.g., the distance between a door D and a window W, or the distance between the wall M1 and another wall M2).
- ○
- Accuracy of shapes (fidelity in shape): This DQE does not appear in ISO 19157, but its inclusion is proposed to consider all the geometric aspects related to the object itself, as opposed to the positional relationships between an object and its environment (e.g., absolute or relative positional accuracy). Fidelity in shape includes, among others, manufacturing tolerance. Therefore, depending on the aspect (e.g., roughness, roundness, etc.), different measures can be defined.

- Thematic accuracy: This category of DQEs is proposed to incorporate all aspects of accuracy that have a thematic component, whether quantitative or qualitative. The following elements are proposed in ISO 19157:
- ○
- Classification correction: This refers to the correct assignment of classes to objects in the BIM data.
- ○
- Correction of non-quantitative attributes: This refers to the correction of the values registered as attributes of the objects. Thus, there is an error if the material of a plinth, which is registered as granite, is actually marble and there is no attribute error if you register a RAL (Reichs–Ausschuß für Lieferbedingungen und Gütesicherung) color for a window, and the color matches the one that actually has the window in reality.
- ○
- Accuracy of quantitative attributes: Objects can have quantitative attributes (e.g., thermal or light transmissivity values). This element means that the values that are registered must be as accurate as possible.

## 3. Count-Based Quality Control

**Hypothesis 0 (H0).**

**Hypothesis 1 (H1).**

#### 3.1. Single Proportion

#### 3.2. Multiple Proportions

- Good, if its actual length differs by less than ±2% from the design length.
- Acceptable, if its actual length differs by less than ±5% but more than ±2% from the designed length.
- Unacceptable, if its actual length differs by more than ±5% from the designed length.

- If the population size is infinite (or very high with respect to sample size $n$), the distribution under the null hypothesis given in Equation (3) is a multinomial $\left(M\right)$ distribution, with parameters $\left(n,\text{}{\pi}_{01}{}_{\text{}},\text{}\dots ,{\pi}_{0k}\right)$
- If the population size $N$ is finite, and we assume that each category has a finite size ${N}_{i},{N}_{1}+\dots +\text{}{N}_{k}=N$, the distribution under the null hypothesis given in Equation (3) is a multivariate hypergeometric $\left(MH\right)$ distribution, with parameters $\text{}\left({N}_{01}{}_{\text{}},\text{}\dots ,{N}_{0k}\right)$. We can relate ${N}_{0i}$ with ${\pi}_{0i}$, considering that, under the null hypothesis, ${N}_{0i}=N\times {\pi}_{0i}$, so each ${N}_{0i}$ must be an integer.
- In both cases, the sampling statistics are $T=\left({t}_{1},\dots ,\text{}{t}_{k}\right),$ and to obtain the p-value, we use the probability of T and all possible points that are worse than $T$ (in the sense of the alternative hypothesis). For the multinomial case, see References [25,26], and for the multivariate hypergeometric distribution, see Reference [27]. More information is available in Appendix B.

## 4. Extension of the Method

#### 4.1. Control of Quantitative Elements

#### 4.2. Seriousness of Defects

- Critical: The defect affects critical functionality or critical data.
- Major: The defect affects major functionality or major data.
- Minor: The defect affects minor functionality or non-critical data.
- Trivial: The defect does not affect functionality or data.

#### 4.3. Joint Control of Several DQU

#### 4.4. Realization of the Global Contrast

- Take an independent sample for each DQU.
- Count the number of nonconforming items found in the sample of each DQU.
- Calculate the corresponding p-values for each DQU.
- Check whether the global H0 hypothesis is accepted or rejected according to MHTM correction.

## 5. Example of Application

#### 5.1. Concretions of the Control

#### 5.2. The Case

#### 5.3. Execution and Results

- Generating random sampling positions over which the completeness control is performed.
- Performing the control by visiting the positions of the building that are part of the sampling and where the reality to BIM data and BIM data to reality perspectives are taken into account. In this step, the measurements of quantitative attributes, preferably using a laser distance meter and assessments of qualitative attributes are performed on the correct items (neither omission nor commission). This phase is very important: The data taken here are considered to be the ground truth or reference. Therefore, extreme care is required with the working methods to ensure that the captured data (qualitative and quantitative) are accurate.
- Analyzing the results and making a final acceptance/rejection decision. The defect case counts are computed (Table 4). Based on these counts, and applying the functions “pbinom” and “phyper” of R [32] (indicated in the annexes), the p-values that appear in Table 4 are obtained. As can be seen, the hypergeometric model has been considered for the case QC2, and in the rest of the cases, the binomial model has been applied. Here, a MHTM is needed, so we apply Bonferroni because of its simplicity. Since α = 5% was adopted, the global null hypothesis should be rejected for any p-value less than 0.05/6 = 0.083. Given that the lowest obtained p-value is 0.0004 < 0.083, it is possible to reject the hypothesis that the BIM data complies with the specifications imposed by Table 4, since the observed data provide evidence of this.

#### 5.4. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

BIM | Building information model |

CoI | Category of Interest |

DQE | Data Quality Element |

DQU | Data Quality Unit |

GIS | Geographic Information Systems |

MHTM | Multiple Hypothesis Testing Method |

NIST | National Institute of Standards and Technology |

## Appendix A

#### Appendix A.1. Binomial Approach

- $p$:
- the p-value.
- $x$:
- the number of defective items found in the sample.
- $n$:
- the sample size.
- ${\pi}_{0}$:
- the maximum acceptable probability of the defective items.
- $B\left(n,\text{}{\pi}_{0}\right)$:
- the binomial distribution of parameters $n$ and ${\pi}_{0}$.

#### Appendix A.2. Hypergeometric Approach

- $p$:
- the p-value.
- $x$:
- the number of nonconforming items found in the sample.
- $n$:
- the sample size.
- $N$:
- the population size.
- ${\pi}_{0}:$
- the maximum acceptable probability of nonconforming items.
- $H\text{}\left(N,\text{}n,\text{}{\pi}_{0}\right)$:
- the hypergeometric distribution of parameters $N,\text{}n$ and ${\pi}_{0}$.

#### Appendix A.3. Multinomial Distribution

#### Appendix A.4. Multivariate Hypergeometric Distribution

## Appendix B

Planned Length (cm) | Observed Length (cm) | Absolute Error (cm) | 2% | 5% | Class |
---|---|---|---|---|---|

240 | 237.00 | 3.00 | 4.8 | 12 | A |

240 | 247.68 | 7.68 | 4.8 | 12 | B |

180 | 176.84 | 3.16 | 3.6 | 9 | A |

180 | 169.29 | 10.71 | 3.6 | 9 | C |

300 | 303.24 | 3.24 | 6.00 | 15 | A |

300 | 288.91 | 11.09 | 6.00 | 15 | B |

240 | 240.39 | 0.39 | 4.80 | 12 | A |

240 | 241.41 | 1.41 | 4.80 | 12 | A |

180 | 173.75 | 6.25 | 3.60 | 9 | B |

180 | 179.83 | 0.17 | 3.60 | 9 | A |

300 | 306.04 | 6.04 | 6.00 | 15 | B |

300 | 303.82 | 3.82 | 6.00 | 15 | A |

180 | 181.68 | 1.68 | 3.60 | 9 | A |

240 | 241.10 | 1.10 | 4.80 | 12 | A |

240 | 240.45 | 0.45 | 4.80 | 12 | A |

180 | 173.54 | 6.46 | 3.6 | 9 | B |

180 | 182.75 | 2.75 | 3.6 | 9 | A |

300 | 308.68 | 8.68 | 6 | 15 | B |

240 | 244.44 | 4.44 | 4.8 | 12 | A |

180 | 173.18 | 6.82 | 3.6 | 9 | B |

${\mathit{m}}_{\mathit{A}}$ | ${\mathit{m}}_{\mathit{B}}$ | ${\mathit{m}}_{\mathit{C}}$ | Probability |
---|---|---|---|

12 | 7 | 1 | 0.004858 |

12 | 6 | 2 | 0.006376 |

12 | 5 | 3 | 0.004081 |

12 | 4 | 4 | 0.001373 |

12 | 3 | 5 | 0.000244 |

$\dots $ | $\dots $ | $\dots $ | … |

0 | 17 | 3 | 0.000000 |

0 | 16 | 4 | 0.000000 |

0 | 15 | 5 | 0.000000 |

0 | 14 | 6 | 0.000000 |

0 | 13 | 7 | 0.000000 |

0 | 12 | 8 | 0.000000 |

0 | 11 | 9 | 0.000000 |

0 | 10 | 10 | 0.000000 |

## References

- Weygant, R.S. BIM Content Development (Standards, Strategies and Best Practices); John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2011. [Google Scholar]
- Ariza-López, I.A.; Ariza-López, F.J.; Reinoso-Gordo, J.F.; Gómez-Blanco, A.; Rodríguez-Moreno, C.; León-Robles, C. Data Quality Elements for BIM applied to Heritage Monuments. In Proceedings of the XIII International Forum Le Vie dei Mercanti, Aversa, Italy, 11–13 June 2015. [Google Scholar]
- Sun, J.; Harrie, L.; Jensen, A.; Eriksson, H.; Tarandi, V.; Uggla, G. Description of Geodata Quality with Focus on Integration of BIM-Data and Geodata. 2018. Available online: https://www.smartbuilt.se/library/3878/description-of-geodata-quality-2018-04-16-002.pdf (accessed on 14 October 2019).
- Song, Y.; Wang, X.; Tan, Y.; Wu, P.; Sutrisna, M.; Cheng, J.C.P.; Hampson, K. Trends and Opportunities of BIM-GIS Integration in the Architecture, Engineering and Construction Industry: A Review from a Spatio-Temporal Statistical Perspective. ISPRS Int. J. Geo-Inf.
**2017**, 6, 397. [Google Scholar] [CrossRef] [Green Version] - Lai, H.; Deng, X. Interoperability analysis of IFC-based data exchange between heterogeneous BIM software. J. Civ. Eng. Manag.
**2018**, 24, 537–555. [Google Scholar] [CrossRef] - COBIM. Series 6 Quality Assurance; Common BIM Requirements; 2012; Available online: https://buildingsmart.fi/wp-content/uploads/2016/11/cobim_6_quality_assurance_v1.pdf (accessed on 10 December 2019).
- Zadeh, P.A.; Wang, G.; Cavka, H.B.; Staub-French, S.; Pottinger, R. Information Quality Assessment for Facility Management. Adv. Eng. Inform.
**2017**, 33, 181–205. [Google Scholar] [CrossRef] - Donato, V.; Lo Turco, M.; Bocconcino, M.M. BIM-QA/QC in the architectural design process. Archit. Eng. Des. Manag.
**2018**, 14, 239–254. [Google Scholar] [CrossRef] - Park, S.; Kim, I. Bim-based quality control for safety issues in the design and construction phases. Archnet-Ijar
**2015**, 9, 111–129. [Google Scholar] [CrossRef] - Cheng, Y.M. Building Information Modeling for Quality Management. In Proceedings of the 20th International Conference on Enterprise Information Systems (ICEIS 2018), Funchal, Portugal, 21–24 March 2018; Volume 2, pp. 351–358. [Google Scholar]
- RIB. Available online: https://www.rib-software.co.uk/bim-qualifier-itwo (accessed on 14 October 2019).
- Cheok, G.; Filliben, J.; Lytle, A.M. NISTIR 7638. In Guidelines for Accepting 2D Building Plans; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2008. [Google Scholar]
- Cheok, G.; Franaszek, M. NIST 7659. In Phase III: Evaluation of Acceptance Sampling Method for 2D/3D Building Plans; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2009. Available online: https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7659.pdf (accessed on 10 December 2019).
- Cheok, G.; Franaszek, M.; Filliben, J. Evaluation of an Acceptance Sampling Method for 2d/3d Building Plans Derived from 3D Imaging Data; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2009. Available online: https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=907878 (accessed on 10 December 2019).
- ISO. ISO 19157: Geographic Information—Data Quality; International Organization for Standardization: Geneva, Switzerland, 2013. [Google Scholar]
- Yang, X.; Blower, J.; Bastin, L.; Lush, V.; Zabala, A.; Masó, J.; Cornford, D.; Díaz, P.; Lumsden, J. An integrated view of data quality in Earth observation. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci.
**2013**, 371, 20120072. [Google Scholar] [CrossRef] [PubMed] - ISO. ISO/TS 8000-1: 2011, Data Quality—Part 1: Overview; International Organization for Standardization: Geneva, Switzerland, 2011. [Google Scholar]
- ISO. ISO 8000-8: 2015, Data Quality—Part 8: Information and Data Quality: Concepts and Measuring; International Organization for Standardization: Geneva, Switzerland, 2015. [Google Scholar]
- ISO. ISO/IEC 25012: 2008 Software Engineering—Software Product Quality Requirements and Evaluation (SQuaRE)—Data Quality Model; International Organization for Standardization: Geneva, Switzerland, 2008. [Google Scholar]
- ISO. ISO/TR 21707: 2008 Intelligent Transport Systems—Integrated Transport Information, Management and Control—Data Quality in ITS Systems; International Organization for Standardization: Geneva, Switzerland, 2008. [Google Scholar]
- ISO. ISO 28590: 2017 Sampling Procedures for Inspection by Attributes—Introduction to the ISO 2859 Series of Standards for Sampling for Inspection by Attributes; International Organization for Standardization: Geneva, Switzerland, 2017. [Google Scholar]
- ISO. ISO 3951-3: 2007 Sampling Procedures for Inspection by Variables—Part 3: Double Sampling Schemes Indexed by Acceptance Quality Limit (AQL) for Lot-by-Lot Inspection; International Organization for Standardization: Geneva, Switzerland, 2007. [Google Scholar]
- ES. Guía para la Elaboración del Plan de Ejecución BIM. BIM, 2019. Available online: https://www.esbim.es/wp-content/uploads/2018/10/GUIA-ELABORACION-PLAN-DE-EJECUCION-BIM.pdf (accessed on 10 December 2019).
- Montgomery, D.C.; Runger, G.C. Applied Statistics and Probability for Engineers, 3rd ed.; Springer: New York, NY, USA, 2003. [Google Scholar]
- Ariza-López, F.J.; Rodríguez-Avi, J.; González-Aguilera, D.; Rodríguez-Gonzálvez, P. A New Method for Positional Accuracy Control for Non-Normal Errors Applied to Airborne Laser Scanner Data. Appl. Sci.
**2019**, 9, 3887. [Google Scholar] [CrossRef] [Green Version] - Ariza-López, F.J.; Rodríguez-Avi, J.; Alba-Fernández, V.; García-Balboa, J.L. Thematic Accuracy Quality Control by Means of a Set of Multinomials. Appl. Sci.
**2019**, 9, 4240. [Google Scholar] [CrossRef] [Green Version] - Johnson, N.L.; Kotz, S.; Balakrishnan, N. Multivariate Discrete Distributions; Wiley Series in Probability and Mathematical Statistics; Wiley: New York, NY, USA, 1997. [Google Scholar]
- Juran, J.M.; Godfrey, A.B. Juran’s Quality Handbook, 5th ed.; McGraw-Hill Companies: New York, NY, USA, 1998. [Google Scholar]
- GPO. Quality Assurance through Attributes Program for Printing and Binding; Government Printing Office Publication: Washington, DC, USA, 2002.
- Johnson, N.L.; Kotz, S.; Kemp, A.W. Univariate Discrete Distributions, 3rd ed.; Wiley Series in Probability and Mathematical Statistics; Wiley: New York, NY, USA, 2005. [Google Scholar]
- Dmitrienkoa, A.; D’Agostino, R. Traditional multiplicity adjustment methods in clinical trials. Stat. Med.
**2013**, 32, 5172–5218. [Google Scholar] [CrossRef] [PubMed] - R Development Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2019; Available online: https://www.r-project.org/ (accessed on 10 December 2019).

Group | Categories of Interest | Cases (N) |
---|---|---|

Elements | C1 = Doors and windows | 119 |

C2 = Bathrooms and Kitchens | 14 | |

C3 = Balconies and terraces | 29 | |

C4 = Other rooms | 18 | |

C5 = Living rooms and bedrooms | 16 | |

C6 = Common zones | 6 | |

C7 = Enclosures (walls) | 179 | |

C8 = Slabs and paving | 25 | |

C9 = Pillars | 105 | |

C10 = Sales unit | 6 | |

C11 = Interior walls | 200 | |

Facilities | C12 = Electricity installation | 7 |

C13 = Heating and air-conditioned installations | 7 | |

Total | 731 |

**Table 2.**Definition of the data quality units to be considered for the control (cases for the population and sample size).

Data Quality Units | Cases in the Population (N) | Sample Size |
---|---|---|

DQU1 = Completeness of elements | ||

DQE = Commission + omission | 511 | |

CoI = C1 + C2 + ··· + C11 | 50 | |

DQU2 = Completeness of facilities | ||

DQE = Commission + omission | 182 | |

CoI = C12 + C13 | 40 | |

DQU3 = Shape Fidelity | ||

DQE = Fidelity in shape | 1605 | |

CoI = C1 + C2 + ··· + C10 | 160 | |

DQU4 = Attributes of elements | ||

DQE = Correction of non-quantitative attributes | 462 | |

CoI = C1 + C2 + ··· + C10 | 50 | |

DQU5 = Attributes of installations | ||

DQE = Correction of non-quantitative attributes | 491 | |

CoI = C12 + C13 | 50 | |

DQU6 = Shape Fidelity of walls | ||

DQE = Fidelity in shape | 200 | |

CoI = C11 | 20 | |

Total | 3451 | 350 |

**Table 3.**Definition of the quality controls by means of the data quality units and the conformity levels.

Quality Control | Data Quality Unit | Data Quality Measure and ID * | Conformity Level (Maximum Proportion of Defects) |
---|---|---|---|

QC1 | DQU1 | Rate of excess items (ID = 3) + Rate of missing items (ID = 7) | 1% |

QC2 | DQU2 | Rate of excess items (ID = 3) + Rate of missing items (ID = 7) | 3% |

QC3 | DQU3 | Rate of unfaithful items (ID = **) | 5% |

QC4 | DQU4 | Rate of incorrect attribute values (ID = 67) | 10% |

QC5 | DQ5 | Rate of incorrect attribute values (ID = 67) | 10% |

QC6 | DQ6 | Rate of unfaithful items (ID = **) | 80%, 15%, 5% *** |

Quality Control | Number of Nonconforming Items | Sample Size (n) | p-Value | ||
---|---|---|---|---|---|

Binomial | Hypergeometric | Multivariate Hypergeometric | |||

QC1 | 0 | 50 | 1.000 | ||

QC2 | 5 | 40 | 0.0004 | ||

QC3 | 11 | 160 | 0.179 | ||

QC4 | 5 | 50 | 0.569 | ||

QC5 | 2 | 50 | 0.966 | ||

QC6 | 7.1 (*) | 20 | 0.0236 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ariza-López, F.J.; Rodríguez-Avi, J.; Reinoso-Gordo, J.F.; Ariza-López, Í.A.
Quality Control of “As Built” BIM Datasets Using the ISO 19157 Framework and a Multiple Hypothesis Testing Method Based on Proportions. *ISPRS Int. J. Geo-Inf.* **2019**, *8*, 569.
https://doi.org/10.3390/ijgi8120569

**AMA Style**

Ariza-López FJ, Rodríguez-Avi J, Reinoso-Gordo JF, Ariza-López ÍA.
Quality Control of “As Built” BIM Datasets Using the ISO 19157 Framework and a Multiple Hypothesis Testing Method Based on Proportions. *ISPRS International Journal of Geo-Information*. 2019; 8(12):569.
https://doi.org/10.3390/ijgi8120569

**Chicago/Turabian Style**

Ariza-López, Francisco Javier, José Rodríguez-Avi, Juan Francisco Reinoso-Gordo, and Íñigo Antonio Ariza-López.
2019. "Quality Control of “As Built” BIM Datasets Using the ISO 19157 Framework and a Multiple Hypothesis Testing Method Based on Proportions" *ISPRS International Journal of Geo-Information* 8, no. 12: 569.
https://doi.org/10.3390/ijgi8120569