Development of an Open Government Data (OGD) Evaluation Framework for BIM

Abstract

Open government data (OGD) provide an opportunity for developing various services by disclosing information monopolized by the government to the public so that the private sector can use it. The private sector is utilizing this to improve the work efficiency and productivity by collecting, analyzing, and reprocessing OGD for various work steps of a BIM-based design project. However, most studies on OGD focus on the functionality and usability of data portals and the factors for evaluating the data itself such as openness, accountability, and transparency. This study aims to provide an evaluation framework for OGD for the AEC industry to assess the data utilization environment in order to improve the productivity of BIM-based projects. Several OGD principles found within related literature are discussed, and from them we extract evaluation framework levels. Then, we validate the proposed framework by applying it to a case of developing a BIM-based design support system using OGD datasets. This research concludes by suggesting that to effectively utilize OGD in the construction industry, the private sector should simply view data after collecting them, create an institutional environment for creating new values by reprocessing data, and build an active data utilization roadmap based on this environment.

1. Introduction

Open government data (OGD) are governmental data that should be open to anyone without any copyright restrictions to redistribute in any form [1]. From socio-economic perspectives, OGD has recently gained considerable attention because of advancements in information technology and new business opportunities to utilize open data on web and mobile platforms. OGD have the potential to stimulate economic growth by increasing the participation, interaction, and contribution of open data users and providers [2,3]. Given that information is an important resource for added social, economic, and health-related value services and products, the power imbalance of accessibility to data is facing a global push for a fairer balance of access to data. This is particularly true in the context of recent global challenges such as the COVID-19 pandemic. A wide variety of open data policies and open government data portals, such as open.usa.gov, data.gov.uk, data.gov, and data.go.kr, have been developed to address the challenges faced in our times [4,5,6,7,8,9].

In the architecture, engineering, and construction (AEC) industries, technology has advanced, which has allowed stakeholders to collect, manipulate, update, and exchange information throughout the life cycle of construction projects. Building information modeling (BIM) serves as an intermediary to maintain a vast amount of information that is utilized and newly accumulated in the architecture, construction, facilities, and maintenance stages. BIM-based construction projects are highly regulated, fragmented, and data-intensive, primarily consisting of structured iterative processes using a large amount of unstructured and semi-structured data. Regarding the type or method of use of the information used, collected, or produced at each stage, the more complex the construction project, the more important the data management process at a file level becomes. Over the last decade, numerous international studies have focused on the smooth production, utilization and sharing of information, including construction information exchange systems, such as Industry Foundation Classes (IFC) [10] and Construction Operations Building Information Exchange (COBie) [11], and improvement of information interoperability among project participants. However, most studies were focused on advancing the BIM-based information operability, such as improving the production and sharing efficiency of BIM model data or preventing information leakage in the BIM process [12,13,14].

Therefore, this study examines the current status of work data used in each stage of the BIM-based construction process and investigates how OGD should be disclosed and provided to more efficiently and actively utilize the OGD, which can be collected from the government or external organizations. In particular, this research aims to define the challenges faced in terms of data creation, distribution, and utilization and the corresponding evaluation metrics rather than on various attributes of data itself such as accuracy, timeliness, completeness, relevancy, objectivity, believability, understandability, consistency, conciseness, availability, and verifiability [15,16,17,18,19,20,21], which have been the criteria for evaluating data quality in existing OGD studies.

2. Terminology

Before providing context on the usability and possibility of open data to be discussed in the research, the definitions of open data, open government data, and linked data are described first. The concept and relationships among these elements will be explained.

2.1. Open Data

The Open Knowledge Foundation [22] defined “openness” as follows: data is open “if anyone is free to access, use, modify, and share it”. Moreover, the people who use open data must not be restricted, and the application field of open data must not be specified. In other words, the data provided in open data format must be “platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information” [23]. Open data is considered to be a key factor of the open government [24].

2.2. Open Government Data

OGD are a subset of open data, which are created and published to the public, provided via information technology (IT) platforms [25]. Government data include various open datasets including datasets directly produced by the central and local governments such as administration, finance, social welfare, and healthcare and indirectly produced datasets such as food, culture, tourism, and weather. The positive effects expected by opening government data that are produced daily and reusable include economic benefits, development of communities, and support for public administrative functions, as well as improving the transparency and accountability of the government [26,27,28,29,30]. The Open Government Partnership (OGP) was established in 2011 as part of efforts of each country to open the government data to the public. Currently, 78 countries are members of the OGP, working collaboratively with the civil society to strengthen government transparency, accountability, and public participation [31]. Leading countries such as the US (data.gov), the UK (data.gov.uk), South Korea (data.go.kr), France (data.gouv.fr), and Singapore (data.gov.sg) make these datasets more accessible and thus easier to re-use with the use of information technology.

2.3. Linked Data

Data linking is the process of following “a set of best practices for publishing and connecting structured data on the Web”, leading to a global information space containing billions of assertions where both documents and data are linked [32]. On a technical level, linked data, apart from being machine readable, are linked to other external datasets published on the Web from diverse domains, relying on the typed statements that are defined in the Resource Description Framework (RDF) Syntax (www.w3.org/TR/rdf-syntax-grammar/, accessed on 20 December 2021) [33]. Data linking is the last step of the star deployment scheme (

Table 1. Levels of five-star open data plan [34].

Level	Descriptions
*	Datasets available on the Web in any format under an open license
**	Datasets available as structured data
***	Datasets available in a non-proprietary open format, e.g., CSV
****	Uniform Resource Identifier (URIs) provided to denote the datasets
*****	Datasets linked to other data to provide context

) for open data developed by Tim Berners-Lee, founder of the Web, in 2006. Owing to the explosive expansion of the World Wide Web, linked data allow data to be discovered and used by various applications based on semantic web technologies, exploiting new relationships between data from heterogenous sources that are interlinked through typed links.

2.4. Linked Open Data and Linked OGD

Linked Open Data are “all stored data connected via the World Wide Web which could be made accessible in a public interest without any restrictions for usage and distribution” [35]. Open data publicly obtained from various sources are connected to each other through a Uniform Resource Identifier (URI) and RDF in the online space and can be used in various fields. Moreover, anyone can produce customized applications (apps) and tools for specific purposes using the Application Programming Interfaces (APIs).

Conventional OGD have often been published as raw OGD in a format that does not allow automated machine processing or are encoded as heterogeneous structures by the source due to the restrictions in interoperability, scalability, and usability [36]. Such technical and infrastructural challenges can be overcome because the aggregation and integration of data from heterogeneous sources are facilitated by the linked OGD (LOGD) approach that integrates Semantic Web technologies and linked open data principles (Figure 1) [37,38,39]. LOGD using linked data have been applied to open government platforms in numerous countries, such as data.gov of the US and data.gov.uk of the UK [40,41]. In addition, several research projects have explored ontological frameworks and implementation models [42], the potential benefits of different LOGD governance modes [43], and the methodological guidelines for publishing LOGD [44].

2.5. Open Data Initiatives

The OECD has been working on illustrating the benefits of opening government data since 2014 to highlight the importance of data sharing in public by assessing governments’ efforts in three critical areas: openness, usefulness, and re-usability of government data [45]. According to the 2019 OECD Open, Useful and Re-usable data (OURdata) Index, which measures the availability, accessibility, and re-usability of government data of OECD member and partner countries, a growing number of OECD countries are enabling open government data portals to foster the usage of open government data in public sectors (Figure 2). However, many are still utilizing them primarily as websites, as no more than data catalogs, rather than collaborative platforms to foster innovative knowledge sharing practices that fully embrace the “open by default” and “government as a platform” approaches [45].

Since 2015, South Korea has ranked first in the OURdata Index for three consecutive years and received the highest score in the data availability and government support to re-use pillars in 2019 again after 2017. Since the enactment of the Act on Promotion of the Provision and Use of Public Data in 2013, South Korea has promoted quantitative openness policies which continuously provide open data to public institutions. Since then, the policy has been expanded to open high-quality public data centered on demand to meet the requirements of the public and the private industries, who are the primary users of public data. Recently, integrated data maps (www.bigdata-map.kr (accessed on 20 December 2021)) have been established through the Presidential Committee on the Fourth Industrial Revolution (PCFIR) chaired by the Prime Minister; these maps connect representative data platforms of public and private sectors in 16 fields including finance, distribution, and telecommunications, and AI Hub (https://aihub.or.kr (accessed on 20 December 2021)) which provides data for artificial intelligence learning to enhance Data, Network, and AI (DNA).

France, which ranked 2nd in OURdata Index, is an early adopter of OGD policies and one of the countries operating the most advanced OGD portals (data.gouv.fr (accessed on 20 December 2021)) among the OECD members. In addition to the basic “certified” datasets disclosed by public agencies, France also provides portal services that serve as a collaborative digital space, in which general users can also add datasets, which can be classified as “public interest”. Through this, France could build more user-driven open government data portals, and this is one of the reasons that the country received a high rating in data accessibility [45].

Australia, which ranked 6th in OURdata Index, has been implementing an open data policy with the goal of implementing a digital government since the Declaration of Open Government in 2010 and the Public Data Policy Statement in 2015. In 2010, Australia constructed a platform (data.gov.au (accessed on 20 December 2021)) that integrates open data of all ministries and local governments and allows anyone to access and utilize the data publicly and privately disclosed and available for purchase by federal, state, and local government agencies and private entities [46]. Australia received a high score in data availability by collaborating with stakeholders in various organizations (e.g., universities, private entities and companies, and industrial bodies). However, data accessibility was evaluated to require improvement because datasets were provided in non-machine-readable formats (e.g., PDFs) or proprietary formats (e.g., Excel files) [45].

As can be seen in the countries that ranked high in the OURdata Index, an open data ecosystem is required, which creates useful data governance values in the public sector by establishing a cooperative and active integrated data management system for smooth data interconnection and utilization. In addition, quantitative openness of public data can be achieved, which promotes business innovation and a data-driven digital economy that can create economic value that allows anyone to easily identify, connect, and combine data in the private sector of civil society and businesses.

3. Evaluation Framework for OGD in the AEC Industry

Existing studies in various fields have investigated the attributes affecting data quality, such as accuracy, reliability, and related context [47]. In contrast, this study aims to define evaluation metrics related to the operational challenges that may arise in terms of creation, distribution, and utilization of data by the users. To develop the evaluation framework, a systematic review of literature, with the associated phases of selection, assessment, extraction, and synthesis of framework metrics, was conducted. The approach to defining the evaluation framework for OGD in the AEC industry consists of the following four parts:

• Analyzing the status of data utilization in the work environment of BIM-based design and the effectiveness of the construction process to identify the possibility of utilization and importance of data provided by public institutions (Section 3.1);
• Analyzing the principles of various open government initiative evaluation frameworks from the perspectives of data creation, distribution, and utilization (Section 3.2);
• Classifying the evaluation metrics of OGD appliable to the AEC industry, followed by defining detailed levels of the evaluation metrics from the primitive level to the progressive level (Section 3.3);
• Finally, defining the integrated framework levels that combine each evaluation metric to evaluate the environment of OGD utilization for data-centric BIM (Section 3.4).

3.1. Data-Driven Process for BIM-Based Architectural Design

As digital data have increased tremendously over the last decade, the innovation paradigm is changing all over the world to actively introduce and spread big data as a core resource that can create value for the public sector. Numerous architectural offices and construction companies around the world are trying to build more efficient and higher quality architectures using such a vast amount of data and introducing data-oriented approaches to maximize return on investment. The AEC industry is developing into a core data-centric smart construction industry, as it is being merged with state-of-the-art technologies of the Fourth Industrial Revolution such as big data, AI, and IoT. In particular, the role of BIM is being emphasized more because it is the core technology of smart construction that can be used to maximize the productivity, constructability, and efficiency of the construction process while integrating and utilizing the information that is being produced and used in the full cycles making up the construction industry, such as design, procurement, construction, building operation, and maintenance.

BIM involves “a series of data-centric processes”, in which effective management and sharing of robust data is the core of delivering a project [48]. As a source of information regarding a building, BIM aims for a seamless collaboration among all project participants by improving the accessibility, usability, management, and sustainability of digital data in the building asset industry, which is transforming traditional peer-to-peer work processes. According to a report of JB Knowledge published in 2017, 30% of all construction companies were using applications that were not integrated with one another; as a result, 48.7% of them were manually transferring data and had many difficulties with data integration [49]. This occurs when most data sources are heavily siloed, or data collected from disparate sources are unstructured, thus causing productivity lags. By increasing the level of data integration via a seamless digital workflow, AEC firms can increase collaboration among myriad design and construction solutions with big data, increasing productivity and profitability.

The successful delivery of a construction project is a highly complex process requiring collection, collation, and exchange of vast structured and/or unstructured information across the entire building life cycle. Therefore, the task of setting the data management plan for a project is becoming ever more challenging because of the significant changes in the way buildings are briefed, designed, constructed, and used. In particular, the activities of the next stage are affected by the data created and used in the previous one, or they can have a considerable influence on the iterative process that needs to correct the work by returning to the existing work. Therefore, the data collection and utilization plan should be a key consideration in the BIM process.

The Royal Institute of British Architects (RIBA) divides the process of building a project into eight stages related to the briefing, design, construction, handover, and use of a building, as a constant point of reference [50]: stage 0 (strategic definition) determines the best means of achieving the client’s requirements; stage 1 (preparation and briefing) develops the details of the brief; stage 2 (concept design) develops an architectural design concept and determines the task and information requirements for achieving the stated outcomes; stage 3 (spatial coordination) spatially coordinates the design in response to the project brief, spatial requirements, and architectural concept; stage 4 (technical design) develops the information required to manufacture and construct the building; stage 5 (manufacturing and construction) involves the manufacture and construction of the building; stage 6 (handover) is when the project team finalizes the project and building use begins; and stage 7 (use) is when the building is in use, and continues until it reaches the end of its life.

It is impossible to apply the process proposed by the RIBA to all construction projects in the same way. Because projects of different scales require different data types, scopes, and utilization methods, construction-related organizations and associations are proposing BIM-based design processes from various viewpoints [50,51,52,53,54]. Therefore, the data utilization status in BIM-based construction projects dealt with in this study cannot be used as a standard. However, the RIBA process is used as a reference because it reflects all stages including the briefing stage for preparing the performance of a construction project, construction stage, and the use and life span of a building.

Figure 3 shows the core tasks from the process perspective in eight stages of a BIM-based construction project, including briefing, designing, delivering, maintaining, operating, and using a building. When each stage is completed, the project team members and outside parties exchange a large amount of information with each other. Such information exchange is based on BIM data and reference data from external sources and serves as the key input data for carrying out core tasks. Furthermore, the BIM model generated at the end of a stage plays the role of the foundation of the next stage. Therefore, it is crucial to strategically collect data required for carrying out the project from various sources and to use these data in accordance with the purpose of information generation for performing core tasks in each stage or for decision making.

A large part of the referenced data in each stage of work are composed of the data provided by government agencies or external organizations [55] or spatial data that can be collected at multiple levels from infrastructures and technologies such as the Internet of Things (IoT) [56] and BIM models, especially during the operation phase [57,58]. Notably, as depicted in Figure 3, geographic information system (GIS), weather data, and BIM-extracted data have emerged as having the most potential to provide support to BIM-related activities for the planning, construction and operation of buildings. In particular, the integration of BIM and spatial information can provide holistic digital representation of the built environment to achieve better understanding, collaboration and decision making through activities including the site appraisal [59], engineering technical design [60], climatic assessment [61], design authorization [62], performance evaluation [63], construction supply chain management system [64], post-occupancy evaluation [65], energy sustainability [66], and facility management [67] phases throughout the life-cycle of a building. Nowadays, the AEC sector requires an ever-deeper integration between BIM and geospatial data at the application, process, and data levels to create a richer digital container of information on a building [68,69,70]. Various attempts have been made to propose different methods to improve the interoperability between BIM and GIS domains [71]. They include data standard conversion, mostly from IFC to CityGML [68], data standard integration into a unified building model (UBM) [72], and linked data approaches through semantic web technologies [73].

In recent years, OGD portals and open spatial data infrastructures (SDI), at national and international levels, have been playing an important role in supplying geospatial data typically provided for free in a machine-readable format with minimal restrictions on re-use [74,75,76]. In the geospatial domain, significant advances have been made to foster success and innovation in industry [75]. Many important initiatives have been taken by European agencies and other governmental institutions across the globe. The Copernicus Programme [77], the ESA (European Space Agency) Thematic Exploitation Platforms (TEP) [78], and the Open Data Cube (ODC) [79] are pivotal among them. They are user-friendly open-source platform environments providing users with services and functions to test, run, and manage resource utilization.

Open and unrestricted access to information is likely to have an even more profound impact on the global open data ecosystem and lead to smarter and more sustainable outcomes for AEC projects [80]. However, the intrinsic differences in data structure, level of development and configuration methods have created various barriers in information collection and utilization [81,82]. Thus, without an organized information management system and well-designed open data standards, information loss or quality degradation occur with high probability during projects. Therefore, by ensuring that information is structured in an open and consistent manner from the initial stage of a project, significant improvements in cost, value, and work performance can be achieved. In particular, data fragmentation can be prevented, and the same level of information value can be generated while responding to low-quality collaboration environments resulting from low compatibility when using open shareable asset information based on the same non-proprietary format.

3.2. Evaluation Metric Definition: Method

Based on the definition of the principles, the subset of principles from different open governmental evaluation frameworks is classified and the evaluation metrics used in this study are derived. For these purposes, we utilized the intrinsic and operational characteristics of the data system based on the external and internal views of an information system [83].

3.2.1. Open Government Evaluation Frameworks

The value of all open data generated in the process of government activities as information is created throughout the entire cycle of use and re-use by the processes of curation and publication after the data are collected, produced, and processed by means of the government data-value cycle [84]. In particular, the sharing, curating, publishing, use and re-use processes have the greatest effect on the creation of public values. The data analyzed through the feedback loops of the value creation process are used in the decision-making process and contribute to the creation of new or more data as a result [85].

In numerous countries, the government and agencies, civil society, private companies, and open data experts collaborate to establish core principles regarding the opening, reusing, and publishing forms of open data (

Table 2. Open government initiative evaluation frameworks.

	Principles	Description
Public Sector Information Directive [88]	Available formats	Data must be available in any pre-existing format or language through electronic means.
	Principles governing charging	Where charges are made, the total income from supplying and allowing re-use of data shall not exceed the cost of producing and publishing the data.
	Transparency	Public sector bodies should pre-establish any applicable conditions and the standard charges for the re-use of data.
	Licenses	Public sector bodies may allow for the re-use of data without conditions or may impose conditions.
	Practical arrangements	Public sector bodies should facilitate the search for data available for re-use, where available through asset lists or portal sites.
	Non-discrimination	Any applicable conditions for the re-use of data shall be non-discriminatory for comparable categories of re-use.
	Prohibition of exclusive arrangements	The re-use of data shall be open to the public without granting any exclusive rights.
Open Knowledge Foundation [22]	Open license or status	Data must be in the public domain or provided under an open license.
	Access	Data must be provided as a whole with a reasonable one-time reproduction cost and be downloadable via the Internet without charge.
	Machine readability	Data must be easily accessed and modified in a form readily processable by a computer.
	Open format	Data must be provided in an open format which can be fully processed with at least one free/open-source software tool.
30 Open Government Advocates [89]	Completeness	Data should be released to the public as complete as possible with the explanations at the greatest possible level of detail, except to the extent necessary to comply with privacy laws
	Primacy	Data should be published with the highest possible level of granularity, not in aggregate or modified forms.
	Timeliness	Data should be published as quickly as necessary to preserve the value of data.
	Accessibility	Data must be available to the widest range of users for the widest range of purposes.
	Machine readability	Data must be properly encoded to allow automated processing
	Non-discrimination	Data must be available to anyone, with no requirement of registration.
	Non-proprietariness	Data must be available in a format over which no processing program has exclusive control.
	License-free	Data must not be subject to any copyright, patent, trademark, or trade secret regulation.
G8 Open Data Charter [90]	Open data by default	All government data must be published openly by default, while recognizing national and international legislation in regard to intellectual property and personally identifiable and sensitive information.
	Quality and quantity	Data must be released as high-quality open data, fully described, and as early as possible at the finest level of granularity available.
	Useable by all	Data must be available in open formats to all people to obtain and re-use it, free of charge, without bureaucratic or administrative barriers.
	Releasing data for improved governance	The release of open data should be transparent about data collection, standards, and publishing processes to strengthen democratic institutions.
	Releasing data for innovation	Open data in machine-readable formats should encourage individuals and civil society organizations to stimulate creativity and innovation.
Open Data Charter [91]	Open by default	Data must be created for governments and public, without compromising the right to privacy
	Timely and comprehensive	Data must be published quickly and in a comprehensive manner.
	Accessibly and usable	Data must be machine-readable and free of charge under an open license.
	Comparable and interoperable	Data must be in commonly agreed standards.
	For improved governance & citizen engagement	Data should be transparent to improve public services and civil engagement.
	For inclusive development and innovation	Data should help spur inclusive economic development.

). These principles are based on the concept that open data are a public property that can provide economic benefits and support the development of all members of the society and provide innovative policy solutions. Since the end of the 1980s, the European Commission has been trying to promote the digital information market by making as much public sector information available for re-use as possible, providing a common legislative framework. The European Commission Directive 2003/98/EX on the re-use of public sector information (PSI Directive)—now called the Open Data Directive [86]—is an EU directive to encourage the re-use of public sector information of EU member states for commercial or non-commercial purposes. The directive was established with two main objectives: providing public sector data to third parties at low cost under non-restrictive conditions, and ensuring a level playing field for both public bodies and private information industry in the market [2]. Subsequently, the European Commission performed a review of this directive, together with a public online consultation, and proposed the new Directive (EU) 2019/1024 in 2019, which introduced the concept of high-value datasets, ensuring their free availability in machine readable formats, provided via application programming interfaces (APIs) or as bulk download [87].

3.2.2. Derived Metrics from the Open Government Evaluation Frameworks

In some principles of the open government evaluation frameworks outlined in Table 2, similar characteristics are defined differently. They are based on dimensions from very heterogeneous perspectives, such as the administration, portal, dataset, and cell levels. Therefore, the metrics are reorganized using the internal and external views of an information system [83]. The external view of an information system addresses the use and effect of the system and its deployment in the organization, whereas the internal view focuses on the construction and operation of the system necessary to meet the functional requirements [16]. In this study, metrics related to the collection and generation of data constituting the information system are defined as the intrinsic view of the data system, and those related to activities of managing the data system and disclosing the data to the public are defined as the operational view of the data system. The intrinsic metrics are responsible for the quality assurance of data and affect the true value creation, whereas the operational metrics are related to effective management of data within the system and provide an environment that facilitates efficient data utilization by consumers.

Table 3. Metrics as related to the intrinsic and operational view of the data system.

	Metrics
Intrinsic view (capture, value)	Data-related
	completeness, primacy, quality, and quantity
	System-related
	open format, available format, non-proprietariness, for inclusive development and innovation
Operational view (maintenance, delivery)	Data-related
	open format, available format, non-proprietariness, machine readability
	System-related
	transparency, licenses, practical arrangements, non-discrimination, prohibition of exclusive arrangements, timeliness and comprehensiveness

shows the reclassification of the principles of OGD by applying these two views. The metrics of each view are divided into data- and system-related metrics based on the definition of principles. However, there are cases where the same metrics are related to both intrinsic and operational views. For example, open format, available format, and non-proprietariness appear as being related to both the internal and operational views. Furthermore, they appear to be both system- and data-related in the intrinsic and operational views, respectively.

This research focuses on the possible operational challenges in terms of creation and distribution of data, as well as the utilization of data by users. Therefore, metrics relevant particularly to the intrinsic view of the data system are excluded from the development of the evaluation framework in the following sections.

3.3. Evaluation Metrics for OGD in the AEC Industry: Results

OGD are supposed to foster collaboration, transparency, creativity, and innovation, and the public sector is the major producer and holder of information, which ranges from geographical data to weather data. In recent years, many studies have evaluated the functionality, usability, and limitations of data portals constructed by governments [28,92,93,94]. Furthermore, factors for evaluating the data itself disclosed on such portals, such as data openness, accountability, transparency, collaboration, legal obligations, and principles have been extensively researched [42,95,96,97,98,99,100]. However, evaluation frameworks to enhance the collection, processing, and usability of OGD from the perspective of a specific industry have not been researched in detail. Therefore, the objective of the framework is to evaluate the usability of OGD in a BIM-based construction project from the data processing viewpoint of users.

Based on the preliminary sorting, the metrics listed in Table 3 are developed into evaluation metrics by first sorting the metrics into a small set of categories, followed by dividing each category into five levels. The metrics of operational view related to the creation and distribution of data are classified into four categories based on their characteristics—interoperability, accessibility, reliability, and license (

Table 4. Four target categories for the metrics.

Categories	Descriptions	Metrics
Interoperability	The level of availability of data and the degree to which heterogeneous datasets can be linked or combined with each other	Open format
		Available format
		Non-proprietariness
Accessibility	The level of accessibility of data, the degree to which data are present, obtainable, and collected	Machine-readability
		Practical arrangements
		Non-discrimination
Reliability	The level of data reliability and accuracy by providing context and achieving traceability of datasets	Timeliness and comprehensiveness
License	The extent to which data can be copied, distributed, edited, remixed, and built upon	Transparency
		Licenses
		Prohibition of exclusives arrangements

). Although each category is conceptually correlated in terms of the management and distribution of data, they should be treated independently to facilitate more efficient utilization and processing of OGD in the AEC industry.

Subsequently, we divide each category into levels of evaluation that range from the most rudimentary level of data management and distribution to the most progressive and open level (

Table 5. Levels of the four-evaluation metrics for OGD.

Metric	Level	Description
Interoperability	Level 1	Data published in any format under an open license (OL)
	Level 2	Data published as structured data (OL/RE)
	Level 3	Data published in a non-proprietary open format (OL/RE/OF)
	Level 4	Data published with URIs (OL/RE/OF/URI)
	Level 5	Data published as linked data to other data to provide context (OL/RE/OF/URI/LD)
Accessibility	Level 1	Datasets published as a list, simply providing download and search capabilities
	Level 2	Datasets from different sources categorized into a set of collections based on the metadata
	Level 3	Datasets mapped into standardized data catalogs based on machine-readable metadata schema, such as XML or RDF
	Level 4	Datasets accessible through RESTful API based on the formalized metadata from the backend
	Level 5	Cloud computing service as Data as a Service (DaaS) or Analytics as a Service (AaaS), providing an extra service of data exploration or analysis based on the datasets
Reliability	Level 1	Data once produced cannot be managed or traced through additional updates.
	Level 2	A new dataset is provided for revised or added information, separate from the existing dataset.
	Level 3	New updates are notified, but the continuity of work cannot be guaranteed because the existing file is overwritten when data are updated (manual).
	Level 4	New updates are notified, and the continuity of work can be guaranteed because only the changed parts are updated (automatic). However, updated versions cannot be managed and data recovery is impossible.
	Level 5	New updates are notified, the details are provided, and partial updates (automatic) and version management are possible. High flexibility and usability of data management are provided.
License	Level 1	Data can be copied and distributed in any format in an unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the data provider
	Level 2	Data can be copied and distributed in any format in an unadapted form only, for commercial use, and only so long as attribution is given to the data provider
	Level 3	Data can be distributed, remixed, adapted, and built upon in any format for noncommercial purposes only, and only so long as attribution is given to the data provider
	Level 4	Data can be distributed, remixed, adapted, and built upon in any format for commercial use, so long as attribution is given to the data provider
	Level 5	Data can be distributed, remixed, adapted, and built upon in any format without any conditions

). The capability of providing maximum efficiency to the private sector by disclosing OGD or operating a system for OGD disclosure can be assessed. Further, detailed levels are defined for each evaluation metric. The data are searched and then applied to work in Level 1, whereas, in Level 5, new values can be created in an automated environment using linked data and APIs.

3.3.1. Data Interoperability

Data interoperability considers the format in which data are published and states whether such data are open to the public in a machine-readable and non-proprietary format. Numerous governmental data providers still publish data in a wide variety of, as well as in proprietary, data formats, which, in reality, require significant effort to make them usable in business [93,100,101,102]. As described in Section 3.1, the data used in the BIM process have various forms of information, such as geometric and spatial data derived from the BIM model, externally referenced data such as geographical data, statistical data, and weather data, and self-produced data such as office documents from daily operations. Therefore, to ensure enhanced interoperability of data among various stakeholders in different phases of the overall BIM-based design process, it is crucial to provide data in a form that can respond to the established open standards and tools in terms of semantics, standards, and, most importantly, schema.

The data interoperability metric is based on the five-star open data plan defined in the W3C and consists of five levels in total. Based on the features open license (OL), machine readable (RE), open format (OF), Uniform Resource Identifier (URI), and linked data (LD), the five levels range from Level 1, where data are disclosed as simple documents, to Level 5, which is an open state that complies with the principles of linked data and enables interoperability with various open data. As the level increases from 1 to 5, data usability increases, free data processing and recycling are easy independently from software, and smooth collaboration among stakeholders is guaranteed. Moreover, data users can search and expand to other related data based on data structure and connection information. In contrast, data providers must build a technical environment to convert existing data generated in a proprietary format into a non-proprietary format, and time and funds are required for data analysis and management to secure precise control and connectivity of data.

3.3.2. Data Accessibility

Data accessibility is a measure of the extent to which data are present, obtained, and collected and made ready for use [103]. It defines how easy it is for a data consumer to discover specific data and/or relevant datasets through a data catalog or repository. As described in Section 3.3.1., even when the provided data are built in a machine-readable format or open format, they will require much effort for data users to download, analyze, and classify information they need. Moreover, it would lead to difficulties when attempting to find relationships between two datasets with minimal background knowledge of the subject [101]. The accessibility of data is even more crucial when it comes to data published by structured or semi-structured sources such as institutions or organizations.

• Level 1 (data list-up): Data consumers can directly find and download the relevant data that they need among available data that are simply listed like posts in message boards on data portals. Only a simple search function is supported based on the basic information of the data, such as titles, publishers, or abstracts, which may result in the user being overloaded to go through all the results to potentially identify the useful and relevant datasets [104]. In particular, this can be a major challenge in a situation in which similar data are provided from various decentralized data sources [105,106].
• Level 2 (data collection): The published data are provided in a form so that they can be filtered as a collection and dataset based on low-level metadata records including basic information. Because the metadata records in this step are not configured in a standardized manner by data type or each institution that provides the data, there can be a heterogeneity issue in terms of semantics, standards, and schema. Thus, like Level 1, much effort is required by data users to select useful data.
• Level 3 (catalog using metadata): Datasets mapped into a standardized metadata schema enable machine-readable representations of the predominant data catalogs [107]. It can achieve better searchability and, subsequently, better accessibility for catalogs differing widely in scope, terminology, and structure. The use of extensible mark-up language (XML) or resource description framework (RDF) as a data format is preferrable, as they are highly descriptive data formats that help data consumers to clearly understand the data. A further aspect of versioning would also be recommended to capture how a dataset evolves over time [101,108].
• Level 4 (open APIs): Datasets are accessible through a content management system, such as Representational State Transfer Application Programming Interface (RESTful API), which allows agents to interact with data portals and easily retrieve the metadata in a structured format (e.g., as JavaScript Object Notation (JSON) data) with the respective dataset from the backend [38]. This allows for flexibility in provisioning data as well as full-scale data processing operations at the underlying database level [109]. Furthermore, it provides better accessibility to the raw datasets for further operations such as data analytics techniques, including dashboards and predictive modeling.
• Level 5 (cloud computing for Data as a Service, DaaS, or Analytics as a Service, AaaS): Data providers or portals provide an extra service with exploration or analysis tools running on a cloud infrastructure, which enable a data consumer to easily look through the published raw data in a visual manner using graphs or knowledge maps. Data consumers can extract valuable insights, usually on demand, regardless of the location and affiliation, from statistical models, and analysis can be conducted against the existing structured data or simulated future data. These analytics and data reports can also be downloaded in a machine-readable format, such as .csv or tab delimited .txt files, as another raw dataset for further usage.

3.3.3. Data Reliability

To encourage the active utilization by users when government agencies produce and disclose open data to the public, public authorities must create and maintain reliable, accurate, and trustworthy datasets. Government agencies must ensure that data users have the right to expect that they will be provided with accurate and up-to-date information [110]. In particular, data disclosed based on Internet technology are characterized by high accessibility and immediacy that the data can be quickly acquired anytime, anywhere. Hence, the misuse and distortion of inaccurate or incomplete data should be prevented through basic record management. In this context, for data to be utilized as indicators with high value and reliability, sufficiently meaningful contextual information should be provided along with the latest reliable information based on contents that change over time.

The authenticity and reliability of data can only be guaranteed when the up-to-dateness of data and information traceability can be secured through the provision of contextual information and the verification and management of the provided datasets. When outdated data are released as open data without context, the value of information can be damaged, and re-use of data will be discouraged and/or re-users will have to invest in checking and performing data cleansing activities which may increase the cost of accessing and interpreting data [111].

• Level 1 (update is not available): There is no additional verification process for the accuracy or completeness of data once they are produced; thus, data management or tracking becomes impossible. There is a high possibility of unreliable and inaccurate data because data that need to be periodically changed such as statistical and weather data cannot be updated.
• Level 2 (separate updated dataset is provided): When the information of a dataset is revised or updated or new contents are added, a new dataset is provided separate from the existing file. Therefore, data users must manually update through research when needed because connectivity with the existing dataset is not guaranteed, and it is impossible to track changes. Thus, there exists a data-compatibility risk.
• Level 3 (with broken continuity): When changes are made to the datasets, prompts and notifications are expected, and details about the updated contents are provided. However, to apply the updated dataset, it must be manually updated, which is only possible by overwriting the entire existing dataset. Therefore, if additional work is performed using an existing dataset, continuity of the work details cannot be guaranteed, and the work details may disappear in some cases.
• Level 4 (with half automation): Similar to Level 3, details regarding the updated contents are provided; however, only the changes in the dataset can be partially updated. Therefore, although there is the advantage that existing work details are maintained, the update history cannot be managed. Hence, the data user should take care when deciding whether to update the data, and the data cannot be restored once updated.
• Level 5 (with full automation): If data are newly updated, they can be automatically updated based on the importance of updates defined by the data provider or data user, and the details regarding data revisions and the update history are managed by version. Therefore, the update can easily be reverted back if anything goes wrong or newly updated data are not necessary. Moreover, higher data usability can be expected because data can be managed and analyzed by version.

3.3.4. Data License

The data license is a metric related to the standards for the scope of data that can be disclosed and the recycling scope when open data are produced and published. As the trend of OGD occurrences is rising with the will of governments and the establishment of a foundation through legislation and institutionalization, the importance of standards for distinguishing data that can be disclosed or defining the scope of utilization is emerging. The license standard that is almost universally used in open data is the Creative Commons License (CCL) [112]. The CCL allows the free use of copyrighted materials in principle under some terms of use rather than limiting the scope of use through contracts between parties. Data users can freely copy, revise, and redistribute data disclosed online under the condition that they comply with the conditions for use given by data providers. There are six license types based on four terms of use in the CCL (attribution, noncommercial, no derivative, and share alike). In addition, there is a perfect public domain, CC0 (also known as CC Zero), which can be used by anyone in the world without any additional condition. CC0 can guarantee the smooth free use of reproduced data and copyrighted materials. In this study, five levels were defined using attribution, noncommercial, and no derivative among the terms of use in the CCL from the perspective of the OGD provider’s rights and data users’ free data use.

• Level 1: Data can be copied and distributed in any format in an unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the data provider.
• Level 2: Data can be copied and distributed in any format in an unadapted form only, for commercial use, and only so long as attribution is given to the data provider.
• Level 3: Data can be distributed, remixed, adapted, and built upon in any format for noncommercial purposes only, and only so long as attribution is given to the data provider.
• Level 4: Data can be distributed, remixed, adapted, and built upon in any format for commercial use, so long as attribution is given to the data provider.
• Level 5: Data can be distributed, remixed, adapted, and built upon in any format with no conditions.

3.4. Framework Levels Based on the Four Evaluation Metrics

Collecting, processing, and analysis of OGD plays a key role in efficiently utilizing the OGD collected through various government and regional agencies in the AEC industry. In particular, in stages that have a significant effect on the value of construction projects such as decision making, forecasting, construction management, and process optimization, the data collection, storage, analysis, and exploitation of the results have the greatest impact on the development of data-centric BIM workflow. In this regard, the following framework evaluates the base environment for active utilization of data while developing services for public convenience using public data in various industries including the AEC industry, with a combination of the above-mentioned four evaluation metrics.

This evaluation framework moves through three levels, beginning with a simple batch processing model in which information of structured data format is periodically and directly updated by data users. As the model evolves, automatic update of the changed data only plays a key role. By the highest level (Level 3 hybrid processing), the model operates as a mixture of batch processing and real-time processing methods (Table 6). In particular, it provides the highest reliability and flexibility in terms of data management because data versions can be managed. This framework helps organizations and data providers to revise the current open data strategies and tries to enhance their capabilities to achieve data reliability and interoperability among data provided in various forms of information. In addition, it provokes active and meaningful participation from data users in the AEC industry by reproducing new business values with these OGD in diverse purposes and ways.

• Level 1 (batch processing): Batch processing, a general data processing method, is an efficient way of processing a large volume of static data collected over a certain period [113]. It manually updates data depending on the user’s need or data provision method periodically, such as daily or monthly, when the metadata of OGD are changed or updated. One example is the case of geographical or material data, which are updated only when needed by batch processing when real-time interconnection is not necessary depending on the purpose of data processing or there is a large computer load for data processing. Data update on this level does not guarantee continuity of the existing work because updates are performed by overwriting the entire dataset with a new dataset. Hence, more attention is required for data management. When data must be processed and applied in real time such as energy simulation or construction site management, the real-time processing method should be considered rather than batch processing, which processes data by batch at regular intervals.
• Level 2 (real-time processing): Real-time (streaming) data processing involves the continuous collection and automatic processing and analysis of OGDs whenever there are changes in the metadata of OGD utilized by a user or the contents are updated. To enable real-time processing of OGD, the prerequisites such as the provided data format, method of provision, and update method must be met. For example, the data must be built in a non-proprietary open format (interoperability level 2) at a minimum. Moreover, the data must be provided by RESTful API (accessibility level 4) so that only changed parts of the data can be partially updates in real time (reliability level 4). In this way, the continuity of existing work must be guaranteed. Several applications, such as post occupancy evaluation, energy supply, and fault detections in a built environment, require real-time processing of data streams gathered from heterogeneous sources and networks [114]. In the case of a license, batch processing allows the utilization and redistribution or original data as they are; however, real-time processing also enables secondary works such as data remix and adaptation (license level 3 or higher).
• Level 3 (hybrid processing): Hybrid models can deal with application domains where data processing can benefit from both the volume and processing time of static (previous behavior) as well as dynamic (current status) data. This is a hybrid method in which users can directly update the entire dataset to a new dataset when necessary, and it is also possible to automatically update only the changed part of data in real time. In particular, unlike real-time processing, data users can systematically manage data because work history can be managed based on the updated version (reliability level 5). This also has the advantage of securing data reliability and building a flexible work system. Like real-time processing, the data license also allows secondary works such as data remix and adaptations (license level 3 or higher).

Table 6. Integrated framework levels based on four-evaluation metrics for data-centric BIM.

Framework Level	Definition
	Level 1: Batch processing
	The newly changed data are periodically and manually updated (accessibility level 1 or 2) when a large volume of static data in the form of structured data (at least interoperability level 2) are changed. The continuity of work is not guaranteed because the entire data are overwritten (reliability level 2). The original form of data is used as is (license level 1 or 2).
	Level 2: Real-time processing
	Only the changed part of the existing information among data in the form of structured data (at least interoperability level 2) can be updated (at least accessibility level 3, but preferably level 4). Continuity of work is guaranteed because only the changed part of data can be automatically updated (at least reliability level 4). The method of showing data in their original form (license level 1 or 2) and the method of newly remixed, adapted and built upon in any format (license level 3 or higher) are both possible.
	Level 3: Hybrid processing
	A hybrid method that allows both batch processing, which can update the entire dataset in the form of structured data (at least interoperability level 2) with a new dataset and real-time processing that can update only the changed part of data (accessibility level 4). The highest reliability and flexibility are provided in terms of data management because the updated versions can be managed (reliability level 5). The method of showing data in their original form (license level 1 or 2) and the method of newly remixed, adapted and built upon in any format (license level 3 or higher) are both possible.

Table 6. Integrated framework levels based on four-evaluation metrics for data-centric BIM.

Framework Level	Definition
	Level 1: Batch processing
	The newly changed data are periodically and manually updated (accessibility level 1 or 2) when a large volume of static data in the form of structured data (at least interoperability level 2) are changed. The continuity of work is not guaranteed because the entire data are overwritten (reliability level 2). The original form of data is used as is (license level 1 or 2).
	Level 2: Real-time processing
	Only the changed part of the existing information among data in the form of structured data (at least interoperability level 2) can be updated (at least accessibility level 3, but preferably level 4). Continuity of work is guaranteed because only the changed part of data can be automatically updated (at least reliability level 4). The method of showing data in their original form (license level 1 or 2) and the method of newly remixed, adapted and built upon in any format (license level 3 or higher) are both possible.
	Level 3: Hybrid processing
	A hybrid method that allows both batch processing, which can update the entire dataset in the form of structured data (at least interoperability level 2) with a new dataset and real-time processing that can update only the changed part of data (accessibility level 4). The highest reliability and flexibility are provided in terms of data management because the updated versions can be managed (reliability level 5). The method of showing data in their original form (license level 1 or 2) and the method of newly remixed, adapted and built upon in any format (license level 3 or higher) are both possible.

4. Application of the Evaluation Framework: Case Study

The development of BIM design support systems using OGD was analyzed to verify the metrics suggested in this study. This system was a result of the project, “BIM-based design support system using national GIS public data”, supported by the Korea Agency for Infrastructure Technology Advancement of the Ministry of Land, Infrastructure and Transport of South Korea. The objective was to develop the Spatial Information Open Platform map service (Vworld; www.vworld.kr (accessed on 20 December 2021)) of the Ministry of Land, Infrastructure and Transport and a system that utilizes the information provided by a public data portal (Kportal; www.data.go.kr (accessed on 20 December 2021)) operated by the Ministry of the Interior and Safety to effectively acquire GIS information for architectural design. Vworld is opening various 2D and 3D national space information (e.g., national land management, regional development, road and transportation, maps, and facilities information) and search functions through open API so that users can create the desired content. This case aims to acquire in real time the topographical shape, cadastral map, land use, development restrictions, and 3D shapes (a shape similar to the actual shape rather than a simple box shape) of buildings around the target site from Vworld and Kportal in the building design and planning stage (Stage 1 Preparation and Briefing as described in Figure 3), and use them in Revit. This case study was conducted using published data [115,116] and an interview with the principal investigator of this project. The status of OGD provided by Vworld and Kportal used in this study was analyzed with the proposed OGD evaluation framework (Figure 4), and challenges that may occur in the process of developing a building design system were discussed using the OGD.

4.1. Data Interoperability

It is believed that Vworld, among the OGD used in this case study, could support up to interoperability level 5, but it only supports up to level 4 due to copyright problems. To answer our question regarding the reason for not disclosing the API of the cadastral map data, the Information Industry Promotion Institute (SpaceN), the operating agency of Vworld, answered that SpaceN cannot provide it because the Land and Geospatial Informatrix Corporation (LX), which is the cadastral map data operator, restricted information disclosure, and that we should contact LX directly. After that, we asked LX for the data. However, they answered that they cannot open it because the system for opening the information has not been developed. However, considering that Naver and Kakao, the representative map service portals of South Korea, also provide cadastral map services, it seems that LX does not want to provide information that can be reprocessed for the license and commercial purposes of LX. Therefore, it can be said that interoperability is affected by the issue of interest, rather than by technical issues. However, although Kportal supports up to level 5, the higher the level, the smaller the amount of data supported.

4.2. Data Accessibility

Accessibility level 4 is essential for the system developed in this case because it was developed to process and provide data as the data format of Revit in real time by dynamically bringing data to Revit. Vworld and Kportal provide basic data for development through the RESTful API; however, some data may not be supported, although they can be provided, due to interest and policy issues such as the case of data interoperability. A representative case is that Vworld provided 3D building data of a high level of detail through the RESTful API; however, they stopped the provision of the API. At that time, a South Korean company provided services that can be used in Rhino3D through this API. However, SpaceN demanded that the company stop providing the service for security reasons and shut down this API when the company continued to provide the service, claiming that the government was obliged to open the information.

4.3. Data Reliability

Vworld and Kportal support up to reliability level 5. This is based on the relationship whereby, when accessibility level 4 is satisfied, reliability level 5 is automatically supported. Moreover, both Vworld and Kportal notify users via their registered e-mails when the data update occurs.

4.4. Data License

Vworld and Kportal permit up to license level 5 in principle. However, their policy involves storing original data locally and prohibiting the reprocessing of data. A good example is the closing of the 3D building data API of data accessibility. After this incident, SpaceN added a regulation that restricts the local storage of open data. This has the pretext for preventing misuse of data. However, it is considered to be in violation of the government’s principle of information disclosure. Furthermore, this has the problem of equity because data provided as files can be stored locally by default. In fact, OGD should be supported up to level 5 in principle.

5. Discussion and Limitations

OGD refers to disclosing public information that was monopolized by the government before OGD. It is a system in which the government plays the role of opening information to the public while the private sector develops and provides various services for public convenience using the information. This system is being implemented smoothly in various fields; however, through this case study, it seems that the smooth operation of the system in the architectural design field is difficult. The architectural design field can achieve productivity improvement using OGD only if they cannot only simply see data, but also reprocess data. As discussed in Table 6, the data environment of level 3 (hybrid processing) is required in order to establish flexible data-centric BIM that enables systematic data management from the viewpoints of data users and ensures data reliability. In particular, the highest levels of data accessibility and reliability are the two most essential conditions required for optimum data-centric BIM design process, which requires iterative application of various data from heterogeneous sources in the different design phases of the data processing loop.

However, Vworld interferes with this by placing a restriction on the local storage of provided data. In the case of Kportal, we did not identify such a restriction, although it is difficult to assert that such a restriction does not exist. In the case study, there was a case of violating such a restriction, but it did not become an issue because it was developed for research and not many users are using it. Thus, careful evaluation is required because it is unclear whether the system will be actively utilized if the data cannot be stored separately, thereby limiting to only the real-time exploration of local terrain and buildings on the screen. In addition, even if OGD-related laws and copyright laws stipulate assurance of the publicity and openness of OGD as much as possible, it is necessary to consider that the restrictions in the use of OGD are not specified in some provisions on exceptions.

Through this case study, it seems that at least framework level 2 in Table 2 is required to effectively acquire and reprocess OGD for BIM-based architectural design process. If an environment in which data can be collected in real-time from OGD portals and reprocessed, is provided for user applications or provision of new services, the number of people using OGD will increase, leading to the recognition of higher values of the data. In addition, these attempts to increase the meaning and reusability of public sector information will encourage a prolific re-use ecosystem of OGD, as well as the building and development of datasets in areas where updates are not properly implemented.

In this case study, the proposed evaluation framework is applied in the case of the BIM/GIS integrated platform development. The framework allows search and immediate reprocessing in software for BIM design by using GIS-based geographic and spatial data. Further, the status of the application was analyzed for each metric. The limitations of this study are listed below.

First, because of the huge amount of time and effort required to manually define and evaluate some metrics, the number of principles evaluated in our literature review is small. As discussed in the Introduction of this paper, rather than focusing on the quality of data itself, we focused on the development of a theoretical framework based on the existing principles published by governments and public institutions in different countries. Thus, we could not include sufficient opinions from experts who implement BIM-based design. Therefore, development into an advanced evaluation framework is possible by incorporating the opinions of data administrators responsible for the collection and reprocessing of data in the actual work process and practitioners using the reprocessed data.

Second, research that specializes in an evaluation framework specialized for georeferenced data, which is most frequently used in BIM-based design work, is also necessary, because this study focuses on the generation and distribution of OGD, which can be used for BIM-based design. Particularly when we focus on data types of a particular field, it is necessary to further subdivide the evaluation framework of a comprehensive range proposed in this study, and to conduct advanced research into evaluation metrics closely related to the work in the applicable fields, such as content, representation, infrastructures, and standardization of data.

Lastly, while defining the three levels of the evaluation framework, metrics were selected based on the currently available level of infrastructure. Therefore, it is necessary to improve and update the definitions by incorporating the continuously changing ICT environment. In particular, in an environment with an increasing variety of tools and media for production and utilization of data, such as IoT, VR/AR, technology development on sensor networking, and accelerating pace of work integration, more detailed and comprehensive evaluation metrics with improved sophistication is required. Furthermore, the metrics must be defined considering the relationship of complex interplay and the influence between them.

6. Conclusions

The openness of OGD indicates a concept in which anyone can freely access, use, and share data for any purpose. The concept of open data is being universally applied as part of the open movement in various fields such as open source, open content, open access, and open knowledge, and the opening of public data based on open data is being actively promoted at the government level. Particularly to support such efforts, various institutions and platforms have been prepared to expand the information ecosystem so that anyone can freely access data to implement innovative and creative ideas. To this end, the production and distribution of high-quality public data through a data portal and the usability considering data users who collect and utilize such data must be strengthened. Collecting, processing, analyzing, and provisioning reliable open data are key challenges for the construction industry. Data provide opportunities to improve decision-making using information, create data-driven outcomes, and discover hidden values.

The future success of the AEC industry is related to how they find valuable insights from the large amounts of various types of data provided by the governmental organizations and adapt to data-centric approaches with the new technological advancements in order to make improvements in efficiency, productivity, and the fast innovation rate. Importantly, this huge amount of data does not always mean that they are accurate, reliable, and useful.

In this study, we reviewed the literature on linked data, OGD, open data initiatives, value creation chain of OGD and their evaluation frameworks. We also investigated and summarized data-oriented core tasks and information utilization within the BIM-based design process. Based on this review, we propose an evaluation scheme for OGD in the AEC industry with two main dimensions.

The first dimension refers to an approach from the perspective of a data provider to effectively and efficiently disclose and distribute high-quality OGD with high usability. The government or data providers should concentrate its efforts to understand the heterogeneous and dispersed nature of the published data to generate useful data and improve its relevance and usefulness. This includes four main evaluation metrics for OGD:

• Making data available as linked structured data in non-proprietary open format to ensure high interoperability that enables interconnection or combination between datasets;
• Making data available through open APIs and/or DaaS or AaaS for high levels of accessibility and expandability;
• Making data more reliable by supporting updates that enable tracking and managing data change details;
• Making data available to create an environment where free data movement and reprocessing are possible by granting permission to use data in fully open manners.

The second dimension refers to the integrated approach based on the abovementioned four metrics followed for evaluating the context in which the government plays the role of opening OGD to the public and the private sector plays the role of developing and providing services using OGD for various public conveniences. The conditions for the minimum support levels of the four key metrics were configured based on whether reprocessing is permitted for accessibility, scalability, and new value creation of public data from the perspective of the private industry sector as follows:

• Batch processing: This method manually updates the entire structured dataset when needed. It does not guarantee the continuity of work.
• Real-time processing: Continuity of work is guaranteed and data reprocessing is possible because an automatic partial update of structured dataset is possible when required.
• Hybrid processing: A mixture of the batch processing method, which updates all data as needed, and the real-time processing method, which can only update the changes. This method can manage various data versions.

Based on the proposed evaluation framework, cases of developing BIM design support systems using OGD in the AEC industry were analyzed. The analysis results suggest that although matters from the viewpoint of a data provider that distributes data through a unified public data portal are also important, it is essential to establish an open request system for OGD, an open API for collection, a clear legal basis for the limitations of use, and a stable system for continuous business promotion from the perspective of industrial users of OGD.

The proposed evaluation framework can evolve further by comprehensively expanding the evaluation metrics for the profound integration between meta-BIM models and open data analytics based on geospatial and energy-related data. This data-centric assessment strategy linking various domains of knowledge can provide multiple benefits both for industries and research communities, especially in the field of the sustainable built environment. This enables continuous elaboration of the performative BIM design process by creating data-driven design outcomes for improving decision-making and applying simulations for assessing the performance of different building design variants.

Data interoperability and reproducibility strategies are key components for future research initiatives in the built environment because they can fulfill fundamental industrial needs, such as the increase of work efficiency and productivity, while promoting multi-disciplinary research, especially in an environment with an increasing variety of ICT technologies. However, the provision of large amounts of OGD does not always mean that such data are of good quality and available, especially for commercial purposes in the private sector. The government needs to expand the production of high-quality reliable public data and actively promote the public data opening policy to increase the usability of public data, and to provide a basis to enable preemptive data opening and economic value creation considering user needs. Furthermore, not only the AEC industry, but also other private sectors should contribute to the creation of new business innovation markets through user-centric OGD reprocessing. To that end, it is necessary to secure manpower and organization for collecting and analyzing valuable data and construct a value creation mechanism in practice by creating a data management procedure and data utilization roadmap based on the reliability and scalability of data.