Text Mining Applications in the Construction Industry: Current Status, Research Gaps, and Prospects

Abstract

With the advent of the Industry 4.0 era, information technology has been widely developed and applied in the construction engineering field. Text mining techniques can extract interesting and important data hidden in plain text, potentially allowing problems in the construction field to be addressed. Although text mining techniques have been used in the construction field for many years, there is a lack of recent reviews focused on their development and application from a literature analysis perspective; therefore, we conducted a review with the aim of filling this gap. We use a combination of bibliometric and manual literature analyses to systematically review the text mining-based literature related to the construction field from 1997 to 2022. Specifically, publication analysis, collaboration analysis, co-citation analysis, and keyword analysis were conducted on 185 articles collected from the SCOPUS database. Based on a read-through of the 185 papers, the current research topics in text mining were manually determined and sorted, including tasks and methods, application areas, and core methods and algorithms. The presented results provide a comprehensive understanding of the current state of TM techniques, thereby contributing to the further development of TM techniques in the construction industry.

1. Introduction

Text mining (TM) refers to the process of extracting interesting and non-trivial information hidden in plain text, in order to obtain useful insights [1,2]. It has been applied to a wide range of industries, including biography, medicine, and manufacturing [3,4,5]. Studies in these areas have shown that TM techniques can offer valuable information for decision making and improving industrial productivity.

Due to the one-time feature of construction projects and organization, practitioners and academics tend to use experience-based information to make decisions. Experience-based methods, such as brainstorming, Delphi method, questionnaires, interviews, literature studies, and their combination, are primarily used to collect data and information [6,7,8]. This may lead to biased judgements, and the result will be limited by the number and knowledge of the experts. In particular, with the development of the construction industry, the density of information is increasing. The large amount of information (over 80%) stored in daily project management documents or standardized texts makes the use of traditional methods for information retrieval and management difficult [9,10]. There is a lot of potentially valuable information contained in text documents, but it is difficult to transform this information into knowledge by individuals or researchers. Therefore, in the context of Industry 4.0, there is a need to introduce emerging information technologies for the mining and research of text-based information in the construction field.

TM has been recently applied in the construction industry with the aim of extracting useful information from unstructured text documents that was not previously known and is not easily revealed. However, it should be noted that studies related to TM technology in the construction industry are still in the early stages, with research being very fragmented. There have been some review studies related to TM techniques based on qualitative analysis [11]. However, few studies have examined TM technology in the construction industry from a bibliometric and visualization perspective. Most existing reviews have focused on a certain aspect of the construction field (e.g., construction management or safety management), thereby lacking a review of the overall status of text mining applications in the construction industry. In order to address the limitations of these existing studies, in this paper, we use a combination of bibliometrics and manual analysis to analyze the research on text mining technology in the construction field, providing a more comprehensive introduction to current research hotspots and future development trends, in order to help readers understand the current situation of text mining in the construction field and provide a reference for experts, scholars, and engineering practitioners; this will help enable them to deeply grasp the research themes and trends of text mining technology in the construction field.

This paper presents a systematic and comprehensive review of the application of text mining techniques in the field of construction from 1997 to 2022, including publication analysis, collaboration analysis, co-citation analysis, keyword analysis, and topic analysis, as well as detailing some commonly used algorithmic models. Section 2 describes the data collection process and methodology; Section 3 provides a systematic econometric analysis of the data set through the use of software; Section 4 introduces some specific methodological models for text analysis; Section 5 summarizes the specific themes of text mining applications in the construction field, and Section 6 discusses future research directions for text mining in construction; and, finally, Section 7 summarizes the paper.

2. Data and Methods

For this paper, we used SCOPUS—a foreign language database—to obtain the literature data set by defining query keywords. After manual reading, we eliminated the papers that did not match the topic, and used two software programs, VOSviewer (version 1.6.18) and CiteSpace (version 5.6.R5), to conduct systematic analyses.

2.1. Research Framework and Methodology

The research framework is shown in Figure 1.

• For data collection, we used SCOPUS, a representative journal search engine, to collect data.
• In the data cleaning step, we aimed to filter out unrelated articles and papers, as well as normalize synonymous keywords to improve the performance of the bibliographic analysis.
• In the bibliographic analysis, we used a word co-occurrence network (CCN) for analysis and visualization, aiming to reveal the trends of publication, most productive authors, co-authorships, and collaboration between organizations.
• In the theme and topic status analysis, we combined human analysis and a knowledge map to discover the current research topics related to TM in the construction industry, considering the perspectives of applications of TM, tasks of TM, and TM techniques (algorithms). In this step, we also determined the research gaps and potential future research directions.

Science mapping is an information representation technique for finding and visualizing useful information; for example, by presenting it in a knowledge map [12]. VOSviewer (Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands) and CiteSpace (Chaomei Chen, China) are the most widely used software for bibliometrics analysis [13]. For this study, CiteSpace was used to depict co-authorship and co-citation networks, as well as the keyword timeline. Meanwhile, VOSviewer was used for clustering of the text data (including title and abstract).

In summary, the use of these three different approaches—bibliometric analysis, content analysis, and a quantitative systematic literature review—appropriately complemented each other and allowed us to uncover the theoretical foundations and structure of TM research, as well as to identify key research themes, uncover research gaps, and outline areas for future research.

2.2. Data Collection

The literature data were downloaded from SCOPUS, which is the world’s largest and most-cited database of its kind, involving approximately 22,700 international academic journals and conferences published by approximately 5000 publishers worldwide [14]. It offers intensive coverage of a wider range of journals compared to Web of Science. As such, SCOPUS was considered to be more suitable for our study, in terms of bibliometric analysis [15].

2.2.1. Process of Data Collection (Search Strategy)

Search queries for data collection were determined through expert consultation.

The definition of appropriate search words is significant for data collection. As TM research is still in its infancy, synonyms may exist. To achieve all the synonyms, the keywords of papers obtained from each search activity can be used to supplement the initial search words. The process of selecting search words is depicted in Figure 2.

2.2.2. Initial Search Query

In order to explore the relevant literature, we used the synonyms of TM, tasks of TM, and the application domain to define the search words. According to [16], text mining, also referred to as Text Data Mining, is roughly equivalent to Text Analytics or Knowledge-Discovery in Text (KDT). Thus, the synonyms of text mining were set as query #1.

Text mining describes a range of technologies for analyzing and processing text data. To cover a wider range of publications, the tasks of TM were also considered as search words. Typically, text mining tasks include text categorization, text clustering, information extraction, information retrieval, and sentiment analysis [9,16]. “Information extraction” and “information retrieval” may refer to a broader processing area, such as geological information extraction from point clouds [17,18], or information retrieval platforms (e.g., Google) [19]; therefore, we used object constraints to specifically target the processing of text or document(s). Thus, the tasks of text mining and object constraints were set as query #2.

In addition, considering that TM relates to multi-disciplinary research, such as building and construction, computer science, and management, we chose to use search words (i.e., construction industry and construction project) to focus on the domain literature, instead of setting a specific subject area. Thus, domain constraints were set as query #3.

Moreover, the range of data types collected was limited to articles and conference papers, as query #4.

Based on the above analysis, the initial search query was designed as the following combination of queries: (#1 OR #2 AND #3) AND #4.

The time span was not limited, as we wanted to reveal the emerging and developing trends of TM techniques. The query in SCOPUS was defined as follows, with a total of 144 articles and papers meeting the selection criteria:

1#: TITLE-ABS-KEY (“text mining” OR “text-mining” OR “text data mining” OR “text analytics” OR “knowledge-discovery in text”)

2#: TITLE-ABS-KEY ((“text categorization” OR “text clustering” OR “information extraction” OR “information retrieval” OR “sentiment analysis”) AND (“text” OR “document*”))

3#: TITLE-ABS-KEY ((“ (“construction industry” OR “construction project”))

4#: (LIMIT-TO (DOCTYPE, “ar”) or LIMIT-TO (DOCTYPE, “cp”))

2.2.3. Final Search Query

SCOPUS provides the high-occurrence keywords along with the search result. We reviewed the keywords to check whether there were any new synonyms of the search words. We performed three rounds, until no further synonyms occurred. The new words that added in each round are presented in

Table 1. Supplementary search words.

Category	Initial Search Words	Newly Added Words (Second Round)	Newly Added Words (Third Round)
Synonyms of text mining	text mining, text data mining, text analytics, knowledge-discovery in text	text-mining, text processing, text analysis
Tasks of text mining	text categorization, text clustering, information extraction, information retrieval, sentiment analysis	document classification, document clustering, information extracting, information retrieving
Object constraints	text, document(s)
Domain constraints	construction industry, construction project	construction industries, construction projects, building industry, construction sectors, construction sites	construction engineering, construction management

. Thus, the final search query was defined as described below, and 125 more (a total of 269) articles and papers were obtained.

TITLE-ABS-KEY ((“text mining” OR “text-mining” OR “text data mining” OR “text analy*” OR “knowledge-discovery in text” OR “text categorization” OR “text clustering” OR “document classification” OR “document clustering” OR “information extracti*” OR “information retriev*” OR “sentiment analysis” OR “text processing”) AND (“text” OR “document*”) AND (“construction industr*” OR “construction project*” OR “building industry” OR “construction sectors” OR “construction sites” OR “construction engineering” OR “construction management”)) AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “cp”)).

The various types of constraints and keywords are provided in

Table 2. Keywords for literature search.

Binding	Scope of Search	Keywords
#1	synonyms of text mining	“text mining” OR “text-mining” OR “text data mining” OR “text analy*” OR “knowledge-discovery in text” OR “text categorization” OR “text clustering” OR “document classification” OR “document clustering” OR “information extracti*” OR “information retriev*” OR “sentiment analysis” OR “text processing”
#2	tasks of text mining and object constraints	“text” OR “document*”
#3	domain constraints	“construction industr*” OR “construction project*” OR “building industry” OR “construction sectors” OR “construction sites” OR “construction engineering” OR “construction management”
#4	type of literature	LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “cp”)

.

2.2.4. Data Screening and Collation

For better analysis performance, human inspection was conducted to filter out the unrelated articles and papers, as shown in Figure 3. We read the abstracts of the collected articles and papers, and filtered out 87 unqualified ones. They could be divided as described below. Finally, we obtained a data set consisting of 185 articles shown in Figure 3 and papers closely related to TM in the construction industry, after excluding

• 22 papers with the search word “information retrieval”, using “information retrieval systems” to refer to information or document management systems. These papers were published before 2009;
• 24 papers with the unproper indexed keywords. They were chosen not by the authors but by content suppliers, and are standardized based on publicly available vocabularies in SCOPUS;
• 17 papers with the search word “information extraction” to cope with GIS data or 3D object recognition, which is not closely related to text information;
• 7 papers with the search word “text analysis” and “information model” using non-text mining techniques (e.g., ground theory) to examine the text;
• 13 papers with the search word “text mining” or other synonyms were review papers. Although we only chose journal articles and conference papers in the search query, review papers were still collected. These papers used a TM technique to analyze research trends and knowledge gaps in a certain area, which are not within the discussion scope of our paper;
• 3 of the papers were proceeding prefaces, aiming to introduce a collection of conference papers. TM techniques were only considered in review papers indicating the overall research status, which did not benefit the analysis.

There were 123 journal articles and 62 conference papers, ranging from 1997–2022. All documents were downloaded on 8 September 2022. The list of the articles is provided in the Supplementary Materials (available online at https://github.com/Nina-cumt/Paper-list-of-TM/tree/master, accessed on 12 December 2022).

2.3. Data Cleaning: Thesaurus Construction

There are some problems that must be addressed when using bibliography analysis software to process keyword analysis.

(1) Improper text segmentation needs to be standardized. Bibliography analysis software, such as VOSviewer, uses nouns or noun phrases as terms to represent the document. However, there are some terms with particular meanings in the construction domain. For instance, the term of “fall” means “fall from height”, as a type of safety accident; also, the term “classical vector space model approach” denotes “vector space model”. Thus, these domain terms need to be replaced with a consistent expression. Otherwise, the co-occurrence network may be redundant and fail to depict the significant terms.

(2) Meaningless terms need to be deleted. The general terms widely used in publications, such as lack, gap, understanding, and so on, need to be filtered out to highlight the meaningful terms. Terms such as construction industry and text mining are also considered as meaningless terms in this study.

(3) Synonym terms need to be normalized; for example, similar terms such as model, modelling, and modelling should be normalized. In addition, abbreviations need to be replaced by full phrases, such as IE (information extraction), CNN (convolutional neutral network), and so on.

Thus, a thesaurus was used to cope with the above problems, in order to highlight the meaningful keywords (available online at https://github.com/Nina-cumt/Paper-list-of-TM, accessed on 12 December 2022).

2.4. Text Visualization: Word Co-Occurrence Network

A word co-occurrence network (CCN) was used to represent the relationships between co-authors, co-citations, and co-keywords. In the CCN, the sizes of a node represent the weight (usually the full count of the number of appearances) of a word; the larger the node, the heavier the weight. An edge appearing between two nodes represents that the two words have appeared together (i.e., they co-occur); the thicker the edge, the more frequently they co-occur. The distance between two nodes reflects the strength of the relation between them.

3. Bibliographic Analysis

Next, we analyzed the annual trends; the journal distribution and the most highly cited publications; the most productive players and their collaboration relationships, in terms of author, institute, and country level; and the most influential journals.

3.1. Publication Analysis

This section focuses on the analysis of annual trends in the number of publications issued, the sources of bibliographic journals, and information relating to papers with high citations in the data set.

3.1.1. Annual Trend Analysis

Figure 4 shows the annual trends in related publications. The growth rate of related publications was relatively slow between 1997 and 2009, with the number of papers published being only in the single digits. However, the volume of papers published has increased significantly since 2010. This may be related to the fact that, at this point, people began to have a strong interest in text extraction from construction documents. In addition, there is an increased awareness of the importance of knowledge storage, knowledge exploitation, and the development of effective mining tools for knowledge acquisition. Although there were some individual decreases in 2011, 2012, and 2014, they did not affect the overall trend. With the development of computational technology, the demand for information and intelligence in the construction industry has also grown. In particular, after 2018, the number of papers presented a clear upward trend, with the number of papers published in 2019 alone equivalent to the number of papers published in both 2017 and 2018, and the highest number papers (25) published in 2021. At the same time, this trend also indicates that the application of text mining in the construction industry is a new and growing research field.

3.1.2. Journal Distribution Analysis

This section mainly focuses on journal articles (123 publications), excluding conference papers. The articles were distributed in 58 journals, of which 43 journals published only one article and seven journals published two articles; only eight journals published more than three articles, representing only 0.14% of the total number of journals. The scattered distribution stems from the multi-disciplinary nature of TM applications. The top four relevant journals are listed in

Table 3. Top four most relevant journals.

Published Journals	Number of Articles
Automation in Construction	22
Journal of Computing in Civil Engineering	14
Journal of Construction Engineering and Management	9
Advanced Engineering Informatics	9
Total	54

. A total of 54 articles, accounting for 44% of journal articles, were concentrated in four journals: Automation in Construction, Journal of Computing in Civil Engineering, Advanced Engineering Informatics, and Journal of Construction Engineering and Management. Automation in Construction was the most frequent journal for publication, with 22 articles in total. At the same time, this also revealed a problem: except for high-profile publications, most journals had only single-digit publication volume; thus, it is still worth exploring the areas of application and depth of text mining.

3.1.3. Highly Cited Publications

To highlight the most influential papers in the data set, we selected the top 10 papers with the most citations.

Table 4. The top 10 highly cited publications.

Citations	Title	Authors	Publication Year	Journal
151	Semantic NLP-Based Information Extraction from Construction Regulatory Documents for Automated Compliance Checking	Zhang, J.	2016	Journal of Computing in Civil Engineering
142	Automated content analysis for construction safety: A natural language processing system to extract precursors and outcomes from unstructured injury reports	Tixier, A.J.P.	2016	Automation in Construction
142	Automating hierarchical document classification for construction management information systems	Caldas, C.H.	2003	Automation in Construction
125	Automated classification of construction project documents	Caldas, C.H.	2002	Journal of Computing in Civil Engineering
124	Construction site accident analysis using text mining and natural language processing techniques	Zhang, F.	2019	Automation in Construction
96	Construction accident narrative classification: An evaluation of text mining techniques	Goh, Y.M.	2017	Accident Analysis and Prevention
96	Textual data mining for industrial knowledge management and text classification: A business oriented approach	Ur-Rahman, N.	2012	Expert Systems with Applications
92	Predicting construction cost overruns using text mining, numerical data and ensemble classifiers	Williams, T.P.	2014	Automation in Construction
91	Integrating semantic NLP and logic reasoning into a unified system for fully-automated code checking	Zhang, J.	2017	Automation in Construction
74	Automated information transformation for automated regulatory compliance checking in construction	Zhang, J.	2015	Journal of Computing in Civil Engineering

lists these highly cited papers, in terms of citations, title, authors, publication year, and journals.

The highest cited article, entitled “Semantic NLP-Based Information Extraction from Construction Regulatory Documents for Automated Compliance Checking”, was written by Zhang, J. and published in 2016, having 151 citations. A semantic, rule-based NLP method was presented in this study for the automated information extraction from building regulation documents. This method makes use of a set of pattern matching-based information extraction rules and conflict resolution rules. The patterns of IE and CR rules make use of a range of syntactic (syntax-related) and semantic (meaning/context-related) text properties. To lessen the number of necessary patterns, phrase tagging based on phrase structure grammar (PSG) is proposed, along with the separation and ordering of semantic information parts. To help in identifying semantic text features, ontologies are employed (concepts and relations). The 2009 International Building Code quantitative criteria were tested using the suggested IE algorithm, which yielded precision and recall values of 0.969 and 0.944, respectively. The majority of highly referenced papers were centered in the years 2014 through 2019, with a small number of excellent papers receiving numerous citations in 2002 and 2003.

3.2. Collaboration Analysis

A co-occurrence network was developed to represent the collaboration between authors, institutes, and countries. The large nodes represent the influential entities (i.e., authors, institutes, and countries), and the edges between nodes indicate collaborative relationships between entities.

3.2.1. Author Collaboration Analysis

The author co-authorship analysis is shown in Figure 5. Different colors indicate different research fields of the authors, and the lines describe publications jointly written by two researchers. The size of a node represents the number of articles published by the author. The distance between nodes indicates the relevance of co-authorship; the closer the distance between two author nodes, the stronger their relevance. The largest node in the figure is that of Zhang, J., revealing that he has published the largest number of papers. Due to mutual exchanges and cooperation between scholars, several author sub-network systems formed in the figure. The notable ones are the sub-network system headed by Zhang, J. and Xue, X. (green nodes), the sub-network system represented by Chi, S. (orange nodes), the sub-network system represented by Wang, J. (blue nodes), and the sub-network system formed by Jiang, S. and Zhang, H. (red nodes). There were few connections between several high-yield authors, indicating that scholars have insufficient awareness of communication and cooperation in text mining research, and so, academic exchanges and cooperation urgently need to be strengthened.

The top ten productive authors are listed in

Table 5. Most productive authors.

Author’s Name	Author’s Institute	Author’s Country	Number of Publications
Zhang, J.	University of Illinois at Urbana-Champaign	United States	7
Kandil, A.	Purdue University	United States	6
Rezgui, Y.	Cardiff University	United Kingdom	5
Jiang, S.	Dalian University of Technology	China	5
Al Qady, M.	Purdue University	United States	4
Lin, K.Y.	National Taiwan University	China	4
El-Gohary, N.M.	University of Illinois at Urbana-Champaign	United States	4
Hsieh, S.H.	National Taiwan University	China	4
Soibelman, L.	University of Illinois at Urbana-Champaign	United States	4

. Zhang, J. had the largest number of articles, followed by Kandil, A. and Rezgui, Y.; the number of articles published by these authors accounted for 25% of the total number of articles. An article may contain several authors at the same time.

3.2.2. Institute Collaboration Analysis

Figure 6 lists the top ten influential institutes. It can be seen, from this figure, that National Taiwan University, the University of Illinois at Urbana-Champaign, and Purdue University had a high number of publications. The number of articles published by these three institutions accounted for 10.3% of the total number of articles retrieved. China accounted for two of the top five institutions, in terms of publication volume. On one hand, this shows that China has many problems to be solved in the application of TM technology in the construction industry, such as the degree of automation of text mining and the learning ability of machine learning classifiers. On the other hand, it also indicates that China has invested more researchers in this field, and has accordingly developed rapidly.

Figure 7 shows the collaborative relationships between the considered institutions. When two institutions appear in the same publication, it means that they have collaborated once. Thicker lines denote closer institutional collaboration, and larger indicate show that the institution has produced more works in the topic. As can be seen from the figure, National Taiwan University, University of Illinois at Urbana Champaign, Purdue University, Deakin University, Curtin University, and Dalian University of Technology were the most influential institutions in terms of publications related to text mining, but there was almost no connection between these influential institutions, indicating that the cooperative relationships between the major institutions are very weak. Therefore, strengthening cooperation and exchanges between institutions may promote the publication of related articles and diversified development of TM in the construction industry.

3.2.3. International Collaboration Analysis

Cross-country authorship analysis is an important way to reflect the degree of communication between countries in a given field between related countries. The country co-authorship graph is depicted in Figure 8. The thickness of the node connections represents the level of cooperation between countries. The thicker the connection, the more the cooperation between the countries. The research was mainly concentrated in the United States, China, the United Kingdom, Australia, and South Korea, showing that these countries have more research results and strong research strength. In addition, there were cooperations between different countries. The link strength represents the cooperative relationship between nodes; the greater the value of the link strength, the stronger the cooperative relationship. In the figure, the connection strength between China and Australia was the highest, reaching 7. The strength of the connections between the USA and the U.K., as well as between the USA and Australia, was 2. Meanwhile, the link strength between the United Kingdom and China was 1, indicating that the cooperation between these prolific countries was not close. This may be related to the development level of the construction industry in different countries and/or the differences in construction technology between countries.

3.3. Co-Citation Analysis

When two items (e.g., references and journals) are cited in a publication reference list, they have a co-citation relationship. Therefore, reference co-citation analysis and journal co-citation analysis were conducted.

3.3.1. Reference Co-Citation Analysis

Reference co-citation analysis serves to identify whether a discipline has an inward- or outward-looking approach, links the flow of new ideas, and detects the structure and evolution path of a specific field [13]. When two publications are cited by one publication at the same time, it is regarded as a co-citation. Figure 9 shows the reference co-citation evolving with time. Different colors represent the times when the publications were cited. The size of the node indicates the number of times the document has been cited. The color changes from purple to yellow indicate the time in the past from far to near, respectively. The time span was from 1997 to 2021, in increments of one year. The size of a node represents the number of times the document has been cited, and the presence of an edge between two nodes indicates that the articles were cited at the same time.

It can be seen, from the figure, that in the reference time of the purple area (spanning from 1997 to 2000), the network connection is intricate, which means that these articles have been frequently cited and have similar research topics. The orange connection period was from 2000 to 2008, also presenting a relatively tightly structured network. The analysis of the literature indicated that the main focus has been on the classification and retrieval of information. For example, the study by Caldas C.H. [20] is representative of the research focused on improving the unique methods for information organization and access to construction project document management. On the basis of information retrieval science, Rezgui, Y. [21] has employed user profiling and document summarization techniques. For professionals in the construction sector, the conceptualization of domains through ontologies offers a practical answer for information and knowledge management. The yellow connection period indicates the duration from 2009 to 2020, and the more frequently cited authors were Zhang, F., Zou, Y., Williams, TP., and Baeza-Yates R., each cited five times. In 2014, Williams TP. published “Predicting construction cost overruns using text mining, numerical data and ensemble classifiers” in The Journal of Automation in Construction. He [22] has shown how to combine text describing construction projects with digital data and use data mining classification algorithms to determine the level of cost overruns. Prediction may be an important turning point regarding the application of text mining in the construction industry. Secondly, Zhang, F. [23] has applied text mining and natural language processing techniques to analyze construction accident reports, in order to prevent similar accidents. Building on case inference, Zou, Y. [24] has created a database of construction accidents and created a system for automatically finding related instances for risk management of building projects. This demonstrates that the application field of text mining in the construction industry has been gradually expanding.

It should be noted that, in the reference co-citation network graph, the importance of nodes did not reveal a large number of citations, but shows that this article excavated construction industry-related research topics. Betweenness Centrality measures the importance of nodes in the network. The specific calculation method involves calculating the percentage of the shortest paths between a pair of nodes in the network through a given node. In addition, the top ten cited references are also listed in

Table 6. The top ten highly co-cited references.

Frequency	Year	Author	Title of Publication	Journal
5	2019	Zhang, F.	Construction site accident analysis using text mining and natural language processing techniques	Automation in Construction
5	2017	Zou, Y.	Retrieving similar cases for construction project risk management using natural language processing techniques	Automation in Construction
5	2014	Williams, T.P.	Predicting construction cost overruns using text mining, numerical data and ensemble classifiers	Automation in Construction
5	1999	Baeaza Yates, R.	Modern Information Retrieval	USA: Addison Wesley
4	1999	Eastman, CM.	Automated code compliance checking for building envelope design	Journal of Computing in Civil Engineering
3	2013	Fan, H.	Retrieving similar cases for alternative dispute resolution in construction accidents using text mining techniques	Automation in Construction
3	2017	Goh, Y.M.	Construction accident narrative classification: an evaluation of text mining techniques	Accident Analysis and Prevention
3	2012	Caldas, C.H.	Automated classification of construction project documents	Journal of Computing in Civil Engineering
3	2012	Zhong, B.T.	Ontology-based semantic modeling of regulation constraint for automated construction quality compliance checking	Automation in Construction
3	2014	Chi, N.W.	Using ontology-based text classification to assist Job Hazard Analysis	Advanced Engineering Informatics

. The publications mainly focused on information retrieval, building quality compliance inspections for regulations, and the use of natural language processing, text classification, and other methods to analyze accident injury reports to prevent future accidents.

3.3.2. Journal Co-Citation Analysis

This enabled us to reveal the characteristics and subject structure of the cited journals. Figure 10 displays the journal co-citation network. The figure shows the structural distribution of journals with frequencies greater than or equal to three times, including 179 nodes and 2183 connections. The larger the label node, the more times the journal had been co-cited, and the closer the distance between the nodes, the closer the connection between them. Clusters with different topics are distinguished by different colors, divided into eight clusters. For example, the green cluster contains 28 nodes (i.e., 28 journals). The red cluster mainly represents journals related to construction information, visualization, and computers. The green cluster shows journals related to building energy, environment, and sustainability. The purple cluster indicates journals related to construction accidents, safety, and risk, including safety science, risk analysis, and so on. Automation in Construction, Journal of Construction Engineering and Management, and Journal of Computing in Civil Engineering ranked in the top three nodes, and the distance between these nodes was very close. The three journals were the top three on the fault line, with 147, 66, and 64 citations each and connection strengths of 3028, 1636, and 901 overall, respectively.

3.4. Keyword Analysis

The keyword co-occurrence network and density map were used to reflect the research hotspots of TM in the construction industry.

Keywords that appeared two times or more were used for analysis. We used a thesaurus to standardize the different descriptions of keywords. Of the 1241 keywords, 287 items met the threshold. The keyword co-occurrence network is displayed in Figure 11. Nodes with the same color belong to the same cluster. Nine clusters were generated, based on the association strength of keywords, which are detailed in

Table 7. Clustering of frequent keywords.

Cluster No.	Description of Cluster	Frequent Keywords
1	NLP-based	automated compliance checking, BIM, information extraction, machine learning, natural language processing
2	Document-based	contract, databases, documentation, information management, sustainability
3	Big data processing	data mining, deep learning, optical character recognition, text classification, web crawling
4	Unstructured data processing	social network analysis, text mining, twitter, unstructured data
5	AI-based	artificial intelligence, information retrieval, knowledge-based systems
6	Ontology and text representation	heterogeneous data, ontology, text representation
7	Knowledge discovery	k-nearest neighbor, knowledge management, knowledge map
8	Risk management	pre-bid clarification, risk management
9	Accident prevention	accidents, safety

.

The links reflect the relationships between keywords. Taking the node “text mining” as an example, strong links with “BIM” (5), “natural language processing” (5), “data mining” (4), “safety” (4), “knowledge management” (3), “machine learning” (2), and “web crawling” (2) can be observed. The link between “text mining” and “data mining” indicates their close relationship; this means that, to some extent, the two can be interchanged. However, text mining focuses on textual information (unstructured data), while data mining focuses on structured data. The links between “text mining” and “BIM”, “safety”, and “knowledge management” (KM) indicate that text mining has mainly used been to provide solutions related to these areas. The relationships between “text mining” and “natural language processing”, “machine learning”, and “web crawling” reflect the fact that new computational technologies have been utilized to carry out automatic mining.

Table 8. The top 10 keywords from the perspective of occurrence and total link strength.

Rank	Keywords	Total Link Strength	Occurrence
1	natural language processing	34	14
2	text mining	34	28
3	text classification	30	12
4	knowledge management	30	12
5	semantics	23	8
6	information retrieval	23	11
7	safety	22	8
8	ontology	21	8
9	information management	16	13
10	BIM	12	9

shows the top 10 keywords with high total link strength and occurrence. The total link strength of a node is the sum of link strengths of the node with respect to all other nodes. The link strength between two nodes refers to the frequency of co-occurrence, which can be used as a quantitative index to depict the relationship between two nodes. These keywords are considered as the core keywords for the application of text mining in the construction industry.

3.5. Keyword Timeline Analysis

Next, we generated a timeline to the keywords, as shown in Figure 12. There were almost no related applications of text mining before 2000. From 2000 to 2006, it was focused on information retrieval, information acquisition, and so on, while, at the same time, promoting the development of knowledge management in the construction industry, thereby laying a certain foundation for follow-up research. From 2007 to 2012, with the emergence of ontology and semantics, mining tools ranged from simple information retrieval to the use of natural language to process complex construction texts, which paved the way for broader text mining. From 2013 to 2020, new breakthroughs were made in text mining, mainly in the following three aspects: the support of big data (e.g., machine learning, deep learning, and artificial intelligence); the use of information theory, building information modelling, classification algorithm, information extraction, and other tools to improve the information acquisition and retrieval functions; and the continuous development of research in the field of construction safety risk management. These factors indicate that the use of text mining to process large amounts of construction safety risk information to prevent accidents and enhance safety is a topic that people in the construction industry should focus their attention on.

4. Text-Based Analysis Methods

Text analysis techniques typically involve the following phases when used in construction-related text mining research: corpus acquisition, pre-processing, text representation, and model training. Here, the corpus refers to paper or electronic text (e.g., Word, PDF, Excel) and image files, as described in Table 8. The prior work is summarized in model training, where an algorithm is taught to extract the necessary information from the corpus data.

4.1. Text Pre-Processing

Text pre-processing, including text cleaning, error correction, formatting, word separation, lexical annotation, and de-activation filtering, is the preliminary work performed on the original text in order to adapt it to a machine-readable form. For text separation and lexical annotation, more advanced open-source NLP tools are currently available; see

Table 9. NLP open-source tools and implemented functions.

Country	Tool Name	Function
Foreign	Apache OpenNLP	A Java-based machine learning work package that also supports ME (maximum entropy) machine learning
	NLTK	Lexical tagging, chunking, sentiment analysis, classification, semantic reasoning, and so on
Domestic	FudanNLP	Chinese word separation, lexical tagging, named entity recognition, text classification, news clustering, and other information retrieval functions
	Pkuseg	Chinese word separation (high accuracy)
	LTP	Chinese word segmentation, lexical tagging, named entity recognition, dependent syntactic analysis, semantic role annotation, and so on
	ICTCLAS	Chinese word separation, lexical tagging, text clustering, sentiment analysis, summary entities, code-switching, and so on

. The ICTCLAS Chinese word separation system has been employed in the field of construction engineering, in order to divide words and lexical annotation for construction quality acceptance requirements and documents in the area of coal mine safety [25]. Urban rail transit construction safety risk management has been conducted by utilizing the LTP method [26]. Xue and Zhang [27] have pointed out that generic lexicons are limited and the performance of open-source pre-processing tools may be degraded when dealing with domain-specific documents. Therefore, future studies will also require the manual building of dictionaries and ontologies that are relevant to the construction domain [28].

4.2. Text Representation

Text representation (i.e., text feature generation) enables the digitization of text with the help of data structures, such as vectors or matrices, characterized as machine-readable.

Table 10. Feature generation methods and introduction.

Method	Description	Feature
BOW/VSM	A discrete representation of text, where text objects are numerically represented as discrete features	The model is easy to understand and simple to implement, but only focuses on the text content, not the word order, and there is a semantic gap
TF/TF-IDF	If a word or phrase appears frequently in one text and rarely in others, the word or phrase is considered to have good category differentiation and is suitable for classification	Information retrieval and text classification based on frequency, but without consideration of semantic information of words
LSA/LDA	This approach focuses on the text vocabulary, counting the relationships between text topic words to achieve a textual representation.	The numerical modelling of text objects is based on their context, considering the connections between words is a continuous textual representation
Pre-trained language models	A neural network model that obtains semantic similarity by considering surrounding terms and embedding them in a high-dimensional vector space (e.g., Word2Vec, ELMO, Transformer, ERNIE, and BERT)	Continuous-type representation based on neural networks, wherein the model automatically realizes the learning of lexical, syntactic, and semantic features; and realizes text representation learning and the effective combination of subsequent natural language processing tasks

provides a brief overview of current feature generation methods based on modern developments. Traditional NLP techniques extract features from text data by analyzing the syntactic structure. A literature search revealed that the vector space model (VSM) is relatively simple and dominates feature generation methods. While TF has been historically popular as a metric for identifying key features, TF-IDF, first proposed by Jones [29], has become the main method for determining feature weights in documents. With the development of computer technology, deep learning algorithms based on neural networks began to appear, including Word2Vec, ELMO, and BERT.

4.3. Model Training

The last phase in text mining is model training, which uses the previously created features to carry out various tasks such as document classification, incident analysis, and compliance evaluation. Several algorithmic models that arose often in the literature analysis are listed in

Table 11. Training model and introduction.

Models	Advantages	Disadvantages
SVM	Small sample classification is better, the model is not very computationally intensive, and the generalization accuracy is higher	The model is sensitive to parameter tuning and kernel function selection, takes up more memory and running time in storage and computation, and is deficient in large-scale sample training
CRF	Seen as the dominant model for named entity recognition; internal and contextual feature information can be used in the annotation process	Slow convergence and long training time
CNN	Automatic feature extraction by convolution kernel	Inability to solve long-distance dependencies, ignoring the relationship between the local and the whole
RNN	The model specializes in sequences with timing information and is able to capture the hidden relationships between sequence units	Long input sequences are prone to gradient disappearance or gradient explosion problems
Bi-LSTM	Memory with network structure, remembering information in full sentences	Not as well-utilized as CNN for parallel computing
Self-Attention	The algorithm is able to learn the internal structure of a sentence by calculating dependencies directly, regardless of the distance between words, and is relatively simple to implement	——

. The majority of earlier studies employed conventional machine learning techniques, such as SVM, KNN, and CRF, with SVM models outperforming the others in terms of performance. Convolutional neural networks (CNN), recurrent neural networks (RNN), bidirectional long and short-term memory (Bi-LSTM), and other neural network architectures have received significant attention in recent years. The BERT model was put forth by Google in 2018, and since then, self-attention mechanisms have been used in the construction field. In the years to come, the number of publications on self-attention mechanism-based approaches for construction text analysis is anticipated to rise [30].

5. Current Theme and Topic Analysis

The results of the keyword analysis provided in Section 3.4 were slightly modified, based on a thorough reading of the 185 articles. The selected articles were grouped into four application directions: Document Management (DM), Automated Compliance Checking (ACC), Security Management (SM), and Risk Management (RM). In order to better structure the analysis of the selected articles, these four main categories were further sub-divided according to text mining tasks, as detailed in

Table 12. Statistical results of articles by category and task.

Construction Domains	Task	Data Source	Reference
Document Management	Knowledge extraction	Construction project documents, contract documents, engineering design plans, construction process documents, post-project review documents, tender documents, statements of work	[10,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]
	Knowledge search
	Classification/clustering
Automated Compliance Checking	Compliance checking	Building/construction specification documents	[51,52,53,54,55,56]
Security Management	Accident analysis	Accident reports, summary reports of disaster investigation	[22,57,58,59,60,61,62,63,64,65,66,67,68,69,70]
Risk Management	Factor analysis	Contractual texts, work reports	[23,30,71,72,73,74,75,76,77,78,79,80]
	Risk forecast

.

5.1. Document Management

The main research objectives of DM can be divided into the following three areas: knowledge extraction, knowledge retrieval, and document classification/clustering.

In document knowledge extraction, Al Qady and Kandil [39] have used NLP techniques to parse contract documents into noun phrases (NP), verb phrases (VP), and prepositional phrases (PP). By identifying subject and object triads <subj, VP, obj>, they extracted contextually relevant semantic knowledge to improving functions such as document classification and retrieval, with an F-measure score of 90%. Ren, R. et al. [81] have proposed a semantic rule-based information extraction method to automatically extract construction execution steps from construction procedure documents, reducing the workload of manually collecting information from construction procedure documents while achieving an accuracy of 97.08% and a recall of 93.23%.

Due to the availability of data and algorithms, document retrieval was the main topic of research, particularly in the beginning. NLP was progressively used for numerous applications up until 2015. This is often accomplished for document knowledge retrieval, by comparing the similarity of two representation vectors. Using TF-IDF and cosine similarity, Li and Ramani [82] have created an ontology-based design document query system that outperformed keyword-based search methods. By utilizing a Bayesian classifier to retrieve feature documents through similarity matching, Yu and Hsu [42] have developed a technique to reduce the dimensionality of VSM, enabling automatic and quick retrieval of CAD documents from 2094 Chinese annotated CAD drawings gathered from two actual building projects.

In document classification/clustering, the general process of document classification is pre-processing, text representation, and classification modelling. In the early literature, Caldas and Soibelman [33] have implemented an automated document classification system that can automatically classify construction project documents according to project components, with an average classification accuracy of 92.05% for the three levels. The recent research of Hassan and Le [83] has classified contract language into requirement and non-requirement material using Word2Vec and SVM, in order to shorten reading times and enhance comprehension of the contract scope. As a supplement to analytical tools such as CiteSpace, some researchers [84] have recently used LDA topic modelling to automatically assign one or more topics to documents, in order to achieve document tagging, which is used to analyze historical documents and extract clustered subject terms.

5.2. Automated Compliance-Checking

Automated compliance-checking using NLP techniques is another hot topic in text mining applications. Automated compliance-checking requires understanding and extracting constraints from various building regulation documents, followed by converting them into a formal format that allows for checking/reasoning. Two authors—Zhang, J. and El-Gohary—have made significant contributions to this field. In 2015, Zhang, J. and El-Gohary [53] proposed the extraction of rules based on pattern matching and conflict resolution rules. The same year, they [51] proposed a bottom-up conversion method based on semantic mapping and conflict resolution rules, in order to extract constraints and convert them into first-order logic using Prolog syntax. Building on previous research, Zhang and El-Gohary [85] have extracted regulatory concepts and industry base category (IFC) concepts from compliance documents. They then identified the relationships between each pair of regulatory and IFC concepts to create extended IFC schemas. At present, NLP-based compliance checks are mainly used to assess architectural designs [54] and work process dependencies [86].

5.3. Safety Management

The scope of safety management includes scheduling, cost, construction process, and so on. Rupasinghe et al. [76] have used support vector machines (SVM), linear regression (LR), k-nearest neighbors (KNN), decision trees (DT), plain Bayesian (NB), and integrated models to analyze construction accident reports and classify the causes of accidents. Tixier et al. [63] have developed a manual rule-based NLP program to automatically extract attributes and results from injury reports with an F1 score of 96%. Chi et al. have combined TF-IDF, principal component analysis (PCA), and SVM to classify accident categories from documents.

5.4. Risk Management

Risk management is broadly defined as the measurement, assessment, and development of contingency strategies for all aspects of the construction production process. Current research in the field of text mining related to risk management focuses on risk factor identification and analysis, as well as risk prediction. Siu et al. [73] have applied NLP software to identify 16 new risk categories for engineering contracts from unstructured text descriptions of NEC projects in Hong Kong, and used decision trees to analyze risk ratings. Kim and Kim [74] have identified factors related to building fire accidents from news articles, and then analyzed the main factors causing fire accidents in different seasons by using principal component analysis (PCA). Li et al. [87] have developed four main safety accident data sets, where the documents were represented by doc2vec vectors. As new incident reports emerged, the most similar data sets were selected based on doc2vec similarity, in order to share key factors that predict injury levels. The data sets were then trained to recommend deep learning models, based on their meta-features (e.g., proportion of category factors), in order to maximize prediction performance. Xu, N et al. [88] have proposed an information entropy-weighted term frequency (TF-H) for term importance assessment regarding the case of a Chinese metro construction project, extracting 37 safety risk factors from 221 metro construction accident reports.

6. Research Gaps and Future Studies

One may monitor the future development of a specialty field by tracking research frontiers that have been built on in the early stages of the field. This can also be achieved by monitoring emerging trends and patterns, in terms of the major dimensions in the latent semantic space spanned by each year’s publications connected to this particular cluster. On the basis of recent research, we have identified a number of shortcomings in the current research field, such as the lack of close institutional collaboration and the low quality of the sample. In response to these existing problems, we also provided future research directions for text-based research in the field of construction industry in this section.

6.1. Domain Customization Text Mining Technology

Although NLP-based text mining applications have yielded good results in the general-purpose domain, the results are not the same as those specific to the professional domain.

Corpora in the construction domain are primarily composed of accident reports or standard specifications, whereas the corpora in the medical domain are composed primarily of diagnostic reports and related books. Both of these feature document structures are very different from those for the general-purpose domain. Other sectors, including the medical and legal industries, have worked to develop specialized NLP tools and approaches in an effort to bypass these kinds of domain-specific constraints [89,90]. With the growing trend of electronic information technology and the help of domain experts, text mining can be applied to a wider range of fields.

6.2. Information Openness

The availability and growth of data sources are crucial for the development of text-based mining studies. In addition to existing electronic books and documents, the internet is one of the most significant sources of information, offering a wealth of knowledge. The majority of researchers opt to use Python-based approaches to obtain data from the internet; however, these methods may lead to unused data networks and computer language notation, which can lessen the usefulness of the training model without processing. In addition, both technical and non-technical considerations are required to prevent the unscrupulous collection of web data, which can lead to copyright or web server issues.

Safety- and regulation-related documents are the key data sources from the body of literature on text-based mining research. Regulatory documents issued by official bodies and accident reports published by public organizations are more authoritative and prescriptive, despite the fact that there are numerous stories and evaluations of construction safety accidents available through web-based media. Open data publication and information sharing are currently popular trends across the globe. Some countries have developed their own open data platforms, such as Data.gov in the USA (https://www.data.gov, accessed on 12 December 2022), Find Open Data in the U.K. (https://data.gov.uk, accessed on 12 December 2022), and the European Data Portal (http://www.europeandataportal.eu, accessed on 12 December 2022). Such open public data are expected to be of benefit to the construction industry.

6.3. Cutting-Edge NLP Technology

The NLP techniques used for text mining have progressed from rule- and dictionary-based approaches to traditional machine learning-based approaches (e.g., HMM, ME, and SVM) to modern deep learning-based approaches (e.g., CNN and Bi-LSTM). The most advanced language model in recent years is the BERT model, proposed by Google in 2018, which is a representative language model. No single algorithm can handle all activities with great performance, due to the complexity of data in the construction domain; however, researchers are currently working on more advanced computer models to make the construction industry more intelligent and digital. A combination of multiple algorithms (e.g., BiLSTM–CNN–CRF, BERT–BiLSTM–CRF) is considered more suitable to solve the problem at hand.

6.4. Building Engineering Expert System

Due to the intricate nature of the construction process and the distinctiveness of the project environment, it is challenging to completely integrate data-driven text mining research in the construction industry. The strength of construction industry experts lies not in creating cutting-edge algorithms—despite the fact that it was mentioned above that novel NLP techniques should be used more in the construction field—but rather, it lies in enhancing existing NLP and TM techniques through construction industry-specific domain knowledge, in order to broaden the application domain for TM. Academics have previously concentrated on creating useful text analysis tools or figuring out risk elements to examine in safety incident analysis. This can be advanced in the context of building automation by fusing already known knowledge with recently obtained textual information. For example, by building a knowledge classification system for coal mine construction safety management based on familiarity with pertinent mine standards and specifications, this knowledge can be used to create intelligent knowledge question-and-answer systems [91], or to create design solutions or emergency measures by combining other information (e.g., project context and safety-related knowledge). Automated expert system development will be aided by these applications.

6.5. Building a High-Quality Database

Text mining in the construction field requires a large volume of data for model training; furthermore, the higher the quality of the data format, the better the effect of model training. There are many kinds of existing engineering documents, such as accident reports, contract texts, construction documents, and so on. The document format cannot be fully unified, especially various tables, which are complicated to handle. Incorrect annotation can seriously mislead the machine learning process, and the acquisition and processing of data upfront is a major difficulty. For image and surveillance data, the current databases are limited to specific tasks, such as construction machinery inspection [92]. A deeper understanding of integrated data, based on text, images, and sensors, can bring potential advances to the construction industry, perhaps creating new knowledge and business opportunities. In addition, semi-supervised or unsupervised learning approaches, which require smaller training samples, provide a promising means to accommodate the above challenges.

6.6. Strengthening Partnerships

In text mining research in construction, several authors, including Zhang, J., Kandil, A., Jiang, S., and so on, had a higher volume of publications than others did; however, it is interesting to note that not all of the most-cited literature was authored by authors with a higher volume of publications. Some authors, such as Tixier, A J P. and Caldas, C H., had a low number of relevant publications; however, the article “Automated content analysis for construction safety: a natural language processing system to extract precursors and outcomes from unstructured injury reports” by Tixier, A J P. had a citation count of 142. Obviously, not all of the most prolific authors have established their cooperative networks. Furthermore, the findings indicated that the most important research and developments in the field of text mining take place mainly in the U.S., China, and the U.K., but the collaborative relationships between these countries are not as close, and the partnerships between the most influential institutions are very weak. This suggests that research on text mining in the field of construction is still fragmented, with insufficient partnerships between different institutions, countries, and authors. The next step is to pay attention to strengthening the exchange of research results, as well as jointly exploring richer theoretical research and more diverse application directions in the field.

6.7. The Breadth of Application Needs to Be Improved

The bibliographic analysis in Section 3 revealed that the number of articles published related to text mining has been increasing annually. Over the past two decades, a number of high-output and focused journals were identified, such as Automation in Construction, Journal of Computing in Civil Engineering, Advanced Engineering Informatics, and Journal of Construction Engineering and Management, indicating that these journals have had a significant impact on research in text mining. However, of the more than 50 journals involved in the literature data set, 43 journals had published only one article, and the research content of some of the small-volume journals was inadequate and did not address industry-related issues prominently enough, indicating that the breadth and depth of text mining in specific applications still needs to be improved.

6.8. Information Summary Generation

At present, the application of text mining in the construction field is mostly focused on document knowledge extraction and retrieval, accident cause analysis, compliance review, etc. The object text is also mostly engineering documents, and less attention is paid to engineering news information on the internet, such as accident news reports. News reports on engineering accidents are generally continuous, and different news portals have different focuses on the description of the event, so users often need to spend time browsing multiple portals to get all the information about the news event. The introduction of text mining technology can automatically crawl the news information of the portal; the event data are then extracted and processed with pre-trained models such as BERT [93] and eventually pushed to the user.

7. Conclusions

We presented a systematic analysis of TM applications in the construction industry. Our intention was twofold: first, we wished to demonstrate the potential depth of a systematic review by applying synthetic approaches. In addition to the application of literature analysis software, including CiteSpace and VOSviewer, we enriched the procedure for producing a systematic review regarding a knowledge domain by incorporating evolutionary models for a scientific specialty. The enhanced systematic procedure introduced in this article is applicable to the analysis of other domains of interest. Researchers can utilize these visual analytic tools to perform timely surveys of the literature as frequently as they wish, in order to retrieve relevant publications more effectively.

Our second goal was to reveal the research status and research frontiers of TM applied in the construction industry. Our timely survey demonstrated the highest-impact articles and the evolution of the relevant literature over time. The published articles were found to have mainly focused on the use of ontology, NLP technology, and the use of sentences and semantics to extract relevant construction industry information, as well as the use of text mining technology to classify project documents.

If the authors of the two major research centers strengthen their cooperation, rapid development of text mining technology in the construction industry may be facilitated. A wider range of applications of existing techniques will, in turn, widen our horizon and deepen our understanding of the challenges that need to be overcome in order to advance the state-of-the-art of text mining technology.

Limitations

The scope of the data was limited by the retrieval source (i.e., SCOPUS) and the composite query used. More in-depth analyses of each specialty would be more revealing, incorporating additional methods such as context analysis and studies of other aspects of scholarly publications. Patents and research grants are other types of data sources that may be considered, but for this particular review, we limited our investigation to the scientific literature indexed by SCOPUS.