Developing a Sentiment Lexicon-Based Quality Performance Evaluation Model on Construction Projects in Korea

Abstract

The increasing frequency of structural failures on construction sites emphasizes the critical role of rigorous supervision in ensuring the quality of both construction processes and materials. Current regulatory frameworks mandate the production of detailed supervision reports to provide comprehensive evaluations of construction quality, material compliance, and site records. This study proposes a novel approach to harnessing unstructured reports for automated quality assessment. Employing text mining techniques, a sentiment lexicon specifically tailored for quality performance evaluation was developed. A corpus-based manual classification was conducted on 291 relevant words and 432 sentences extracted from the supervision reports, assigning sentiment labels of negative, neutral, and positive. This sentiment lexicon was then utilized as fundamental information for the Quality Performance Evaluation Model (QPEM). To validate the efficacy of the QPEM, it was applied to supervision reports from 30 construction sites adhering to legal standards. Furthermore, a Pearson correlation analysis was performed with the actual outcomes based on the legal requirements, including quality test failure rate, material inspection failure rate, and inspection management performance. By leveraging the wealth of unstructured data continuously generated throughout a project’s lifecycle, this model can enhance the timeliness of inspection and management processes, ultimately contributing to improved construction performance.

1. Introduction

The prevalence of safety incidents arising from substandard quality performance on construction sites has underscored the critical need for enhanced preventative and mitigative monitoring tools. This necessitates a multifaceted approach encompassing the refinement of deliberative procedures, increased engagement of subject-matter experts, and the bolstering of quality control and inspection protocols. Crucially, mitigating the risk of structural failures necessitates reinforcing the pivotal role of site managers. These professionals possess the comprehensive expertise to oversee structural integrity, rigorously evaluate material conditions, and proactively identify and rectify potential lapses in construction quality [1]. In alignment with this objective, a systematic development of legal frameworks governing structural and construction quality management is underway. These frameworks mandate the stringent enforcement of safety protocols, the establishment and comprehensive review of quality assurance plans, and the rigorous assessment of quality management procedures. Furthermore, regulatory authorities are augmenting direct oversight through institutional mechanisms. Public construction projects commissioned by governmental entities are now subject to mandatory inspection and supervision by designated public officials. For projects exceeding about USD 10 million, comprehensive documentation via video recording is stipulated, ensuring meticulous quality and safety management [2,3]. Within the private sector, the standardization of quality assurance guidelines and diagnostic protocols is gaining prominence. This emphasizes the critical importance of continuous quality monitoring and assessment throughout the construction lifecycle.

The increasing integration of digital technologies within both governmental and private sectors reflects a proactive shift toward mitigating safety incidents often associated with inadequate construction quality. This emphasis on construction quality performance is driving advancements in methodologies for performance monitoring and diagnosis, necessitating further research into robust indicators, models, and assessment methods [4]. Recognizing the multifaceted nature of quality performance in construction sites, influenced by factors such as materials, contractors, and the scheduling of activities across planning, design, structural, and construction phases, there is a growing focus on developing integrated tools capable of leveraging both structured and unstructured data generated throughout these stages [5]. Both public and private entities are actively pursuing proactive identification and management of quality defects. This is achieved through the implementation of systematic standards and frameworks for diagnosing construction quality performance. By strengthening legal obligations related to quality defect management, these entities enforce standardized checklists executed by quality inspection teams assigned by the Enforcement Decree of the Housing Act [3] and conduct user pre-inspections. This comprehensive approach aims to detect and address quality concerns throughout the project lifecycle. Consequently, existing studies are increasingly prioritizing the evaluation of quality performance, classifying it as either conforming or non-conforming to established standards [4]. This research also seeks to develop methodologies for enhancing construction quality outcomes. While most construction sites ultimately comply with legal standards upon completion and handover, the utilization of unstructured data in quality conformance and management remains limited. By leveraging unstructured data generated on-site to assess quality performance, it becomes possible to enhance the accuracy and practical relevance of inspection and management activities. This, in turn, contributes to more effective strategies for improving quality performance [6,7].

Unstructured data, such as text-based reports generated by managers and supervisors vested with the legal authority and responsibility for on-site inspection and oversight, offers a valuable complement to standardized data in evaluating quality performance [8]. The inspection and management processes generate a wealth of data on construction and material quality, including comprehensive evaluation comments and final supervision reports, which accumulate in both structured and unstructured formats. Leveraging these diverse data sources for quality performance evaluation enables a more nuanced and systematic quantification of quality levels. This approach transcends traditional binary conformity/non-conformity assessments, facilitating proactive quality recognition and tracking. By adopting the approach, various challenges can be effectively addressed, including cost escalation due to reconstruction, enhancing customer satisfaction through improved quality, and reducing post-completion defects. Within the Korean construction industry, where construction supervision is mandated by the Building Code, the outcomes of inspection and management activities concerning construction and material quality are meticulously documented in supervision reports. These reports contain a rich tapestry of structured and unstructured data on quality performance, encompassing details of construction activities, material inspections, material grades and specifications, and post-construction quality evaluations [8]. This study proposes an evaluation model quantifying and differentiating construction quality performance levels through the analysis of representative unstructured text-based data. In contrast to conventional performance evaluations that primarily rely on structured data, this model leverages textual descriptions related to construction quality and incorporates a sentiment lexicon to achieve a more granular quantification of performance levels. The utilization of a sentiment lexicon provides a practical mechanism for labeling and scoring unstructured textual data by categorizing words and sentences into positive or negative sentiment scores, thereby enabling a nuanced grading of quality performance reflected in the documentation.

For this reason, the study analyzes documents authored by legally authorized supervisors and managers, which contain crucial information regarding quality performance. It proposes a model for evaluating quality performance levels on a continuous scale, moving beyond traditional binary assessments. Leveraging recent advancements in natural language processing (NLP) and analytical technologies, a sentiment lexicon-based quantification model is introduced to assess construction performance [9]. This model is specifically designed to analyze and evaluate unstructured data, facilitating the systematic accumulation of feedback for inspection and management activities. By grading quality performance levels with greater granularity than the binary conformity/non-conformity results dictated by existing legal standards, this approach offers a more nuanced understanding of quality performance. Employing techniques such as text mining, text extraction, frequency analysis, and sentiment lexicon construction, this study extends the applicability of data relevant to quality performance evaluation. Furthermore, it addresses limitations often observed in existing research concerning quality performance measurement at construction sites. Ultimately, this contributes to improved decision-making processes and more timely responses to quality performance outcome, enhancing the overall efficacy of quality management in the construction sites.

2. Overview of Previous Studies

2.1. Studies Related to Quality Performance Assessment

The Korean construction industry has recently intensified efforts to monitor and assess construction quality performance, with initiatives such as mandating direct supervision by the owner for public sector projects exceeding USD 20 million and requiring comprehensive video documentation of construction processes for all projects over USD 10 million. These measures are anticipated to underscore the importance of improving quality performance by strengthening and segmenting oversight throughout the construction phase [10,11]. Concurrently, research efforts are actively underway to develop reliable indicators for quality performance measurement and to refine systems encompassing quality management roles, duties, and certification mechanisms. Lee and Kim (2018) [12] conducted in-depth interviews with experienced managers possessing over ten years of field experience to estimate labor costs and the deployment scale required for effective construction quality management. Their study provided much-needed clarity regarding the previously ambiguous roles and responsibilities of quality managers. Cho (2017) [13] employed Partial Least Squares Structural Equation Modeling (PLS-SEM) to examine the causal link between management performance and quality outcomes in construction companies certified by ISO-based organizations, including KOSHA, OHSAS, and DQC. The findings indicated that quality performance at construction sites positively impacts organizational business performance and has a direct effect on the financial performance of ISO-certified firms. Furthermore, Jung and Yoo (2017) [14] employed correlation analysis, t-tests, and one-way ANOVA to explore factors influencing the quality management competencies of construction project managers. Their research revealed a significant correlation between managerial competence variations and improvements in quality performance. Lee and Kim (2017) [15] further redefined the roles of personnel involved in quality improvement initiatives, emphasizing the need for detailed standards, such as staffing and labor cost calculations, to support effective quality control. Finally, Lee and Bae (2017a, 2017b) [16,17] used Analytic Hierarchy Process (AHP) analysis to identify essential quality management infrastructure and core competencies. Their study concluded that the competencies of field practitioners are paramount for enhancing quality performance.

Previous studies have predominantly focused on the legal, institutional, and quality management practices governing construction sites, with substantial research dedicated to identifying management indicators and evaluating work performance metrics for assessing quality management outcomes. Methodologies include questionnaires, surveys, and case analyses, accompanied by statistical analyses involving quality managers or practitioners with significant field experience. While some studies have leveraged structured data to derive quality performance indicators and develop assessment models for construction sites, research utilizing representative unstructured data, such as documentation, remains comparatively limited.

International research on construction quality performance has increasingly focused on developing indicators to measure various facets of performance [18,19,20,21]. These indicators encompass cost, schedule, quality, client satisfaction, productivity, profitability, safety, and environment, aiming to provide a comprehensive overview of construction performance. Moradi et al. (2022) [18] in their comprehensive literature review, analyzed and systematized the attributes of key performance indicators (KPIs), ranking their importance as follows: safety, cost, quality, duration, productivity, client satisfaction, constructability, team satisfaction, environment and sustainability. Keenan and Rostami (2019) [19] investigated the influence of quality management systems on overall construction performance. Their findings underscore that effective quality performance management enhances organizational efficiency and streamlines management strategy. Demirkesen and Ozorhon (2017) [20] employed survey-based structural equation modeling to develop a robust performance measurement model. Their research suggested that quality and safety management indicators not only contribute to cost reduction but also serve as strategic guides for achieving project success. Cha and Kim (2018) [21] identified statistically significant indicators—namely, cost, schedule, quality, safety, and environmental indicator—applicable to subsequent construction sites based on rigorous analyses of high-performing projects. Furthermore, Wanaberg et al. (2013) [22] statistically examined the relationship between quality and safety performance, emphasizing that improvements in safety performance are fundamental to enhancing quality performance. However, their study, which analyzed data on rework costs and timelines across 17 sites, acknowledged limitations related to missing data and challenges in accessing granular, site-level data encompassing diverse building attributes. Despite these advancements in deriving quality performance indicators, limitations persist, particularly in the effective segmentation and utilization of unstructured data. Research thus far has largely relied on structured data, leaving a significant gap in studies aimed at leveraging unstructured data—specifically, data generated from inspection and management activities related to construction quality performance. Moreover, while the development of structured indicators for measuring and verifying quality performance has been active, challenges related to data diversity, completeness, and granularity remain. In response to these limitations, this study proposes a novel approach to measuring quality performance based on documents generated from quality inspection and management activities at construction sites. This approach seeks to broaden the utilization and applicability of unstructured data by presenting a quantitative model to assess quality performance levels. Table 1 summarizes the implications from previous studies on quality performance improvement and assessment according to characteristics of outcomes and methodologies.

2.2. Research Efforts for Applying Unstructured Data

Previous studies on quality performance at construction sites have highlighted recurring challenges related to data deficiencies, the absence of comprehensive datasets, and limitations in effectively utilizing unstructured data. In response, recent research has actively sought to advance the application and improvement of analysis techniques for unstructured data. Han and Kim (2023) [23] conducted a trend analysis of building usage satisfaction studies by examining co-occurring terms over time, identifying the development process and characteristics across different periods. Kim et al. (2021) [24] explored efficient management approaches for text data through performance analysis of deep learning-based classification models. Their study emphasized the increased complexity of analyzing text data compared to structured data and underscored the critical influence of data quantity and quality on model performance. Park and Park (2021) [25] utilized text mining techniques to examine networked relationships between specific texts, drawing on the wealth of unstructured big data available on digital platforms. Lee (2019) [8] proposed a framework to support decision-making using natural language processing (NLP) techniques for unstructured text data, which accounts for approximately 80% of data generated at construction sites. Lee highlighted the growing importance of NLP techniques such as Machine Learning, Lexicons, Ontology, Case-Based Reasoning (CBR), and Semantic Query Expansion in this domain. Furthermore, Choi and Kim (2018) [26] applied association rule and frequency analysis to analyze text data patterns, effectively addressing limitations inherent in extracting such patterns from structured data alone. Kang et al. (2017) [27] developed a model for estimating standard construction periods based on unstructured data from 400 projects and established a system to support decision-making processes. Lee et al. (2016) and Youn (2013) [28,29] constructed a corpus to analyze vocabulary usage and behavior from expert-derived data. This corpus, a computerized process of natural language expressions, facilitates information extraction to support decision-making. Yoon’s analysis also revealed that the Korean construction industry possesses a vocabulary that is 166% richer and more specialized than English; however, actual usage is comparatively limited and less diverse than in English. Table 2 presents the implications and characteristics of outcomes and methodologies in applying unstructured data to construction projects.

Table 1. Studies on quality performance of building construction sites.

	Key Implications	Research Category		Methodologies
		Quality Performance	Quality Control System	Statistical Analysis	Data Mining
Lee and Kim (2018) [12]	Statistically validated assessments associated costs through a detailed analysis of legally required quality control measures for quality testing, activity costs, etc.		●	●
Cho (2017) [13]	Analysis of the causal relationship between quality management and business performance using statistical analysis and a 7-point Likert scale questionnaire	●		●
Jung and Yu (2017) [14]	Identification and analysis of the factors influencing the quality management competency of construction project managers using statistical techniques such as correlation analysis, independent sample t-test, and one-way ANOVA	●		●
Lee and Kim (2017) [15]	Defining the roles and titles of personnel involved in quality management for construction projects through in-depth interview surveys	●		●
Lee and Bae (2017a) [16]	Analysis of key quality infrastructure components and core competencies necessary for enhancing construction quality certification systems using the Analytic Hierarchy Process (AHP)	●			●
Lee and Bae (2017b) [17]	Developing a new quality control suitability assessment system, named PQCS, by integrating WASCON and AQUA methodologies	●		●
Moradi et al. (2021) [18]	Systematic analysis of construction performance management from 2000 to 2020	●		●
Keenan and Rostami (2019) [19]	Examining the impact of a Quality Management System (QMS) on construction performance		●	●
Demirksen and Ozorhon (2017) [20]	Developing comprehensive and effective performance measurement indicators for construction projects		●	●
Cha and Kim (2017) [21]	Building a framework for measuring project performance specifically tailored to the Korean construction context		●		●
Wangberg et al. (2013) [22]	Correlation analysis between safety performance and quality performance	●		●
Park and Park (2015) [25]	Proposals for improvement through comparison of construction quality management systems by employing FTA		●	●

Table 1. Studies on quality performance of building construction sites.

	Key Implications	Research Category		Methodologies
		Quality Performance	Quality Control System	Statistical Analysis	Data Mining
Lee and Kim (2018) [12]	Statistically validated assessments associated costs through a detailed analysis of legally required quality control measures for quality testing, activity costs, etc.		●	●
Cho (2017) [13]	Analysis of the causal relationship between quality management and business performance using statistical analysis and a 7-point Likert scale questionnaire	●		●
Jung and Yu (2017) [14]	Identification and analysis of the factors influencing the quality management competency of construction project managers using statistical techniques such as correlation analysis, independent sample t-test, and one-way ANOVA	●		●
Lee and Kim (2017) [15]	Defining the roles and titles of personnel involved in quality management for construction projects through in-depth interview surveys	●		●
Lee and Bae (2017a) [16]	Analysis of key quality infrastructure components and core competencies necessary for enhancing construction quality certification systems using the Analytic Hierarchy Process (AHP)	●			●
Lee and Bae (2017b) [17]	Developing a new quality control suitability assessment system, named PQCS, by integrating WASCON and AQUA methodologies	●		●
Moradi et al. (2021) [18]	Systematic analysis of construction performance management from 2000 to 2020	●		●
Keenan and Rostami (2019) [19]	Examining the impact of a Quality Management System (QMS) on construction performance		●	●
Demirksen and Ozorhon (2017) [20]	Developing comprehensive and effective performance measurement indicators for construction projects		●	●
Cha and Kim (2017) [21]	Building a framework for measuring project performance specifically tailored to the Korean construction context		●		●
Wangberg et al. (2013) [22]	Correlation analysis between safety performance and quality performance	●		●
Park and Park (2015) [25]	Proposals for improvement through comparison of construction quality management systems by employing FTA		●	●

Table 2. Studies of applying unstructured data to construction projects.

	Key Implications	Research Category		Methodology
		Concept of Unstructured Data	Application of Unstructured Data	Statistical Analysis	Data Mining
Lee (2019) [9]	Identifying the trends in research utilizing NLP techniques within the construction industry	●		●
Han and Kim (2023) [23]	Identifying the trends and patterns related to user satisfaction through an analysis of unstructured textual data from 1983 to 2022		●		●
Kim et al. (2021) [24]	Presenting construction accident types by analyzing unstructured textual data using CNN and text mining techniques		●		●
Park and Park (2021) [25]	Investigating the relationship among keywords by analyzing the spatial meanings in textual data using a text mining technique		●		●
Choi and Kim (2018) [26]	Analyzing the large-scale unstructured textual data to identify how the meaning of words changes over time		●	●
Kang et al. (2017) [27]	Building a method for automatically generated CPM schedules in the COM network format by deriving regression equations from historical performance data		●	●
Lee et al. (2016) [28]	Structuring and analyzing unstructured data related to overseas construction disputes using the R program language		●	●
Youn (2013) [29]	Collecting and analyzing textual data written by architects, including those who have won PA Awards, to compare and contrast vocabulary usage between domestic and international architects		●		●

Table 2. Studies of applying unstructured data to construction projects.

	Key Implications	Research Category		Methodology
		Concept of Unstructured Data	Application of Unstructured Data	Statistical Analysis	Data Mining
Lee (2019) [9]	Identifying the trends in research utilizing NLP techniques within the construction industry	●		●
Han and Kim (2023) [23]	Identifying the trends and patterns related to user satisfaction through an analysis of unstructured textual data from 1983 to 2022		●		●
Kim et al. (2021) [24]	Presenting construction accident types by analyzing unstructured textual data using CNN and text mining techniques		●		●
Park and Park (2021) [25]	Investigating the relationship among keywords by analyzing the spatial meanings in textual data using a text mining technique		●		●
Choi and Kim (2018) [26]	Analyzing the large-scale unstructured textual data to identify how the meaning of words changes over time		●	●
Kang et al. (2017) [27]	Building a method for automatically generated CPM schedules in the COM network format by deriving regression equations from historical performance data		●	●
Lee et al. (2016) [28]	Structuring and analyzing unstructured data related to overseas construction disputes using the R program language		●	●
Youn (2013) [29]	Collecting and analyzing textual data written by architects, including those who have won PA Awards, to compare and contrast vocabulary usage between domestic and international architects		●		●

An examination of the applications of unstructured data within the construction industry reveals a diverse landscape encompassing images (photos and videos), text-based documents generated on construction sites, and technical drawings. According to Lee (2019) [8], the use of unstructured data gained significant momentum starting in 2015, with text-based unstructured data comprising 70 to 80 percent of all unstructured data. This trend indicates substantial growth potential for its utilization. Lee further argued that advancements in computing technology, which facilitate the seamless integration of structured and unstructured data, will effectively address the issue of data scarcity, a limitation frequently encountered in previous research. Given this context, most studies on quality performance evaluation have historically relied on refined structured data, with limited incorporation of unstructured data. Consequently, applying advanced techniques such as text mining, deep neural networks, convolutional neural networks, and lexicon-based sentiment analysis—key methods for harnessing the potential of unstructured data—is poised to become increasingly valuable tools for performance evaluation.

3. Methodologies

3.1. Outline

By employing NLP techniques, a sentiment lexicon specific to the quality performance is constructed. This is achieved through the preprocessing of text-centered unstructured data embedded within these documents. The model assigns sentiment labels to individual words (nouns, adjectives) and sentence units, categorizing them into negative, neutral, and positive based on the Counting Numbers of Positive and Negative Words (CNPNW) methodology [30,31]. This lexicon is developed through a rigorous process of text extraction and frequency analysis conducted on a corpus of documents related to construction quality performance. Text mining techniques are employed to facilitate effective sentiment analysis training. To rigorously evaluate the model’s performance, legally mandated quality performance indicators, including quality test failure rates, material inspection failure rates, and the adequacy of inspection management, were compared across 30 construction sites. These sites, which were used to develop the sentiment lexicon, were then assessed using the trained sentiment analysis model. The dataset encompasses 30 construction sites, comprising 19 residential and 11 non-residential projects. The research methodology employed this quality performance evaluation model based on the sentiment lexicon (SL) is outlined as follows:

First, a comprehensive review of previous studies on quality performance evaluation and management models within the construction industry was conducted. This review encompassed both domestic and international studies analyzing their distinctive characteristics and identifying research gaps. Additionally, studies exploring the application of unstructured data within the construction domain were examined, with particular attention to cases demonstrating the effective utilization of NLP techniques for analyzing text-based documents.

Text-based evaluations of quality performance for completed construction sites were systemically collected, and a standardized procedure for quantifying performance levels was established, employing association rules and social network analysis to interpret unstructured data and construct a corpus-based sentiment lexicon.

Keywords pertinent to quality performance were extracted through association rules and social network analysis. They served as the foundation for a comprehensive database to label relevant words and sentences.

Text-centered unstructured data were meticulously preprocessed, cleaned, and analyzed using Python 3.8. During the preprocessing phase, the Okt module from the KoNLPy library was employed to extract nouns, adjectives, and verbs from sentence-level data [31]. Irrelevant words were systematically removed, resulting in a refined set of 291 words and 432 sentences directly linked to quality performance and evaluation outcomes.

Sentences and words within the unstructured data, accumulated from 30 construction projects, were assigned sentiment labels: 0 (negative), 1 (neutral), or 2 (positive). Three quantitative indicators—quality test failure rate, material inspection failure rate, and legally mandated inspection management performance—served as benchmarks for traditional quality performance assessment. Quality performance levels for each construction site were then compared using sentiment scores derived from the sentiment analysis model alongside the three indicators set by institutional standards.

Finally, a correlation analysis was conducted between the results from the sentiment lexicon-based quality performance evaluation model and the indicators. This analysis explored the practical application of quality performance sentiment scores in relation to the evaluative judgments of supervisors and managers.

3.2. Lexicon-Based Sentiment Analysis

3.2.1. Understandings of Sentiment Analysis

Sentiment analysis, a field within natural language processing (NLP), text mining, and computational linguistics, seeks to discern knowledge, attitudes, and opinions embedded within textual data, particularly those expressed by managers [30]. Also referred to as opinion mining or emotional evaluation, sentiment analysis quantifies subjective information such as emotions and opinions, enabling their application in targeted contexts. This methodology has witnessed increasing adoption across diverse domains, including user feedback, survey research, and social media monitoring to enhance management and service efficiency [31,32]. By quantifying text that reflects user or managerial perspectives on a service, sentiment analysis can support the analysis and prediction of outcomes resulting from improvement measures or management activities [33]. Sentiment analysis employs various approaches, including Polarity Analysis, which classifies sentiments as positive or negative based on textual experience and knowledge; Emotional State Analysis, which identifies specific emotional expressions, such as conformity and satisfaction; and Intensity Analysis, which measures the strength of positive or negative emotions conveyed in the text. These methods provide nuanced insights, enabling improved decision-making and outcome predictions.

Sentiment analysis can be technically categorized into lexicon-based and supervised learning-based approaches [30]. A sentiment lexicon (SL) comprises a curated list of words or expressions with inherent emotional polarity, such as “good” or “poor,” and idiomatic phrases like “improves performance.” Lexicon-based sentiment analysis thus leverages these predefined sentiment scores to evaluate the polarity and intensity of sentences, documents, and other text units. In contrast, supervised learning-based sentiment analysis employs machine learning techniques including random forest, support vector machines, and artificial neural networks to classify text sentiment. This approach utilizes a training dataset that has been pre-labeled by managers, allowing for sentiment classification based on learned patterns within the data.

Lexicon-based sentiment analysis is a method that utilizes a highly reliable sentiment lexicon, constructed through the collective intelligence of diverse experiences and knowledge, to assign sentiment scores to individual words [33,34]. While building a lexicon through collective input offers robust reliability, it inherently limits the lexicon to words with clear polarity, typically adjectives, adverbs, and verbs. However, if experts with domain-specific knowledge carefully select and score words based on perceived polarity, the resulting lexicon achieves greater accuracy and reliability. Developing a lexicon with input from practitioners in a specific field, such as the construction industry, can yield highly relevant sentiment analysis tools. This approach, however, requires considerable investment of time and resources. When constructing a lexicon by reviewing individual sentences and assigning sentiment scores, this approach proves efficient for sentiment analysis of text that includes opinions or evaluations related to specific issues. Furthermore, when applying sentiment scores at the document level, scoring methods may vary according to the domain, allowing for tailored analysis across different contexts [31].

Performance analysis of construction sites using a sentiment lexicon presents significant challenges, including the difficulty of constructing a reliable lexicon from field-generated documents and the complexity of extracting text segments that accurately reflect emotional polarity. This study addresses these challenges by extracting directly or indirectly related to construction quality performance from supervision documents, which comprehensively capture inspection and management activities across construction phases. This text data serves as the foundation for building a sentiment lexicon to evaluate construction quality performance. This approach addresses the limitations of relying solely on structured data such as institutionally mandated metrics like construction and material quality failure rates by providing a nuanced grading of quality performance levels. By deriving differentiated quality performance insights from inspection and management activities, this model is expected to enhance the efficiency of subsequent site management practices. For example, typical quality indicators from construction supervision documents include material inspection failure rate, material rejection rate, and adequacy of inspection management, which, although institutionally required, may not comprehensively capture overall quality performance. Utilizing unstructured data from construction supervision documents, which contain detailed information on inspection and management outcomes, allows for a text-centered analysis. By integrating data from existing legal requirements, we propose a three-dimensional approach to quality performance analysis, supporting informed decision-making for site management. The model’s effectiveness is validated by comparing quantitative quality performance metrics based on legal standards with the unstructured data-derived evaluations as presented in this study.

3.2.2. Building a Sentiment Lexicon

In this study, the sentiment lexicon (SL) serves as a dictionary that categorizes the polarity of words related to construction site quality performance such as positive or negative, and assigns a score reflecting the intensity of each word’s polarity. This enables the quantification of polarity and intensity in sentences and documents through sentiment analysis based on the SL. By applying the sentiment scores embedded within the lexicon, this study quantifies the quality performance of material and construction inspection and management activities, ultimately providing a measure of quality performance levels.

A sentiment lexicon can be constructed through various approaches, including the manual approach, dictionary-based approach, corpus-based approach, pointwise mutual information (PMI), and frequency of polarity sentences-based lexicon [30]. The manual approach involves researchers themselves compiling sentiment words, relying on the expertise and knowledge of domain specialists. This approach requires the careful selection of words that convey emotions pertinent to a specific topic, assigning polarity and intensity values accordingly. While this process is labor-intensive and time-consuming, it offers high reliability and accuracy. The dictionary-based approach builds a lexicon by extending an existing lexicon with newly gathered sentiment words related to the target domain. This method identifies polarity and intensity in an established lexicon; as new sentiment words are extracted from relevant texts, it checks for their antonyms, synonyms, and related terms within the existing lexicon, assigning scores as needed. Although this approach requires high-quality pre-existing lexicons and alignment with the target domain, it is relatively concise and intuitive. The corpus-based approach employs machine learning or statistical methods to automatically extract sentiment words along with their polarity and intensity from a text corpus. This method can reduce manual effort, especially through unsupervised learning if the corpus is large enough. However, selecting appropriate training data and algorithms is essential to accurately classify sentiment words.

This study employs both the manual and corpus-based approaches to construct a sentiment lexicon tailored to evaluate quality performance at construction sites. The construction supervision documents analyzed are specialized technical documents, and the sentiment scores assigned to words in general lexicons differ in polarity and intensity from those suitable for this study’s context. Existing sentiment lexicons, often used in prior research, rely on generalized sentiment scores based on expertise and knowledge, which may not fully capture the nuances within construction supervision documents derived from inspection and management activities. To address these limitations, this study incorporates domain-specific knowledge related to construction quality, assigning sentiment scores to words and sentences individually to build an effective lexicon. While a dictionary-based approach typically requires an existing lexicon as a foundation, no suitable lexicon currently exists for construction quality performance evaluation. Consequently, this study pioneers a sentiment lexicon development process specifically informed by construction expertise.

To develop a lexicon for evaluating construction quality performance, a group of practicing experts assign sentiment scores to words extracted from construction supervision documents. The manual approach involves experts with field experience and knowledge assigning sentiment scores to words and sentences, thereby creating a lexicon with a high degree of reliability based on expert insight. However, this method may introduce subjective biases due to individual experts’ experiences. To ensure consistency, consultation among multiple experts is necessary when assigning sentiment scores. The PMI is used to assess the probabilistic association between two words, based on the premise that words with similar sentiment polarity are more likely to co-occur within a document. PMI-based sentiment scores represent the relative relationships between words, which poses a challenge in establishing absolute sentiment scores or thresholds. Despite this, PMI is particularly effective for constructing domain-specific sentiment lexicons because it draws on statistical information directly from the data. PMI is calculated by comparing the probability of two words co-occurring within a document with the probability of each word appearing independently, as shown in Equation (1). w₁ and w₂ represent random variables denoting the presence of two words for which we aim to measure the degree of association, and PMI(w₁, w₂) is the probability that these words are involved in the same sentence. This association is quantified as follows: a value of 0 indicates statistical independence between the occurrences of the two words within a document; a positive value signifies a positive correlation; and a negative value indicates a negative correlation. While PMI offers valuable insights into word associations, its susceptibility to fluctuations based on the choice of reference words presents a significant drawback. To address this limitation, this study employs the Semantic Orientation-Pointwise Mutual Information (SO-PMI) method, as illustrated in Equation (2), to assign sentiment scores to individual words [35]. PW and NW indicate, respectively, a group of positive and negative words. If the score of SO-PMI is high, ‘w’ word is close to a group of positive words.

$$\mathrm{PMI}\left(w_1,w_2\right)=\log {\displaystyle \frac{p\left(w_1,w_2\right)}{p\left(w_1\right)p\left(w_2\right)}}$$

(1)

$$\mathrm{SO}-\mathrm{PMI}\left(\mathrm{w}\right)={{\displaystyle \sum }}_{pw\in PW}PMI\left(w,pw\right)-{{\displaystyle \sum }}_{nw\in NW}PMI\left(w,nw\right)$$

(2)

This study adopts an efficient and intuitive approach to building a sentiment lexicon and involves analyzing the average frequency of specific words within documents or sentences of a particular polarity. While this methodology is traditionally employed at the document level [31], the research adapts it for sentence-level analysis. The calculation process of a sentence score begins with the construction of an adjacency matrix, as formalized in Equation (3), where an entry is set to 1 if a word ‘t’ appears in a sentence ‘s’ and 0 otherwise. Subsequently, the sentence score for each word is calculated by multiplying word (t, s) by label (s), as depicted in Equation (4). This product is then normalized by dividing each element by the corresponding word’s frequency across the entire document corpus.

$$word\left(t,s\right) = \left\{\begin{array}{cc}1\hfill & : If the word ‘t’ is included in sentence ‘s’\\ 0\hfill & : Otherwise\end{array}\right.$$

(3)

$$\mathrm{word}-\mathrm{score}\left(\mathrm{t}\right)={\displaystyle \frac{{\sum }_s\left(word\left(t,s\right)\times label\left(s\right)\right)}{{\sum }_{s}word\left(t,s\right)}}$$

(4)

$$\begin{array}{c}label\left(s\right)=\\ \left\{\begin{array}{c}0 : Sentence ‘s’ that is negatively expressed about quality performance\\ 1 : Sentence ‘s’ that is irrelavant or ambiguous to quality performance\\ 2 : Sentence ‘s’ that is positively expressed about quality performance\end{array}\right.\end{array}$$

(5)

This utilizes a sentence-by-sentence labeled dataset, assigning sentiment scores to individual words based on the average sentiment polarity of the contexts in which they appear. With a simple formula for sentiment score calculation, this is well-suited for application to large datasets and enables the efficient construction of domain-specific sentiment lexicons. However, the method’s accuracy depends significantly on the quality and scale of the labeled dataset. Therefore, it is essential that both lexicon development and labeling are performed by a group of experienced practitioners to minimize labeling errors and prevent potential over- or underestimations. In this study, labeling was conducted by experts with extensive knowledge of construction quality control and management. For instance, the word “change” was classified as negative, as it typically denotes an alteration due to defects or unmet specifications—such as material substitutions or modifications to design drawings at the construction site.

4. Development of SL-Based Quality Performance Evaluation Model (QPEM)

4.1. An Overview of QPEM

To construct the sentiment lexicon, we utilized construction supervision documents from 30 construction sites. These documents include both quantitative data, such as the number of defective materials and inspection failures, and qualitative descriptions detailing managerial actions and supervisory behavior. Text extraction is necessary to identify and select only the quality performance-related keywords embedded in documents and was conducted using Python version 3.9. Data preprocessing and tokenization of the extracted text were completed with the Okt module from the Python KoNLPy library. The Okt module facilitates tokenization without breaking sentences into detailed parts of speech, offering ease of use and high execution speed. In the utilization of textual data, tokenization plays a crucial role by defining the basic units of analysis and assigning unique indices to each token. We conducted a frequency analysis of tokenized keywords, excluding non-relevant words. Subsequently, we created two data forms. First, extracted words were reviewed by practicing experts, who assigned sentiment scores based on a consensus, creating a lexicon consisting of a list labeled with sentiment scores. Labeling plays a vital role in clearly defining the dataset for the consistent and reasonable quality performance evaluation, which in turn helps mitigate noise and errors in the data. Second, we constructed a Boolean dataset, setting each token to 1 if it appears in a sentence and 0 if it does not. This sentence-level labeled data enabled us to develop a lexicon based on keyword frequency within sentences. This study assesses construction quality performance using both lexicon forms and compares the results to propose a quality performance evaluation model that adapts to the lexicon form, as illustrated in Figure 1.

4.2. Manual Approach-Based Lexicon for Quality Performance Evaluation

Construction quality performance is primarily influenced by the quality of materials and construction practices. To develop a reliable sentiment lexicon for the performance evaluation model proposed in this study, text was extracted from construction supervision documents, and sentiment scores were assigned to words (nouns and adjectives) appearing in at least two sentences within the text. Each word or sentence was evaluated based on its perceived impact on quality performance: a score of 2 was assigned for a positive impact, 1 for an ambiguous or negligible impact, and 0 for a negative impact. The sentiment scores and their respective weights for the 291 words included in the lexicon are presented in Figure 2 and

Table 3. Comparing sentiment scores of words and sentences consisting of the lexicons.

Sentiment Score	Word Basis		Sentence Basis
	Number	Proportion	Number	Proportion
0 (Negative)	35	12.0%	104	24.1%
1 (Neutral)	164	56.4%	180	41.7%
2 (Positive)	92	31.6%	148	34.3%
Sum	291	100%	432	100%

.

Table 4. Labeling sentiment scores assigned to the top 40 most frequent words.

Frequency Rank	Word	Label	Frequency Rank	Word	Label
1	Management	1	21	Occurrence	0
2	Construction	1	22	Safety	1
3	Quality	1	23	Approval	1
4	Process	1	24	Supervision	1
5	Building	1	25	Securing	1
6	Material	1	26	Completion	1
7	Site	1	27	Finish	1
8	Plan	2	28	Use	1
9	Check	1	29	Project	1
10	Confirmation	2	30	Execution	2
11	Test	1	31	Building	1
12	Implementation	1	32	Comprehensive	1
13	Inspection	1	33	Presence/absence	1
14	Progress	1	34	Carrying-in	1
15	Item	1	35	Technology	1
16	Preliminary	2	36	Material check	2
17	Work type	1	37	Effort	2
18	Work	1	38	Construction	1
19	Change	0	39	Defect	0
20	Design	1	40	Regarding	1

presents the sentiment scores of the top 40 most frequently appearing words. The term “management” appears with the highest frequency, as construction supervision documents comprehensively record inspection and management activities across the construction site, followed by “construction,” “quality,” “process,”, “building”, “materials,” “site,” and “plan”, in that order. Due to the challenge of assigning a clear polarity (0 or 2) to high-frequency terms, these words were assigned a neutral score of 1, reflecting an ambiguous or non-specific impact on quality performance. Within the top 40 most frequent keywords, terms like “change,” “occurrence,” and “fault” were scored as 0, indicating a negative impact on quality performance, while words such as “plan,” “confirmation”, “preliminary”, “inspection,” and “effort” were scored as 2, indicating a positive impact. In total, sentiment scores were assigned to 291 words, including these high-frequency terms, to construct a comprehensive sentiment lexicon.

4.3. Sentiment Lexicon by Sentence-Based Dataset

When sentiment scores are assigned at the sentence level, sentences are evaluated based on their impact on construction quality performance: 2 points for a positive impact, 1 point for an ambiguous or neutral impact, and 0 points for a negative impact. The distribution of these labeling values is presented in Figure 3. From construction supervision documents across 30 construction sites, 432 sentences deemed relevant to quality performance were extracted, and their sentiment score distribution is displayed. Compared to the word-based labels, sentence-based labels are more evenly spread, with 35.3% of sentences indicating positive performance and 22.7% indicating negative performance, as presented in Figure 3.

The sentiment score for each word is calculated using Equations (3)–(5) based on labeled sentences. For instance,

Table 5. Example of sentiment scores of words derived from sentence-level sentiment scores.

$\mathbf{Sentence}(\text{'}\mathbf{s}\text{'})$ ID	Sentences	Sentiment Score	Presence/Absence of ‘Change’
1	Due to discrepancies in the ground investigation, the foundation was changed.	0	1
2	To secure the highest quality of the apartment complex and enhance resident satisfaction, the construction method was changed.	2	1
3	Through pre-construction planning, including the deployment of personnel, materials, and equipment on site, along with systematic construction management and schedule control, solid construction was ensured, and high quality was achieved.	2	0
4	Due to civil complaints, a change in construction method was unavoidable.	0	1
5	The overall planned schedule submitted with the groundbreaking report was applied to construction management, reflecting the as-built drawings and site conditions.	1	0

demonstrates the sentiment score calculation for the keyword “change,” utilizing the sentiment score assigned to each sentence in relation to quality performance and the frequency of the word’s occurrence. In this example, the sentence-level sentiment labels are 0, 2, 2, 0, 1, with occurrences of “change” represented as 1, 1, 0, 1 and 0, respectively. The sentiment score for the keyword “change” is computed using Equation (6), where the numerator is derived from Equation (7) and the denominator from Equation (8). This yields a sentiment score of 0.67 for “change,” as shown in Equation (9).

$$\mathrm{word}-\mathrm{score}\left(\text{‘}\mathrm{change}\text{’}\right)={\displaystyle \frac{{\sum }_{s=1}^5\left(word\left(\text{‘}change\text{’},s\right)\times label\left(s\right)\right)}{{\sum }_{s=1}^{5}word\left(\text{‘}change\text{’},s\right)}}$$

(6)

$${{\displaystyle \sum }}_{s=1}^5\left(word\left(\text{‘}change\text{’},s\right)\times label\left(s\right)\right)=0\times 1+2\times 1+2\times 0+0\times 1+1\times 0=2$$

(7)

$${{\displaystyle \sum }}_{s=1}^{5}label\left(s\right)=1+1+0+1+0=3$$

(8)

$$\mathrm{word}-\mathrm{score}\left(\text{‘}\mathrm{change}\text{’}\right)={\displaystyle \frac{2}{3}}=0.67$$

(9)

The sentiment scores for the 291 words related to quality performance in the supervision documents were calculated based on sentence-level labels.

Table 6. Sentiment scores of the top 40 most frequent words based on sentence-level scores.

Frequency Rank	Word	Sentiment Score	Frequency Rank	Word	Sentiment Score
1	Thoroughness	2.00	21	Request	1.68
2	Law	2.00	22	Good Quality	1.68
3	Emergency	2.00	23	Continuously	1.67
4	Superior	2.00	24	Applicant	1.67
5	Major	1.90	25	Goal	1.67
6	Supplier	1.89	26	Specifications	1.67
7	Target	1.88	27	Presence	1.64
8	Highest	1.86	28	Guidance	1.64
9	Institute	1.83	29	Manpower	1.64
10	Satisfaction	1.83	30	In-detail	1.63
11	Heating	1.80	31	Goal	1.63
12	Relationship	1.80	32	Notice	1.63
13	Deficiency	1.80	33	Request form	1.63
14	Stakeholder	1.80	34	Focus	1.62
15	Apartment	1.78	35	Case	1.60
16	Specification document	1.71	36	Apartment unit number	1.60
17	Specialization	1.71	37	Restriction	1.60
18	Perfection	1.71	38	Traffic	1.60
19	Completeness	1.71	39	Entrance	1.60
20	System	1.69	40	Same	1.60

presents the sentiment scores for the top 40 words derived through this process. Words such as “thoroughness,” “law,” “emergency,” and “superior” received high sentiment scores of 2, as they exclusively appear in sentences where inspection and control activities reflect positively on construction site quality. Additionally, words with inherently positive connotations, including “highest,” “satisfaction,” “good quality,” and “in-depth,” also exhibit high sentiment scores.

Table 7. Sentiment scores of the bottom 40 most frequent words based on sentence-level scores.

Frequency Rank	Word	Sentiment Score	Frequency Rank	Word	Sentiment Score
1	Delay	0.00	21	Request	0.33
2	Preliminary	0.00	22	Change	0.39
3	Crack	0.00	23	Analysis	0.40
4	Expectation	0.00	24	Electricity	0.43
5	Majority	0.00	25	Match	0.44
6	Unstable ground	0.00	26	Problem	0.50
7	Omission	0.00	27	Contract	0.50
8	Case	0.00	28	Opinion	0.50
9	Contracting	0.00	29	Settlement	0.50
10	Slight	0.00	30	Korea	0.50
11	Difficulty	0.08	31	Support	0.50
12	Delay	0.11	32	Supply	0.50
13	Supply and demand	0.17	33	Intervention	0.50
14	Newly constructed	0.20	34	Company	0.54
15	Addition	0.20	35	Machine	0.57
16	Slightly	0.22	36	Concentration	0.57
17	Fire protection	0.25	37	Discussion	0.58
18	Investigation	0.25	38	Details	0.60
19	Adjustment	0.31	39	Resignation	0.60
20	Need	0.33	40	Time frame	0.60

presents the top 40 words ranked by respective sentiment scores. Notably, terms such as “difficulty,” “slightly,” “delay,” and “preliminary,” which often carry negative connotations in the context of construction quality, are assigned lower sentiment scores. The distribution of sentiment scores for individual words, derived from sentence-level labels, is depicted in Figure 4. The construction of a performance-positive sentiment lexicon, centered around scores of 1 to 1.6, reflects the fact that the 30 construction sites were satisfied with the mandated legal standards.

5. Applications of SL-Based QPEM

The SL-based Quality Performance Evaluation Model (QPEM) developed in this study processes and structures unstructured text data from documents generated at construction sites. Given the high proportion of text-based unstructured data in quality performance inspection and management records, accurate labeling of words and sentences is essential for QPEM application. The sentiment score for a sentence related to quality performance within a supervision document is calculated using Equation (11). The average sentiment score is calculated by summing the sentiment scores of all words in a sentence and dividing by the number of distinct words. Subsequently, as shown in Equation (12), the overall quality performance level for a given construction site is determined by aggregating the sentiment scores for all sentences within the documents and dividing by the total number of sentences.

$$match\left(t,s\right) = \left\{\begin{array}{cc}1\hfill & : If the word ‘t’ is included in sentence ‘s’\\ 0\hfill & : Otherwise\end{array}\right.$$

(10)

$$\mathrm{sentence}-\mathrm{score}\left(\mathrm{s}\right)={\displaystyle \frac{{\sum }_t\left(match\left(t,s\right)\times \left(word-score\left(t\right)\right)\right)}{{\sum }_{t}match\left(t,s\right)}}$$

(11)

$$\mathrm{document}-\mathrm{score}\left(\mathrm{d}\right)={\displaystyle \frac{{\sum }_{s\in d}\left(sentence-score\left(s\right)\right)}{S\left(d\right)}}$$

(12)

$\mathrm{S}\left(\mathrm{d}\right) :\mathrm{the} \mathrm{number}\mathrm{of}\mathrm{sentences}\mathrm{appearing}\mathrm{in}\mathrm{the}\mathrm{construction}\mathrm{supervision}\mathrm{document}$

5.1. Comparison of Sentiment Scores Oriented from Words and Sentences

Figure 5 illustrates the distribution of sentiment scores for both words and sentences after constructing a lexicon based on terms and sentences related to quality performance across the 30 selected construction sites. In both distributions, the mean sentiment score for sentences is approximately 1.2. Compared to sentence-based sentiment scores, the word-based sentiment dictionary exhibits greater variation, with scores extending up to the maximum value of 2.

5.2. Illustrative Cases

When the proposed SL-based QPEM is applied to an individual construction site, the average sentiment score of sentences related to quality performance in the supervision documents quantifies the site’s overall quality performance level. To derive the SL-based QPEM results for a specific site, sentences and words from the actual construction supervision documents are extracted, as illustrated in

Table 8. Illustrative computation of lexicon-based sentiment scores.

Sentence ‘a’ in ‘A’ Project	Labeled Words	Sentiment Score of Words	Sentence-Level Basis Computation	Word-Level Basis Computation
The overall planned schedule submitted with the groundbreaking report was revised (supplemented and adjusted) to reflect the as-built drawings and site conditions, and then applied to construction management after approval by the supervision team	Groundbreaking	0.85	1.07	1.06
	Submission	1.19
	Overall	0.92
	Planned	0.86
	Scheduled	1.00
	Implementation	1.50
	Drawings	1.00
	Site	1.45
	Condition	0.93
	Reflection	0.63
	Revised	0.63
	Written	1.36
	Supervision	1.52
	Approval	1.34
	Process	1.08
	Management	1.26
	Application	0.67

. The sentiment dictionary’s labeled list of words and sentences is then used to estimate the quality performance level of a specific construction site.

In calculating the sentiment score for quality performance in sentence “a”, the numerator represents the sum of sentiment scores for labeled words within sentence “a,” while the denominator is the total number of labeled words. The sentiment score of each labeled word is determined based on sentence-level scores, as specified in Equation (9). In cases where multiple labeled words appear in a sentence, each word’s sentiment score is included based on its presence rather than frequency. For example, the sentiment score of the word “quality” is calculated by noting whether it appears in the sentence, regardless of repetition, in quality-related contexts like “Strictly enforcing quality tests increases quality performance.” Once the sentiment score of “quality” is determined, the sentence-level evaluation of quality performance inspection and management activities is finalized.

In this process, all sentences related to quality performance within the construction supervision documents generated at the construction site are assigned sentiment scores. The average of these scores represents the quality performance level based on unstructured data. The process and outcomes of evaluating the quality performance level for construction site “A” are summarized in

Table 9. Sentiment scores of evaluations resulted from quality inspection and management for construction project ‘A’.

Sentence ID	Words Given by Sentiment Score Among Words Appearing in Sentences	Sentiment Score by Sentence
		Sentence-Level Basis	Word-Level Basis
1	groundbreaking, submission, overall, planned, schedule, implementation, drawing, site, condition, reflection, revised, written, supervision, approval, process, management, application	1.07	1.06
2	inspection, testing, applicant, preliminary, quality, process, schedule, plan, implementation	1.42	1.33
3	completion, phase, field, supervision, site, inspection, indication, item, measure, condition, confirmation	1.26	0.91
4	by work type, construction, commencement, plan, details, supervision, review, approval, work, smooth	1.25	1.30
5	work type, by phase, item, quality, securing, request form, written, supervision, confirmation, subsequent, process, process, thorough, implementation	1.38	1.13
6	construction, civil engineering, facility, construction, reciprocal, work, building, work type, cooperation, order, manpower, use, material, placement, equipment, input, adequacy, review, appropriate, adjustment	1.12	1.05
7	rebar, construction, waterproof, heating, package, focus, quality, management, target, work type, preliminary, selection, major, work, supervision, presence, confirmation, process	1.47	1.39
8	quality, test, plan, site, operation, external, request, implementation	1.47	1.13
9	work type, quality, management, test, plan, establishment, building, preparation, work, finish, construction, measure, review, site, confirmation, inspection, thorough, implementation	1.28	1.33
10	site, carrying-in, material, use, specification document, specification, confirmation, supplier, approval, inspection, thorough, defect, management	1.46	1.14
11	site, input, manpower, material, equipment, thorough, preliminary, plan, establishment, system, management, implementation, quality, securing	1.44	1.33
12	quality, management, necessary, site, test, implementation, confirmation, use, material, inspection, precision, thoroughness, building, system	1.42	1.29
13	site, neighboring, entrance, traffic, control, continuously, placement, neighboring, resident, safety, problem, noise, dust, minimum, measure, complaint, occurrence, factor, preliminary, perfection	1.38	1.00
Average		1.34	1.18

.

Table 10. Results from applying SL-based QPEM into 30 sites satisfying legal standards.

Construction Site ID	Sentence-Based Sentiment Score	Word-Based Sentiment Score	Construction Site ID	Sentence-Based Sentiment Score	Word-Based Sentiment Score
1	1.34	1.18	16	0.73	0.92
2	1.33	1.21	17	1.17	1.15
3	1.11	1.21	18	1.16	1.15
4	1.34	1.19	19	1.20	1.17
5	1.20	1.22	20	1.22	1.16
6	1.14	1.17	21	0.93	1.19
7	1.22	1.26	22	0.91	1.04
8	1.28	l1.22	23	0.98	1.02
9	1.29	1.16	24	1.10	1.07
10	1.11	1.20	25	0.99	1.17
11	1.26	1.28	26	1.02	1.16
12	1.21	1.28	27	1.11	1.19
13	1.13	1.18	28	0.86	1.06
14	1.29	1.22	29	1.29	1.22
15	1.22	1.24	30	0.96	1.28

and Figure 6 present the evaluation results of inspection and management activities related to quality performance, derived from the construction supervision documents of 30 construction sites using the sentiment lexicon. The sentence-based sentiment lexicon exhibits a wider distribution than the word-based lexicon, effectively differentiating quality performance levels across sites. Unlike existing legal standards, which provide binary “conformity” or “non-conformity” assessments, the SL-based QPEM proposed in this study offers continuous measurement of quality performance levels based on inspection and management activities conducted at construction sites. Furthermore, even if a construction site is satisfied with the required institutional standards, QPEM defines cases with a decision boundary below ‘1(neutral)’ as low performance, and is utilized to make a decision for additional quality management activities. Thus, as shown in Table 10, the quality performance levels of 30 construction sites meeting all legal standards can be differentiated. This allows for the identification of specific inspection and management activities that impact quality performance, thereby enhancing the efficiency of quality management for future construction sites.

Table 11. Correlation between actual values of legalized quality check items and QPEM outcomes for 30 construction sites.

Legalized Quality Check Items	SL-Based Quality Performance Results Based on Sentences		SL-Based Quality Performance Results Based on Words
	Correlation Coefficient	p-Value	Correlation Coefficient	p-Value
Quality test failure rate	−0.39	<0.001	−0.6	<0.001
Material inspection failure rate	−0.027	0.018	−0.24	0.021
Adequacy of inspection management	−0.17	0.024	−0.16	0.012

presents the correlation between quantitative quality performance indicators—such as quality test failure rate, material inspection failure rate, and adequacy of inspection management—collected from 30 construction sites that meet legal standards and the results generated by the SL-based QPEM. Analysis of Pearson correlation coefficients reveals that QPEM results, derived from sentiment dictionaries labeled at both the word and sentence levels, are negatively correlated with institutionally mandated quantitative quality performance data. Notably, the SL-based QPEM demonstrates a relatively high negative correlation with quality test failure rate data, suggesting that higher QPEM scores correspond to lower quality test failure rates. This indicates that the SL-based QPEM effectively leverages document-centered unstructured data, enhancing the model’s efficiency. In practical terms, a higher SL-based QPEM score, reflecting a more positive performance evaluation, aligns with lower failure rates in quality tests and inspection materials—key determinants of a construction site’s quality performance. Thus, this model can facilitate informed decision-making on quality performance by utilizing unstructured data from inspection and management activities throughout the construction process.

6. Conclusions

In response to recent construction site accidents attributed to inadequate quality inspection, management, and delayed responses, regulatory requirements have been strengthened, emphasizing the need for proactive monitoring of structural performance, material quality, and overall construction quality, along with reactive quality performance evaluation. Objective, expertise-based assessments by legally designated managers on-site can help differentiate quality performance levels and support proactive measures in future construction projects. Although construction supervision documents contain unstructured data related to quality performance, this resource remains underutilized.

Research on construction quality performance has traditionally relied on structured data generated through institutional standards and conventional statistical techniques. However, these approaches face limitations in effectively evaluating and differentiating quality performance levels due to issues such as data privacy concerns, errors, and missing values. Addressing these limitations, document-centered unstructured data utilization is gaining traction as an effective methodology, with approximately 80% of data generated on construction sites being unstructured. This approach holds the potential for enhancing the limited efficacy of structured data in quality assessment. The SL-based Quality Performance Evaluation Model (SL-based QPEM) proposed in this study aims to enhance decision-making efficiency by incorporating unstructured data into the traditionally structured-data-focused quality performance evaluations. The development of SL-based QPEM involves constructing a sentiment lexicon to extract quality-related terms from documents, followed by text preprocessing with tokenization, frequency analysis, and subsequent evaluation of quality performance levels. This process combines manual and corpus-based approaches, supported by a panel of experts.

Structured data, such as material failure counts and inspection failures from 30 construction sites, as well as unstructured text data including construction plans, action measures, and reported quality statuses were utilized to develop and compare models for the SL-QPEM. Text extracted through text mining was tokenized based on frequency analysis. Words were categorized into nouns, adjectives, and adverbs, with sentences streamlined to retain only meaningful words. This process produced 291 and 432 datasets, respectively, which were then labeled by experienced construction inspection and management experts as 0 (negative), 1 (neutral), or 2 (positive). These labeled datasets were subsequently applied to QPEM for analysis. Upon applying the sentiment lexicon-centered quality performance evaluation model, both word- and sentence-based approaches were tested across 30 construction sites. Results showed that sentence-based sentiment scores for quality performance tended to normalize more consistently than word-based scores, indicating a high level of model reliability. The average sentiment scores for quality performance were 1.19 for sentences and 1.17 for words, suggesting a positive level of quality performance. However, examining the sentiment score distribution revealed variations, allowing for a quantitative differentiation in quality performance levels even among construction sites meeting legal standards. Correlations between quality management indicators, namely, quality test failure rate, material inspection failure rate, and adequacy of inspection management and SL-based QPEM results, were all negative, underscoring the model’s significance. When utilizing the word-based sentiment lexicon, the sentiment score for quality performance demonstrated sensitivity to these quality management indicators, reflected in negative correlations. This suggests that the SL-based QPEM can be selectively applied based on the specific characteristics and requirements.

With the increasing application of digital technologies at construction sites, the volume of unstructured data is expanding rapidly. While the use of structured data continues to grow, the integration of unstructured data remains limited. The document-centered, unstructured data-based quality performance evaluation model proposed in this study serves as a tool to facilitate timely quality inspection and management, supporting efficient decision-making processes. The construction of a text-based sentiment lexicon allows for the consistent use of information generated by site managers to classify fluctuations and post-grade quality performance levels. This approach is particularly effective for planning proactive quality inspection and management activities at construction sites. Furthermore, the model can be integrated with structured and unstructured data related to various performance indicators such as air quality, safety, cost, and environmental impact generated at construction sites. This integration is anticipated to enhance and complement existing quality performance evaluation tools, which are predominantly structured-data-based.