Hybrid Analytic Hierarchy Process–Artificial Neural Network Model for Predicting the Major Risks and Quality of Taiwanese Construction Projects

Round 1

Reviewer 1 Report

This is an interesting paper trying to integrate MCDM method with ML method. Although it could contribute novelty to the MCDM research field, there are several issues that must be improved before accepting to publication.

1. The selection and application of AHP to the study require more supportive reasons. There are other MCDM methods that can be used with ML method. Why this work must use AHP? Please provide a strong supportive reason to this part.

2. There are many types of Likert scales. Why did this study apply 5-level Likert scale?

3. The selection of major risk depends on the score which must be higher than 994. Why do we need to depend on this score? 

4. Regarding the samples, why did the study require various samples ranging from low experience to high experience (such as tenure in the industry) not only the experts?

5. To emphasize the significance of non-financial factor selection or prioritization using the MCDM method as the proposed approach of study, therefore, some related studies should be mentioned including DOI: 10.1108/14691930910952669, DOI: 10.1109/ECTIDAMTNCON53731.2022.9720385, DOI: 10.3390/math10040626

6. This work still lacks the discussion part. It should be better to discuss more on the application and results obtained from ANN application. Normally, past works applied MCDM methods to select the factors instead of ML or ANN. What are the advantages and disadvantages obtained from the proposed approach compared to other past related works? In my opinion, the method may provide benefits, but it possibly compensates for some weaknesses in some dimensios. The discussion on this part may help the readers realize the practical implication of proposed method. 

7. The limitation of work should be added. The selection of factors depends on the decision of experts or decision-makers, and ML learned from those decisions. This notice should be also mentioned.

 

Author Response

• The selection and application of AHP to the study require more supportive reasons. There are other MCDM methods that can be used with ML method. Why this work must use AHP? Please provide a strong supportive reason to this part.

Response:

Thank you for your comments. We have strengthened the reasons for using AHP on the reviewer's recommendations (in lines 129-139 on page 3).

The AHP systematizes complex problems through a hierarchical structure. It divides decision-making elements into multiple dimensions; hierarchically decomposes and structures a problem from multiple dimensions to divide a large, complex problem into multiple small subproblems; and assesses these subproblems individually. This process simplifies the decision-making process for complex problems. In contrast to the multi-criteria decision-making component of the Simple Multi-Attribute Rating Technique (SMART), which adopts a direct rating model, the AHP constructs a pairwise comparison matrix by conducting a pairwise comparison of attributes to determine the weights between criteria. The SMART only evaluates a single attribute, whereas the AHP conducts a pairwise comparison to provide decision makers with a basis for comparing and improving the validity of their models and decisions through consistency tests.

• There are many types of Likert scales. Why did this study apply 5-level Likert scale?

Response:

Thank you for your comments. We have explained why a 5-level Likert scale is used in the text (in lines 472-480 on page 11).

The point range of the Likert scale can be increased to 9 points or reduced to 2 points. However, having an excessive number of scale points may increase difficulty of completing the questionnaire for participants, and having an insufficient number of scale points may prevent the collected data from fully expressing the various degrees of participant intention. Given that 46 risk factors were rated in the questionnaire, a 5-point Likert scale was selected to ensure that the participants were not deterred from completing the questionnaire because of its lengthiness and to increase the validity of the questionnaire.

• The selection of major risk depends on the score which must be higher than 994. Why do we need to depend on this score?

Response:

Thank you for your comments. We have added explanations using scores greater than 994 to strengthen confidence (in lines 511 to 515 on page 11 and in line 517 on page 13).

Subsequently, a five-level histogram was mapped on the basis of the Likert scale scores and the 46 risk factors (Figure 3), and the top two-fifths of the risk factors (i.e., those with a score of >994) were selected as the major risk factors.

• Regarding the samples, why did the study require various samples ranging from low experience to high experience (such as tenure in the industry) not only the experts?

Response:

Thank you for your comments. To refine the questionnaire survey, we've added varied samples ranging from low experience to high experience. (in lines 490-496 on page 11).

The construction auditing mechanism is a top–down quality management process. The government establishes the format of the inspection form, experts conduct on-site auditing, and project-related personnel implement construction and management operations. Therefore, 46 risk factors were selected from the five risk dimensions defined by experts and researchers. To identify the major risk factors, project-related and experienced personnel were invited to participate in the questionnaire survey. This approach ensured that the survey results were consistent with actual construction practices.

• To emphasize the significance of non-financial factor selection or prioritization using the MCDM method as the proposed approach of study, therefore, some related studies should be mentioned including DOI: 10.1108/14691930910952669, DOI: 10.1109/ECTIDAMTNCON53731.2022.9720385, DOI: 10.3390/math10040626

Response:

Thank you for your comments. We agree with your opinion and add more references related to the specific topic of study. Thus, we have modified our paper as referee comments and consider recent studies closely related to the MCDM method of prioritization assessment. (e.g. Kim and Kumar 2009; Wudhikarn and Pongpatcharatorntep 2022; Lu and Wudhikarn 2022). (in lines 96-98 on page 2 and lines 148-152 on page 3).

Therefore, developing performance indicators and establishing a prioritization framework can help managers focus on the key components of management and more effectively allocate limited resources within their organizations [13].

Moreover, employing multiple methods (Delphi and AHP approaches) which can overcome the limitation of a single methodology [17]. Hybrid decision science methods (ANP and quality function deployment) were integrated to improve the ability to consider relationships among the critical factors and their impact [18].

• This work still lacks the discussion part. It should be better to discuss more on the application and results obtained from ANN application. Normally, past works applied MCDM methods to select the factors instead of ML or ANN. What are the advantages and disadvantages obtained from the proposed approach compared to other past related works? In my opinion, the method may provide benefits, but it possibly compensates for some weaknesses in some dimensios. The discussion on this part may help the readers realize the practical implication of proposed method.

Response:

We admire your insight on practical situation. To support the rationale of AHP-ANN, we have added a discussion of the advantages and disadvantages of the method (in lines 643-649 on page 17 and lines 650-669 on page 18).

The AHP–ANN method provides several advantages; it addresses various weaknesses in several dimensions and simplifies the dimensions of a considerable number of risk factors to identify the major risk factors and estimate their influence. In addition, it can decompose complex problems one by one and establish a hierarchical structure comprising five risk dimensions, enabling managers to understand the attributes of the factors that affect quality and to effectively manage the major factors. When an empirical model is derived from a large volume of data and the mathematical framework of a system is unclear such that conventional statistical methods based on appropriate assumptions cannot be applied, an ANN-based prediction model becomes useful. However, because the training and model derivation process of an ANN is a black box, the ANN is at a disadvantage because it has difficulty explaining the logical reasoning and meaning of a model in accordance with the applied parameters.

In machine learning, a large volume of historical data are used to predict future actions or outcomes. In this context, prediction involves inputting the features of known variables or factors (e.g., importance) and classifying or regressing the output results. When an unknown set of variables is input into a machine learning model, the model can calculate the probability value on the basis of past experiences and further classify its results. An advantage of an ANN is that it introduces a nonlinear function as an activation function, which can approximate any function. That is, an ANN can produce distributions that approach the distribution of known variables. In the present study, construction auditing defects (major risk factors) reported between 1993 and 2020 in Taiwan were used as input variables, and auditing scores (project quality) were used as output results. When major risk factors affect a project, the prediction model can be employed to estimate the construction quality of the project. Although the accuracy of a prediction model can be verified using known training data, the classification of a prediction can only be obtained through the actual outcome in addition to observation and verification.

• The limitation of work should be added. The selection of factors depends on the decision of experts or decision-makers, and ML learned from those decisions. This notice should be also mentioned.

Response:

Thank you for your comments. We have added work limits to the Conclusion section in response to the reviewers' comments (in lines 722-728 on page 19).

In the present study, an ANN and construction auditing data were applied to predict construction quality outcomes; however, the selection of risk factors was dependent on experts or policymakers, which is a limitation of the study. Nevertheless, managers can still use the prediction results of the study in practical applications and improve their management through these results. Future studies should employ a deep learning model to automatically determine risk factors and adjust the weights and biases of these factors to obtain improved prediction results.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have used some methods, including survey, AHP, and ANN to predict the relationship between risk factors and quality. The manuscript is not suitable for publication as it has several flaws. The main concerns are 1) the need to employ AHP when 250 questionnaires have been distributed already in the previous stage of research and 2) the capability of ANN in predicting the relationship between risk factors and quality using the inputs that are only related to the risk factors and not quality. My specific comments are as follows:

In the abstract, lines 14, 15, what do you mean by risk assessment and management? Are such steps undertaken to reduce risks?

Line15-16, what do you mean by “the relationship between critical risk factors and construction quality is crucial?

What is the aim of the research? It should be highlighted in the abstract.

Lines 414-416 should be elaborated. How did the authors categorize and identify the risk factors? It must be justified; otherwise, the findings are not valid.

Inline 417, the authors have mentioned that Likert Scale questions have been given to

 

Line 442 and 443, no opinion, unimportant, and very unimportant are not correct. How can someone say that it is very unimportant when it is collected from the literature? It might have lower importance compared to others.

An important question is, why do the authors need to use AHP for ranking when a Likert scale questionnaire is already distributed, and the answers are collected? It seems AHP is not needed to be used at all as the weight of risk factors can be obtained from the Likert scale questionnaire.

Line593-594, these sentences are very general and not suitable for the conclusion.

Another important question is, who ANN can predict the relationships between risk and quality? Is there any data available related to the quality of construction projects faced with those risks? The reviewer doubt that using such data can result in the prediction of relationships between risk factors and quality.

Author Response

• In the abstract, lines 14, 15, what do you mean by risk assessment and management? Are such steps undertaken to reduce risks?

Response:

Thank you for your comments. In response to the reviewers' comments, we have added an explanation of risk assessment and management to the Introduction section (in lines 31-38 on page 1).

The uncertain hazards that are present during the construction process are referred to as risk factors. To ensure the efficient completion of construction projects, project managers generally implement a risk control mechanism. A risk control mechanism encompasses risk identification, risk assessment, and risk management. Project managers apply various methods to identify the relationship between risks and project quality (risk identification), quantify the harmfulness of risk factors (risk assessment), and manage major risk factors (risk management) through measures such as reinforcing the inspection of major risk factors.

• Line15-16, what do you mean by “the relationship between critical risk factors and construction quality is crucial?

Response:

Thank you for your comments. We have restated the relationship between critical risk factors to make the sentences clearer (in lines 15-17 on page 1).

Identifying risk factors and the relationship between major risk factors and the quality of construction projects facilitates construction management.

• What is the aim of the research? It should be highlighted in the abstract.

Response:

Thank you for your comments. We agree with your suggestion to illustrate the aim of the research in the abstract (in lines 20-25 on page 1).

The hybrid analytic hierarchy process (AHP) and an artificial neural network (ANN) were employed to develop a model for predicting major risk factors and construction quality. The AHP was used to calculate the weight of major risk factors to verify their influence on construction. The ANN was adopted to extract the features of major risk factors to predict the quality of a construction project.

• Lines 414-416 should be elaborated. How did the authors categorize and identify the risk factors? It must be justified; otherwise, the findings are not valid.

Response:

Thank you for your comments. We agree with your opinion and have added more detailed instructions on how to categorize and identify risk factors (in lines 436-442 on page 10).

The PCMIS a quality inspection scoring system for public works established by the Taiwan government in 1993. In this study, 948 construction auditing records from 1993 to 2020 were collected from the PCMIS. These records contain a total of 948 auditing scores, 499 defects items, and 9,596 defect frequencies. The experts were interviewed to identify five risk dimensions (46 risk factors) from PCMIS and a questionnaire survey (Likert scale) was per-formed among personnel working in construction-related fields to identify 19 major risk factors from five risk dimensions.

• Inline 417, the authors have mentioned that Likert Scale questions have been given to An important question is, why do the authors need to use AHP for ranking when a Likert scale questionnaire is already distributed, and the answers are collected? It seems AHP is not needed to be used at all as the weight of risk factors can be obtained from the Likert scale questionnaire.

Response:

We greatly appreciate your questions and providing insightful feedback on our manuscript. We added a more detailed explanation of why AHP is used (in lines 536-545 on page 13).

The weights (scores) of the risk factors were directly obtained through the Likert scale in the questionnaire. However, the relative effects between multiple factors in a given dimension were not considered. The AHP analyzed problems through a hierarchical structure and correlations, estimated the relative importance of the factors in a given layer (dimension), and conducted evaluations with a pairwise comparison matrix. In other words, two given factors in a layer were evaluated by using the factors of the previous layer as the evaluation standard for calculating the relative importance or contribution of the two factors. Because the factor weights obtained through the AHP were relative weights that ranged between 0 and 1 and add up to a sum of 1, the convergence speed and accuracy of the model could be increased when the weights were input into the ANN.

• Line 442 and 443, no opinion, unimportant, and very unimportant are not correct. How can someone say that it is very unimportant when it is collected from the literature? It might have lower importance compared to others.

Response:

Thank you for your comments. It was our mistake, and we redefine the meaning of Likert scale scores in this manuscript (in lines 470-473 on page 11).

In this study, a questionnaire survey was conducted on the major risks encountered by personnel involved in Taiwanese construction projects by using a 5-point Likert scale (5 = “Very important,” 4 = “Important,” 3 = “Neutral,” 2 = “Low importance,” and 1 = “Not at all important”).

• Line593-594, these sentences are very general and not suitable for the conclusion.

Response:

Thank you for your comments. We have checked the conclusion and deleted some of the unneeded sentences, such as: Therefore, in risk management decision-making, project managers must consider the importance of each risk and the effect of each risk on project quality to make optimal management decisions under limited resources……...

• Another important question is, who ANN can predict the relationships between risk and quality? Is there any data available related to the quality of construction projects faced with those risks? The reviewer doubt that using such data can result in the prediction of relationships between risk factors and quality.

Response:

Thank you for your comments. In response to the reviewer's comments, we have added an explanation of the machine learning predictions. (in lines 655-668 on page 18).

In machine learning, a large volume of historical data are used to predict future actions or outcomes. In this context, prediction involves inputting the features of known variables or factors (e.g., importance) and classifying or regressing the output results. When an unknown set of variables is input into a machine learning model, the model can calculate the probability value on the basis of past experiences and further classify its results. An advantage of an ANN is that it introduces a nonlinear function as an activation function, which can approximate any function. That is, an ANN can produce distributions that approach the distribution of known variables. In the present study, construction auditing defects (major risk factors) reported between 1993 and 2020 in Taiwan were used as input variables, and auditing scores (project quality) were used as output results. When major risk factors affect a project, the prediction model can be employed to estimate the construction quality of the project. Although the accuracy of a prediction model can be verified using known training data, the classification of a prediction can only be obtained through the actual outcome in addition to observation and verification.

Author Response File: Author Response.pdf

Reviewer 3 Report

Please see the attached file.

Comments for author File: Comments.docx

Author Response

• The abstract failed to present results.

Response:

Thank you for your comments. We have added the results of this study to the Abstract (in lines 20-25 on page 1).

The hybrid analytic hierarchy process (AHP) and an artificial neural network (ANN) were employed to develop a model for predicting major risk factors and construction quality. The AHP was used to calculate the weight of major risk factors to verify their influence on construction. The ANN was adopted to extract the features of major risk factors to predict the quality of a construction project. The accuracy of the prediction model was 85%.

• The manuscript contains multiple grammatical and editorial errors.

Response:

Thank you for your comments. We revised the manuscript to correct grammatical and editorial errors. We have re-examined the manuscript to avoid more writing errors. We believe that the overall quality increased after our careful revision. In addition, the paper has been edited by Wallace Academic Editing (Randy Johnson and Hayden Tay), and is considered to be improved in grammar, punctuation, spelling, verb usage, sentence structure, conciseness, general readability, writing style, and native English usage to the best of the editor's ability (The proof is shown in the attachment).

• In Fig. 2, it is suggested adding a short explanation in front of each word in each box. In this format, the process is not clear. Generally, this figure is not readable standalone.

Response:

Thank you for your comments. We agree with your suggestion to add a short explanation in the Fig. 2 (in lines 463-464 on page 20).

• What is the method of data collection: in line 412, it is mentioned that data was collected from PCMIS... While in line 421, it was mentioned “A Likert scale was used to perform a survey”?

Response:

Thank you for your comments. We have reinterpreted and explained this issue in the manuscript (in lines 670-677 on page 18 and in lines 436-442 on page 10).

To improve the quality of public construction, the Taiwanese government established a construction auditing system that implements regular auditing. Experts and researchers are employed to conduct on-site quality audits with a standardized checklist (499 defective items). For each audited project, one to three experts or researchers spend a day to identify the defects in the design, construction, and supervision of the project. Subsequently, they provide a rating on the basis of defect severity and actual construction conditions. The auditing results and identified defects are registered in the PCMIS by the construction agency.

The PCMIS is a quality inspection scoring system for public works established by the Taiwan government in 1993. In this study, 948 construction auditing records from 1993 to 2020 were collected from the PCMIS. These records contain a total of 948 auditing scores, 499 defects items (risk factors), and 9,596 defect frequencies. The experts were interviewed to identify five risk dimensions (46 risk factors) from PCMIS and a questionnaire survey (Likert scale) was performed among personnel working in construction-related fields to identify 19 major risk factors from five risk dimensions.

• The source of risk factors is not clear? for example, if they have been found from literature, it is needed to mention the references of each factor.

Response:

Thank you for your comments. We reiterated in the manuscript how risk factors were obtained (in lines 490-496 on page 11 and lines 439-442 on page 10).

The construction auditing mechanism is a top–down quality management process. The government establishes the format of the inspection form, experts conduct on-site auditing, and project-related personnel implement construction and management operations. Therefore, 46 risk factors were selected from the five risk dimensions defined by experts and researchers. To identify the major risk factors, project-related and experienced personnel were invited to participate in the questionnaire survey. This approach ensured that the survey results were consistent with actual construction practices.

The experts were interviewed to identify five major risk dimensions (46 risk factors) from PCMIS and a questionnaire survey (Likert scale) was performed among personnel working in construction-related fields to identify 19 major risk factors from five major risk dimensions.

• As a reviewer, it is suggested that adding text in front of each factor in Fig. 3. It helps the figure to be understandable and readable.

Response:

Thank you for your comments. We agree with your suggestion to add text in front of each factor in the Fig. 3 (in lines 546-547 on page 14).

• It is suggested separating the sections of results and discussion. It the section of discussion, discuss how the results can be beneficial for audience (researchers and practitioners) who work in construction management.

Response:

Thank you for your comments. We divided the Results and Discussion into two sections based on the reviewers' comments. We've also added some more in-depth discussions. (in lines 636-648 on page 17 and lines 649-668 on page 18).

Managers can use the proposed AHP–ANN model for accurately and rapidly extracting valuable information from big data. Thus, this model can effectively support managers in decision-making. Given that machine learning is an exploratory method, the analysis direction must be determined before conducting data mining. The results of data mining might be unpredictable. However, novel and useful knowledge can be obtained through machine-learning-based training and testing, and this knowledge can be used to construct a decision-making model for construction management. The AHP–ANN method provides several advantages; it addresses various weaknesses in several dimensions and simplifies the dimensions of a considerable number of risk factors to identify the major risk factors and estimate their influence. In addition, it can decompose complex problems one by one and establish a hierarchical structure comprising five risk dimensions, enabling managers to understand the attributes of the factors that affect quality and to effectively manage the major factors. When an empirical model is derived from a large volume of data and the mathematical framework of a system is unclear such that conventional statistical methods based on appropriate assumptions cannot be applied, an ANN-based prediction model becomes useful. However, because the training and model derivation process of an ANN is a black box, the ANN is at a disadvantage because it has difficulty explaining the logical reasoning and meaning of a model in accordance with the applied parameters.

In machine learning, a large volume of historical data are used to predict future actions or outcomes. In this context, prediction involves inputting the features of known variables or factors (e.g., importance) and classifying or regressing the output results. When an unknown set of variables is input into a machine learning model, the model can calculate the probability value on the basis of past experiences and further classify its results. An advantage of an ANN is that it introduces a nonlinear function as an activation function, which can approximate any function. That is, an ANN can produce distributions that approach the distribution of known variables. In the present study, construction auditing defects (major risk factors) reported between 1993 and 2020 in Taiwan were used as input variables, and auditing scores (project quality) were used as output results. When major risk factors affect a project, the prediction model can be employed to estimate the construction quality of the project. Although the accuracy of a prediction model can be verified using known training data, the classification of a prediction can only be obtained through the actual outcome in addition to observation and verification.

• Similar to the previous suggestion, it is suggested that in the section of Conclusion discuss how the results can be beneficial for audience (researchers and practitioners) who work in the area of construction management.

Response:

Thank you for your comments. We have added some more in-depth discussions in the Conclusion section (in lines 669-686 on page 18).

6. Conclusions

To improve the quality of public construction, the Taiwanese government established a construction auditing system that implements regular auditing. Experts and researchers are employed to conduct on-site quality audits with a standardized checklist (499 defective items). For each audited project, one to three experts or researchers spend a day to identify the defects in the design, construction, and supervision of the project. Subsequently, they provide a rating on the basis of defect severity and actual construction conditions. The auditing results and identified defects are registered in the PCMIS by the construction agency. If major defects are identified in a project or a score of less than 70 is given, the relevant personnel responsible for the project are penalized or fined. The AHP–ANN model proposed in the present study was established on the basis of a large volume of training data in the PCMIS. It is suitable for the auditing of public construction projects in Taiwan. When defects (major risk factors) are identified, a prediction of project quality can be obtained by using the model. Therefore, project managers can conduct an examination with the standardized checklist and evaluate the construction quality of their projects prior to an actual government construction audit. Managers can also learn about risk factors through this model and adopt the appropriate risk management and control measures.

• Generally, the contribution of this study is not clear for audience. Does development of a model add a significant value to the body of knowledge?

Response:

You have raised an important point here, thank you. We revised the manuscript and added clear descriptions to the sections to indicate our contribution. We also added scientific references to the Introduction section to explain our motivation from a broader perspective. We also summarized our contribution in the Conclusion section. (in lines 710-721 on page 19).

The hybrid AHP–ANN model for project quality prediction is a case-based knowledge model that is based on the body of knowledge. It makes full use of past cases or data to predict outcomes. By improving its analyses and predictions on the basis of a large volume of historical data, the model learned features from the auditing data, identified hidden rules or knowledge, and produced predictions with improved accuracy. It can be regarded as a function built on an auditing dataset that comprises features. Through the organization of decision-making elements through a hierarchical structure and the incorporation of the opinions of experts and experienced personnel, the model can clarify the relationship between major risk factors and project quality and help managers to develop solutions and countermeasures. This model can contribute to the body of knowledge for construction by continually accumulating data and inputting such data into its knowledge base.

Author Response File: Author Response.pdf

Reviewer 4 Report

I have reviewed an article titled (Hybrid analytic hierarchy process–artificial neural network
model for predicting the major risks and quality of Taiwanese construction projects) and have major concerns on its analysis and applied techniques.

1. First I have a concern, does these applied techniques has not been applied by previous studies, I know many oes who applied these AHP and ANN methods.so what’s the new in it?

2. What’s the authenticity to collect data from the PCMIS- Afterall its Governmental big data. Authors said 948 project records of construction inspection from 1993 to 2020 were collected from the Pub- 17
lic Construction Management Information System (PCMIS). It’s a big tank data based on a country data Base. How this data has been taken and analyzed as researcher.

3. Which scale type has been used. Its grade- is sit 3-4-6-9 grade Likert scale ?

 

4. Why author has chosen ANN and AHP for risk analysis. Authors claimed they have taken a big data based on country level, that’s true. But that big data has not been screened in more profundo way and the adopted techniques are very older one. From last 20 years people are using them and these techniques are already failed in real time projects. Academically the works has been published on them but when it comes to the field work, many reject mangers didn’t find any ting to take as guidance from them. So question raised as- If you are a project manger and you have a Billion-USD Project how would you apply these techniques over there. How would you quantify risk there. What would be the implication there to apply. Please add.

Author Response

• First I have a concern, does these applied techniques has not been applied by previous studies, I know many oes who applied these AHP and ANN methods. so what’s the new in it?

Response:

We are really thankful for your questions and insightful feedback on our manuscript. For this question, we add some explanations about AHP-ANN in construction management (in lines 148-152 on page 3 and lines 153-159 on page 4).

Moreover, employing multiple methods (Delphi and AHP approaches) which can overcome the limitations of a single methodology [17]. Hybrid decision science methods (ANP and quality function deployment) were integrated to improve the ability to consider relationships among the critical factors and their impact [18]. Although the technique for combining the AHP and an ANN has been used in other studies, it is rarely used to predict construction risks and project quality. The present study can fill this gap and provide an alternate project management model for the construction industry. In this study, standard construction inspection data collected by the Taiwanese government were used to identify the risks influencing project quality. The integrated AHP–ANN model can rapidly and effectively evaluate construction risks, thereby providing alternative solutions and preventive strategies for construction problems.

• What’s the authenticity to collect data from the PCMIS- After all its Governmental big data. Authors said 948 project records of construction inspection from 1993 to 2020 were collected from the Pub- 17 lic Construction Management Information System (PCMIS). It’s a big tank data based on a country data Base. How this data has been taken and analyzed as researcher.

Response:

Thank you for your comments. According to Taiwan's The Freedom of Government Information Law, research institutions, schools or individuals can apply relevant information to government units based on academic research or public welfare purposes. The construction audit data for this study was obtained by applying to the government for PCMIS, and was rearranged and analyzed by the authors. The URL of PCMIS is as follows: https://cmdweb.pcc.gov.tw/pccms/owa/cmdmang.userin

• Which scale type has been used. Its grade- is sit 3-4-6-9 grade Likert scale ?

Response:

Thank you for your comments. We have explained why a 5-level Likert scale is used in the text (in lines 470-480 on page 11).

In this study, a questionnaire survey was conducted on the major risks encountered by personnel involved in Taiwanese construction projects by using a 5-point Likert scale (5 = “Very important,” 4 = “Important,” 3 = “Neutral,” 2 = “Low importance,” and 1 = “Not at all important”). The point range of the Likert scale can be increased to 9 points or reduced to 2 points. However, having an excessive number of scale points may increase difficulty of completing the questionnaire for participants, and having an insufficient number of scale points may prevent the collected data from fully expressing the various degrees of participant intention. Given that 46 risk factors were rated in the questionnaire, a 5-point Likert scale was selected to ensure that the participants were not deterred from completing the questionnaire because of its lengthiness and to increase the validity of the questionnaire.

• Why author has chosen ANN and AHP for risk analysis. Authors claimed they have taken a big data based on country level, that’s true. But that big data has not been screened in more profundo way and the adopted techniques are very older one. From last 20 years people are using them and these techniques are already failed in real time projects. Academically the works has been published on them but when it comes to the field work, many reject mangers didn’t find any ting to take as guidance from them. So question raised as- If you are a project manger and you have a Billion-USD Project how would you apply these techniques over there. How would you quantify risk there. What would be the implication there to apply. Please add.

Response:

Thank you for your comments. We admire your insight on practical situation. We further explain the implications of this study for construction management in the Conclusion, as well as the practical application of construction auditing in Taiwan (in lines 655-686 on page 18).

To improve the quality of public construction, the Taiwanese government established a construction auditing system that implements regular auditing. Experts and researchers are employed to conduct on-site quality audits with a standardized checklist (499 defective items). For each audited project, one to three experts or researchers spend a day to identify the defects in the design, construction, and supervision of the project. Subsequently, they provide a rating on the basis of defect severity and actual construction conditions. The auditing results and identified defects are registered in the PCMIS by the construction agency. If major defects are identified in a project or a score of less than 70 is given, the relevant personnel responsible for the project are penalized or fined. The AHP–ANN model proposed in the present study was established on the basis of a large volume of training data in the PCMIS. It is suitable for the auditing of public construction projects in Taiwan. When defects (major risk factors) are identified, a prediction of project quality can be obtained by using the model. Therefore, project managers can conduct an examination with the standardized checklist and evaluate the construction quality of their projects prior to an actual government construction audit. Managers can also learn about risk factors through this model and adopt the appropriate risk management and control measures.

In machine learning, a large volume of historical data are used to predict future actions or outcomes. In this context, prediction involves inputting the features of known variables or factors (e.g., importance) and classifying or regressing the output results. When an unknown set of variables is input into a machine learning model, the model can calculate the probability value on the basis of past experiences and further classify its results. An advantage of an ANN is that it introduces a nonlinear function as an activation function, which can approximate any function. That is, an ANN can produce distributions that approach the distribution of known variables. In the present study, construction auditing defects (major risk factors) reported between 1993 and 2020 in Taiwan were used as input variables, and auditing scores (project quality) were used as output results. When major risk factors affect a project, the prediction model can be employed to estimate the construction quality of the project. Although the accuracy of a prediction model can be verified using known training data, the classification of a prediction can only be obtained through the actual outcome in addition to observation and verification.

Summary:

We appreciate your professional and in-depth comments. The authors consider the reviewer’s comments are very helpful to revise the manuscript. The manuscript has been significantly modified according to the reviewer’s comments to address the reviewer’s concern. We have made revisions based on your comments as much as possible, and will continue to improve our research based on your comments in the future. All major revisions are presented in blue. We are now resubmitting the paper for the reviewers’ further review, and look forward to the reviewers’ further suggestions and comments. Once again, the reviewers’ help and patience are highly appreciated.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have completely improved the manuscript following all my comments. The paper can be now accepted for publication.

Reviewer 2 Report

The authors have addressed my comments and provided acceptable justifications. so, I recommend it for publication. 

Reviewer 3 Report

The authors made major efforts to improve the quality of the manuscript. In the present form, the manuscript is qualified to be published in this journal.

Reviewer 4 Report

one round of proofreading is recommended.