Next Article in Journal
Threat Detection Model for WLAN of Simulated Data Using Deep Convolutional Neural Network
Previous Article in Journal
Analytic Investigation of Hooked Stirrups on Seismic Behavior of Reinforced Concrete 3D Frame Buildings
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Preferences for a Thermal-Camera-Based Abnormal Situation Detection Service via the Integrated Fuzzy AHP/TOPSIS Model

Department of Future & Smart Construction Research, Korea Institute of Civil Engineering and Building Technology (KICT), 283 Goyang-daero, Daehwa-dong, Ilsanseo-gu, Goyang-si 10223, Gyeonggi-do, Republic of Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(20), 11591; https://doi.org/10.3390/app132011591
Submission received: 16 September 2023 / Revised: 19 October 2023 / Accepted: 20 October 2023 / Published: 23 October 2023
(This article belongs to the Section Civil Engineering)

Abstract

:
Research related to thermal cameras, which are major control measures, is increasing to overcome the limitations of closed-circuit television (CCTV) images. Thermal cameras have the advantage of easily detecting objects at night and of being able to identify initial signs of dangerous situations owing to changes in temperature. However, research on thermal cameras from a comprehensive perspective for practical urban control is insufficient. Accordingly, this study presents a thermal camera-based abnormal-situation detection service that can supplement/replace CCTV image analysis and evaluate service preferences. We suggested an integrated Fuzzy AHP/TOPSIS model, which induces a more reasonable selection to support the decision-making of the demand for introducing thermography cameras. We found that developers highly evaluated services that can identify early signs of dangerous situations by detecting temperature changes in heat, which is the core principle of thermography cameras (e.g., pre-fire phenomenon), while local governments highly evaluated control services related to citizen safety (e.g., pedestrian detection at night). Clearly, while selecting an effective service model, the opinions of experts with a high understanding of the technology itself and operators who actually manage ser-vices should be appropriately reflected. This study contributes to the literature and provides the basic foundation for the development of services utilizing thermography cameras by presenting a thermography camera-based abnormal situation detection service and selection methods and joint decision-making engagement between developers and operators.

1. Introduction

1.1. Research Background and Purpose

Climate change, aging infrastructure, new types of mobility, and unspecified criminal accidents have increased the frequency of abnormal urban situations that are difficult to observe and predict. In this sense, demand has increased for real-time monitoring, responses, and prevention of urban life safety anomaly that threatens people’s lives. CCTV is a powerful means for monitoring the life safety anomalies experienced by people. For instance, the New York City Police Department in the United States operates real-time responses and surveillance systems via the Domain Awareness System (DAS). This system is an integrated system for real-time crime monitoring developed by Microsoft and the Department, by linking CCTV information with police database information [1]. China is operating an active urban surveillance system via CCTV-based facial recognition technology, such as Sense Time or Megvii [2]. South Korea is operating CCTV control centers in 226 (lower level) local governments to monitor urban anomalies (e.g., traffic, crime prevention, and disasters) via CCTV images [3]. However, as CCTV is not highly effective in nighttime situations and can suffer from privacy infringement issues, CCTV-dependent systems have several shortcomings [4,5]. A representative example is the disappearance and death of a medical student that occurred late at night on 25 April 2021, which was a big issue in Korea. Although the incident occurred in Hangang Park in the city center with a large floating population, there was no CCTV inside the park except on the access road to protect privacy, making it difficult to determine the circumstances of the incident. CCTV is also not sufficient for identifying early signs of major accidents due to temperature changes, such as fires, road status (e.g., ice and flooding), and facility conditions.
Advanced control technologies combined with the recent rapidly developed cutting-edge technologies (e.g., AI, IoT, and Big Data) have been introduced to overcome CCTV’s limitations. Examples include image analysis for CCTV’s automatic control [6,7,8,9,10,11,12,13] and integrating IoT to specifically identify situations outside the CCTV control zones [14,15,16]. Thermography has been particularly highlighted. It refers to images of radiation intensity measured by an infrared detector which represents the infrared thermal energy radiated from the surface of the object according to its temperature [17]. Infrared rays can be classified into near-infrared rays (NIR) with wavelengths 0.8~1.4 µm, shortwave infrared rays (SWIR) with wavelengths 1.4~3 µm, mid-wave infrared rays (MWIR) with wavelengths 3~5 µm, long-wave infrared rays (LWIR) with wavelengths 8~14 µm, and far infrared rays (VLWIR) with wavelengths 14 µm or longer [18]. LWIR is the wavelength area with a maximum infrared ray for a temperature of 300 K and is the wavelength most closely related to human lives. A thermal camera that uses LWIR is a device that visualizes detected infrared thermal energy captured by a thermal detector through image processing. It enables one to confirm objects regardless of the existence or absence of lighting as it can detect radiant heat emitted by heated objects [17]. Many studies show that Thermography/IR image object recognition based on thermal cameras is superior to general RGB at night [19,20,21]. Indeed, such techniques have been widely applied to diverse sectors, such as traditional military purposes, night surveillance, fire disaster prevention, life rescue, maritime surveillance, traffic, and rescue. This technology especially has the advantage of real-time identification of night situations that cannot be detected by CCTV, changes in object conditions due to temperature changes, and precursor fire events (or events preceding fires). Recent thermography CCTV technology development has enabled fixed installations, and installation in mobile equipment using robots and drones.
Thus, thermal imaging cameras can replace or supplement CCTV according to service characteristics and are essential for preventing civil safety accidents that are difficult to detect on CCTV. In addition, to monitor the safety of more citizens, it should be possible to operate them at the CCTV control center of the local government. Local government officials, who are major consumers, need decision-making support when selecting thermal camera-based services. Local government officials who monitor civil safety have abundant experience and know-how in operating services on civil safety but lack expertise in the technology itself. To this end, it is necessary to reflect the opinions of developers who specialize in understanding the specificity of thermal cameras along with local governments. Academically, related studies have mainly focused on the experimentation and advancement of individual technologies. However, research from a comprehensive perspective for practical urban control is insufficient. This study aimed to evaluate the preference for abnormal detection services using a thermal camera to control threat situations that are difficult to grasp immediately in general CCTV images. The service model presented in this study refers to a control service that detects abnormal situations that threaten the safety of citizens’ lives using images captured by a thermal camera as materials for image analysis. While numerous studies have examined individual technology/services based on Thermography/IR images observed by thermal cameras, few studies examine comprehensive decision-making support to introduce practical thermal camera-based image analysis technologies. Meanwhile, this study conducts in-depth interviews and questionnaires with an expert group comprising operating personnel in local governments’ control centers and Thermography/IR image analysis developers. Through this, we intend to support decision-making related to the selection and preference evaluation of abnormal situation detection services via thermal cameras. Comparing the differences in perceptions between the two groups implies service preference differences between providers (i.e., developers) and consumers (e.g., local governments). The findings can help inform the selection of core services in crime prevention, traffic, and disasters that are related to the safety of citizens within a city. To this end, in the remainder of Section 1, we review the research scope and methods, followed by a literature review. In Section 2, we suggest a methodology for service preferences. In Section 3, we set up abnormal situation detection services based on Thermography/IR images. In Section 4, we conduct an empirical analysis of this model, including evaluation criteria and service evaluation. Based on this analysis, in Section 5 and Section 6, we discuss the findings and their implications.

1.2. Research Scopes and Methods

This study targets urban life safety anomaly detection objects, situations, and precursory phenomena based on Thermography/IR images collected by thermal cameras that can be controlled by local governments’ control centers. As we targeted anomalies within citizens’ living spaces, the scope of service areas, such as crime prevention, traffic safety, and disasters, can be considered to be comprehensive. Devices that collect Thermography/IR images encompass fixed installations and mobile devices (e.g., probe cars, general cars, bicycles, kickboards, drones, and robots). We also considered the speed of technology development and scalability.
Regarding the research method, we selected the abnormal situation detection services based on Thermography/IR images through in-depth interviews with experts. For decision support, we also reviewed the multi-criteria decision-making (MCDM) technique, which can be helpful for deriving systematic and reasonable alternatives when selecting technologies and services. Among MCDMs, we suggested optimal methodologies by reviewing the Fuzzy Analytic Hierarchy Process (AHP), which is a more advanced methodology than AHP, and Technique for Order Performance by Similarity to Ideal Solution (TOPSIS). Next, we selected evaluation criteria considering the control centers and service characteristics. Based on these criteria, we set the hierarchical structure, and assessed evaluation criteria and service models for each expert group. Finally, we undertook our discussion of the findings and implications based on the resultant evaluation criteria and service models’ importance ranking.

1.3. Literature Review

Several studies have examined urban anomaly control using Thermography/IR images. In the field of fire research, one study was conducted to detect flashover phenomena involving rapid fire propagation using IR image sequences [22]. Studies have explored HW and system development for effective day and night surveillance by continuously using thermal cameras [17,23,24,25,26]. Others have examined human face recognition based on recent advanced thermography images [27], recognition of abnormal behaviors [28,29], and falling recognition [30,31]. These studies indicate deeper and more progressive explorations have been conducted besides just identifying general objects via thermal cameras. For instance, road control studies use thermal cameras for various applications, including pedestrian detection on the road at night [19], object detection in under-exposed areas for autonomous driving [32], pothole detection on roads [33], and wildlife detection at night [34,35]. Transportation studies have explored the control of railway infrastructure elements such as train rails [36], while others have analyzed electric bicycle battery packs [37]. This indicates diverse thermography-based image analysis studies for preventing safety accidents in road and transportation. Some studies have also diagnosed buildings using thermal cameras, including detection of issues in building exteriors (e.g., corrosion of steel bars in concrete structures of buildings) [38,39] and diagnosis of thermal bridges in buildings [40,41,42,43,44]. In relation to HW, one study analyzed the effects of the performance and variability of thermal cameras on changes in the body surface temperature of horses [45]. This study investigated the repeatability and reproducibility of thermal image measurements using five thermal camera models based on temperature measurements used in the field. This study emphasizes the importance of ensuring the quality of thermal cameras.
Recent thermography research has focused on a more diverse range of topics, particularly a wider scope of control related to the detection of single objects, urban control, and monitoring diverse disasters, such as volcano monitoring [46], detection of temperature changes in landfills based on infrared satellite images [47], and urban heat island phenomenon [48,49,50,51,52,53]. Some researchers have combined the latest equipment, such as drones and robots, for the detection of human objects based on drones [54,55,56,57], inspection of building exteriors via drones [58], vegetation classification in urban areas via drones [59], and outdoor security surveillance robots [60]. Thus, research using Thermography/IR images has been conducted in various fields: day and night surveillance, road/traffic control, building diagnosis, various disaster monitoring, and control of urban spaces. This indicates an expanded increase in use cases and applications for thermal camera-based services; while selecting a service model in Section 3, we referred to these studies. However, these studies focused on individual fields; few studies adopt a comprehensive perspective related to using thermal cameras for practical urban surveillance services. This study differs from previous studies in that it explores the selection of comprehensive and diverse abnormal situation detection services based on thermal cameras by actual operators, such as local governments and facility managers, who manage abnormal situations in a city.

2. Materials and Methods

2.1. Importance Analysis of Fuzzy AHP-Based Evaluation Criteria

This study aims to support decision-making on the introduction of control services that can detect life safety abnormalities in a city through thermal cameras. One of the most common approaches is MCDM, which has been utilized to suggest objective priority alternatives based on evaluation criteria according to the defined purpose. MCDM is a proven technique that can present a priority alternative based on the understanding and knowledge level of the evaluation criteria, alternatives, and stratification structures. The typical methodology of MCDM is AHP. AHP hierarchically classifies a number of attributes and identifies the importance of each attribute in selecting an optimal alternative. AHP has the advantage of being able to structure alternatives in complex decision-making situations, and easily integrate quantitative/qualitative and subjective/objective factors based on the experience and knowledge of experts. AHP was developed by Saaty and supports decision-making through pairwise comparison between hierarchical elements [61]. Since Saaty’s study, numerous researchers have developed diverse derivative techniques. A representative example is Fuzzy AHP, which combines the fuzzy theory suggested by Zadeh with AHP [62]. Fuzzy Theory is a concept that introduces fuzzy logic and fuzzy sets to overcome the inaccuracy and imprecision of subjective judgment during evaluation. Fuzzy AHP enables a mathematical expression of unclear phenomena (e.g., ambiguous quantitative information, and subjective and uncertain judgment) as well as deriving reasonable decision-making alternatives [3]. Although it is hard to make a direct comparison, some have shown that Fuzzy AHP is mathematically more reliable based on a comparative analysis between the classical method, AHP, and Fuzzy AHP [63]. Numerous studies have also derived priorities for technology/services by evaluating the reliability of the Fuzzy-AHP model [64,65,66,67]. This study performs a relative importance analysis of evaluation criteria by considering the advantages of Fuzzy AHP.
The Fuzzy AHP, as utilized in this study, is a supplemented calculation formula by Laarhoven and Pedrycz using a Triangular Fuzzy Number (TFN) based on Saaty’s AHP [68]. The triangular fuzzy number M used to calculate the weight was expressed by modifying M 1 = ( l 1 ,   m 1 ,   u 1 ) , the membership function of the lower, center, and upper values, to ( a 1 ,   a 2 ,   a 3 ) to compare the pairs (Figure 1).
Here, TFN is expressed in the form of a triangle with three points: (l, m, u; l = lower limit, m = middle, u = upper limit), and their area is the size of TFN [66,69,70]. We utilized the fuzzy style outlined in Table 1, which was presented as a Fuzzy AHP extended analysis technique to derive TFN [3,7,66,71,72].
To derive evaluation criteria per class, the fuzzy scale in Table 1 was utilized to calculate the average value of weights according to pairwise comparison. By utilizing Equations (1)–(4), we derived the TFN S i   = ( l i , m i , u i ) value of the ith attribute, and finally calculated the eigenvector normalization value of the minimum value for defuzzification of the TFN S i per evaluation criterion [3,67,70]. Equation (1) is the sum of the relatively evaluated scores before fuzzification per item. Equation (2) is a triangular fuzzification according to the l, m, and u values according to the fuzzy scale. Equation (3) is the sum of the relatively evaluated items of each evaluation criterion for each value of l, m, and u. Equation (4) is the final TFN value that takes the reciprocal of the sum of l, m, and u.
S i = j = 1 m M i j × i = 1 n j = 1 m M i j 1
s . t j = 1 m M i j = ( j = 1 m l i j ,   j = 1 m m i j ,   j = 1 m u i j )
i = 1 n j = 1 m M i j = ( i = 1 n j = 1 m l i j ,   i = 1 n j = 1 m m i j ,   i = 1 n j = 1 m u i j )
i = 1 n j = 1 m   1 = [ 1 i = 1 n j = 1 m u i j ,   1 i = 1 n j = 1 m m i j ,   1 i = 1 n j = 1 m l i j ]

2.2. Importance Analysis of Fuzzy TOPSIS-Based Alternatives

In Section 2.1, we used the relative importance analysis methodology using a pairwise comparison-based Fuzzy AHP technique for the evaluation criteria. However, for the alternative evaluation, when the number of alternatives that participants need to respond to is large, the pairwise comparison evaluation can result in more questionnaire items, and thus, more insincere responses. As this study evaluates more than 20 alternatives, a pairwise comparison between each item is impossible. Consequently, we attempted to evaluate the alternatives using an absolute evaluation instead of a relative importance evaluation. For absolute evaluation, a classic AHP method can be utilized. However, the TOPSIS technique can also be applied as Fuzzy theory can be applied, as shown in Section 2.1, and rational choices can be induced by simultaneously considering the best and worst alternatives [73,74,75,76]. In particular, the TOPSIS technique can simultaneously display the best and worst alternatives as real values, and it is possible to measure the performance of all alternatives from a multicriteria perspective. It also has the advantages of rational logic and calculation simplicity [77]. This study uses fuzzy language to solve the uncertainty problem of subjective and ambiguous linguistic information. Based on this, we used the Fuzzy TOPSIS technique, which combines the TOPSIS method to provide more accurate criterion weighting by considering the best and worst alternatives.
The widely used TOPSIS method, proposed by Hwang and Yoon, elects alternatives closest to the Positive Ideal Solution (PIS), and farthest from the Negative Ideal Solution (NIS) [78]. As this technique can induce rational choices by simultaneously considering the best and worst alternatives, it is facile to calculate evaluations of all alternatives from a multi-attribute perspective [73,74,75,76]. The fuzzy scale is a 5-point scale utilized in the Fuzzy-AHP analysis. The fuzzification of alternatives per evaluation criterion aggregated from multiple evaluators is calculated in Equation (5), and the fuzzy weights are calculated in Equation (6) [79].
a i j = m i n k a i j k ,   b i j = 1 k k = 1 k b i j k ,     c i j = m a x k ( c i j k )
w j 1 = m i n k w j 1 k ,   w j 2 = 1 k k = 1 k w j 2 k ,     w j 3 = m a x k ( w j 3 k )
Then, the normalized fuzzy decision matrix R ~ = [ r i j ~ ] (Equations (7) and (8)), and the fuzzy decision matrix V ~ = v i j ~ ,   v i j = r i j ˇ × w j with the evaluation criteria weights (TFN) derived by Equation (4), can be calculated [79].
r i j ˇ = a i j c j * ,   b i j c j * ,   c i j c j *   a n d   c j * = m a x i c i j   ( b e n e f i t   c r i t e r i a )
r i j ˇ = a j c i j ,   a j b i j ,   a j a i j   a n d   c j = m i n i a i j   ( c o s t   c r i t e r i a )
Next, the Fuzzy Positive Ideal Solution (FPIS) and Fuzzy Negative Ideal Solution (FNIS) can be calculated as shown in Equations (9) and (10), respectively [79].
A * = ( v 1 ~ * ,   v 2 ~ * ,   v n ~ * ) ,   w h e r e   v j ~ * = m a x i ( v i j 3 )
A = ( v 1 ~ ,   v 2 ~ ,   v n ~ ) ,   w h e r e   v j ~ = m i n i ( v i j 1 )
Two TFN distance calculation methods suggested by Chen were utilized to measure the FPIS and FNIS distances per alternative [80]. When x ~ = a 1 ,   b 1 ,   c 1     a n d     y ~ = a 2 ,   b 2 ,   c 2   are two TFNs, the distance can be calculated as shown in Equation (11), and the final FPIS and FNIS distances in each alternative can be calculated according to Equation (12) [79].
d x ˇ ,   y ˇ = 1 3 [ a 1 a 2 2 + b 1 b 2 2 + c 1 c 2 2 ]
d i * = j = 1 n d ( v ˇ i j ,   v ˇ j * ) , d i = j = 1 n d ( v ˇ i j ,   v ˇ j )
Finally, the proximity coefficient for each alternative C C i can be calculated as shown in Equation (13), and the ranking can be determined from the alternative with the highest C C i [79].
C C i = d i d i + d i *

3. Service Model Selection

In this study, the actual sources of demand are local governments’ Smart City Integrated Operation Centers, which operate CCTVs and IoT sensors that can monitor citizens’ life safety, and facility operators, who manage individual buildings and facilities. As mentioned before, South Korea operates Smart City Integrated Operation Centers in 226 local governments. To practically set an operatable service model, we created a group of five experts including public officials who were in charge of the control center of the local government and had experience in operating thermal cameras, and thermography image analysis developers. We also conducted a Focus Group Interview (FGI) to set alternative service models based on the expert group’s opinions. FGI can be utilized when limited literature is available and solutions with new perspectives are required [81]. Especially, FGI is a highly useful decision-making support method when comprehensive perspectives and insights in diverse fields are needed [3].
In the first half of the FGI, we reviewed thermal camera-based research and application cases prepared by the authors and then focused on discussing available services to be targeted particularly with thermography image analysis developers. The thermal-camera-based research case included not only the results of prior studies of such issues as “1-3” fire sessions, night objects, road/traffic fields, and urban control but also examples of the actual operation of thermal camera products in the field. The main discussion topics were fire and abnormal weather control services using nighttime object detection and temperature differences. Afterward, we classified specific services while targeting the living SOC (Social Overhead Capital), which is controlled by public servants in charge of control centers of local governments. We also derived detailed services based on thermography images that can supplement existing CCTV control in road, pedestrian safety, facility management, and disaster prevention. In the second half of the FGI, we organized stratification and specific control details for the discussed service models.
The final derived service model determined by the FGI was as follows. In the major classification, we derived items related to anomaly detection, such as objects, roads, facilities, and disaster prevention situations. First, we discuss the potential object anomaly detection service. In terms of its medium classification, the preventive detection service of traffic accidents at night includes the nighttime pedestrian detection service, identifying which can be challenging using nighttime CCTVs as well as for vehicle drivers. This classification includes nighttime mobility detection services to prevent high-risk accidents in alleys due to kickboards, and bicycles. The night security incident detection service encompasses the violence-related incident detection service (e.g., crowding, collision, and collapse), detection service for fear of crime related to violence (hereafter, “crime fear”; e.g., loitering and following), and intrusion status detection service at night (e.g., private land and prohibited facilities). The pedestrian hazard detection service includes the detection of dense pedestrian situations within pedestrian spaces (e.g., alleys and pedestrian paths), and a service for computing population clustering at festivals and crowded events.
Second, the road anomaly detection service includes the following. The road weather hazard detection service includes a road surface temperature detection service for detecting the road surface status and abnormally high temperatures of underground facilities due to high temperatures in summer, as well as a road flooding detection service (e.g., heavy rain and drain blockage on roadside) and road ice detection service in winter. The road surface hazard detection service encompasses a road pothole detection service to detect road damage risks, such as potholes and cracks, and fallen objects on the road.
Third, the facility anomaly detection service includes the following. The building status detection service includes the detection service of building exterior damage (e.g., damage to the exterior wall of a building and deterioration levels), as well as the building structure diagnostics service to detect temperature differences for diagnosis per space and the structure of a building. The precursor fire event detection service in facilities encompasses the detection service for precursor fire events within buildings (e.g., underground electric vehicle charging stations and distribution facilities in multi-use facilities), and the detection service of precursor gas and fire events in facilities that may leak from construction sites, traditional markets, and gas facilities. The detection service of facility accident signs includes the detection service for emissions in business places to detect air pollutants (e.g., carbon, nitrogen oxides, and incombustible gases) emitted from places of businesses, and the detection service of explosives at large events with large crowds to prevent terrorism.
Fourth, the disaster prevention anomaly detection service is as follows. The natural disaster prevention detection service includes the detection service for precursor forest fires in forest areas vulnerable to fire (e.g., mountains, forests, and parks), and river flooding detection service to detect river flooding and precursory phenomena in deteriorating weather conditions. Finally, a heat island detection service was derived in response to summer heat waves. Overall, four major services were identified, followed by ten services at the medium classification, and finally, 21 abnormal situation detection services based on thermal cameras (Table 2).

4. Empirical Analysis

4.1. Analysis Overview

We evaluated the preferences for abnormal situation detection services based on thermal cameras by utilizing the integrated Fuzzy AHP/TOPSIS model. Groups of 25 experts, including persons operating local governments’ control centers (12 experts) and Thermography/IR image analysis developers (13 experts), were interviewed to evaluate service models more professionally and practically. Since evaluators are experts who develop and operate practical services, they understood this study and the questionnaire very well, resulting in higher reliability of the analysis results. Furthermore, we could conduct a comparative analysis of the service preferences of providers (developers) and consumers (local governments), as well as the overall service model evaluation. In AHP research, even if the number of respondents is small, the sample size is not a problem if expertise and logical consistency are assumed [82]. We utilized Microsoft Office Excel 2019 as an analysis tool.
Stratification, one of the important tasks of AHP analysis, requires an organic design of decision-making details, or evaluation criteria and hierarchical structure between alternatives, to achieve the final goal [3]. The goal of this study, which is the first class, is the selection of abnormal situation detection services based on a thermography camera. The evaluation criteria for evaluating alternatives can be classified into the second and third classes. Finally, evaluation alternatives that belong to the fourth class are the 21 service models selected previously. This study reviewed the main public guidelines that evaluated the value of technologies and services in Korea to select evaluation criteria for the second and third classes. Reviewing the Smart City Service Certification System [83], Technology Assessment of Technology Guarantee Fund [84], and Technology Value Evaluation Manual of the Ministry of Land, Infrastructure and Transport [85] shows that common evaluation criteria include compliance, technology, marketability, and business feasibility according to the evaluation purpose.
This study was conducted at the stage of establishing service models before operating actual technologies and services. Therefore, we excluded items such as the degree of rights, business feasibility, and functional evaluation necessary at the project implementation stage among the previously investigated items. Three major classification evaluation criteria were finally selected: purpose compliance, service suitability, and service feasibility. Among minor classification evaluation criteria, we identified basic technology/service evaluation items and set new evaluation criteria (e.g., CCTV replacement/complementary effects) appropriate for this study (Table 3). Specifically, purpose compliance consists of the prevention of life safety, CCTV replacement/complementary effects, and conformity to relevant policies. Service suitability consists of service performances/effects, competitiveness versus existing services, and market demand growth. Service feasibility consists of economic efficiency, field applicability, and operational sustainability. Figure 2 shows the final hierarchical structure reflecting these categories and criteria across classes.

4.2. Evaluation Criteria Analysis

Before analyzing the evaluation criteria, we verified the questionnaire’s consistency. The Consistency Ratio (CR) is calculated according to the comparison matrix Consistency Index (CI) and n size and is calculated using the average random index of multiple comparison matrices CI [82]. In general, if CR is less than 0.1, consistency is good, and a level of 0.1–0.2 is determined to be appropriate. The CR of the 25 samples used in this analysis was 0.124, indicating appropriate consistency.
For pairwise comparison analysis of the second and third classes, which are the evaluation criteria classes, we constructed a matrix based on the aforementioned fuzzy scale and then calculated the geometric mean value of the fuzzy preference. Based on this, the TFN value per evaluation criterion and eigenvector value of the minimum value for defuzzification, or the final weights, were calculated. The TFN values, reported in Table 4, of the second class were: purpose compliance had values of 0.24, 0.34, and 0.46, service suitability had values of 0.24, 0.33, and 0.47, and service feasibility had values of 0.24, 0.33, and 0.46. The final calculated normalized weight was the highest for purpose compliance (0.387), followed by service suitability (0.333) and service feasibility (0.331). The configuration of a service model such as prevention of life safety and CCTV complementary effects may be considered to be important to appropriately use special equipment like thermal cameras.
We also calculated TFN and weights per evaluation criteria of three classes in the same manner. The results are reported in Table 5. For the TFN of purpose compliance, prevention of life safety had values of 0.28, 0.39, and 0.54, CCTV replacement/complementary effects had values of 0.24, 0.34, and 0.48, and conformity to relevant policies had values of 0.19, 0.26, and 0.37. Prevention of life safety had the highest weight (0.394), followed by CCTV replacement/complementary effects (0.337) and conformity to relevant policies (0.270). Thus, citizens’ life safety anomaly prevention was a top priority, and the complementary effect of abnormal situations that cannot be solved only by CCTVs was considered more important than governments’ policies. In terms of TFN of service suitability, service performances/effects had values of 0.29, 0.41, and 0.58, competitiveness versus existing services had values of 0.21, 0.29, and 0.40, and market demand growth had values of 0.22, 0.30, and 0.41. Service performances/effects had the highest weight (0.402), followed by market demand growth (0.299) and competitiveness versus existing services (0.295). Clearly, the performance and effectiveness of the service itself, such as response to abnormal situations and accident prevention, are important evaluation factors. Finally, regarding the TFN of service feasibility, economic efficiency had values of 0.22, 0.30, and 0.42, field applicability had values of 0.28, 0.40, and 0.57, and operational sustainability had values of 0.21, 0.29, and 0.40. Field applicability had the highest weight (0.396), followed by economic efficiency (0.308) and operational sustainability (0.295). Field applicability, including personal information protection, risk management such as administrative delays, and actual service operation in the field, appears to be highly evaluated.
Finally, the final importance ranking calculated by multiplying the evaluation criteria weights of the second and third classes and reported in Table 6 revealed that service performances/effects were the most important at 0.134, followed by prevention of life safety (0.132), field applicability (0.131), CCTV replacement/complementary effects (0.113), and economic efficiency (0.102). The performance/effects of the service itself should be excellent depending on the distinct features of a thermal camera. Prevention of life safety for citizens is assumed to be possible by replacing/supplementing CCTVs. Furthermore, selecting a service model that ensures practical field applicability and economic efficiency is essential to realize services such as risk management, cooperation with related organizations, and government support.

4.3. Evaluation of Services

4.3.1. Calculation Process and Analysis Results of All Respondents

We performed triangular fuzzification of alternatives per evaluation criteria by applying the fuzzy scale per evaluation score in Table 1 to evaluate alternatives based on Fuzzy-TOPSIS. By utilizing Equations (5) and (6), we performed a fuzzification of alternative evaluations consisting of the minimum, average, and maximum values of the TFN per item. Table 7 indicates the results from the 25 valid samples.
Using Equations (7) and (8), we calculated the normalized and weighted fuzzy decision matrices. Taking the A-1-1 (Pedestrian detection at night) alternative evaluation of the prevention of life safety evaluation criteria as an example, the upper limit of these criteria ( c i * ) is 4.5. Then, the decision matrix is R ¯ = ( 0.7 4.5 ,   3.1 4.5 ,   4.5 4.5 ) , and the weighted fuzzy decision matrix is calculated as V ¯ = 0.7 4.5 ,   3.1 4.5 ,   4.5 4.5 × 0.28 ,   0.39 ,   0.54 = ( 0.04 ,   0.27 ,   0.54 ) . Using Equations (9) and (10), we calculated FPIS and FNIS, respectively, as shown in Table 8.
Using Equation (11), we measured the distances of FPIS and FNIS per alternative. Taking FNIS of the A-1-1 (pedestrian detection at night) alternative to the prevention of life safety evaluation criteria as an example, we obtain:
d x ¯ ,   y ¯ = 1 3 [ 0.04 0.04 2 + 0.27 0.17 2 + 0.54 0.42 2 ] = 0.093 .
The final FPIS ( d i * ) and FNIS ( d i ) for each alternative calculated by Equation (12) are reported in Table 9 and Table 10, respectively.
Finally, the proximity coefficient ( C C i ) is calculated by Equation (13), and the ranking of each evaluation alternative is reported in Table 11. C C i of Pedestrian detection at night (A-1-1) and nighttime mobility detection (A-1-2) are 0.896, and 0.859, respectively, and ranked first and second. Thus, the importance of preventing traffic accidents that threaten pedestrian safety at night, when CCTV video monitoring is ineffective, was the most highly evaluated. Next, detection service of precursor fire events within buildings (C-2-1, C C i : 0.854) and facilities (C-2-2, C C i : 0.810) were also highly evaluated, ranking third and fourth. Clearly, the interest of evaluators and the importance placed on these issues have increased due to the occasional fire accidents during charging electric vehicles, and at construction sites and traditional markets. In particular, in the case of fires in electrical equipment, the prime time is short; however, precursors to fires, such as temperature changes before a fire occurs, can be confirmed through a thermal camera. The relatively high ranking of these factors suggests that these special characteristics of thermal cameras were reflected in the evaluation.
Other similarly highly ranked services included precursor forest fire (D-1-1, C C i : 0.767) and heat island (D-2-1, C C i : 0.568) detection services ranked fifth and ninth, respectively. Clearly, the pre-fire phenomenon was ranked high, perhaps reflecting the fact that heat waves are occurring more frequently due to abnormal weather conditions in summer. Despite being related to disaster, river flooding detection (D-1-2, C C i : 0.419) was ranked low (15th). This may reflect the fact that it is possible to control flooding even with other measurement sensors like radars, even if a thermography camera is not used. Road ice (B-1-3, C C i : 0.658) and road flooding (B-1-2, C C i : 0.529) detection services were ranked sixth and 10th, respectively. The relatively better ranking may reflect the importance of preventing road vehicle accidents in inclement weather. Detection service of dense pedestrian situations (A-3-1, C C i : 0.640) and calculation service of cluster population (A-3-2, C C i : 0.588) were also ranked high at seventh and eighth, respectively. This may have happened due to the Itaewon stampede in October 2022, in which 159 people died.
Service models ranked low included the detection services for night security incidents, which had, violence-related incident detection (A-2-1, C C i : 0.479), detection of crime fear (A-2-2, C C i : 0.355), and intrusion status detection (A-2-3, C C i : 0.521) ranked 13th, 18th, and 11th, respectively. This may be due to the technological development of general CCTVs, which has made it easier to detect objects at night even with RGB images. Detection services for emissions in business places (C-3-1, C C i : 0.517), and explosives at large events (C-3-2, C C i : 0.370) were ranked 12th and 17th, respectively. These services can be replaced with other measurement sensors which need not be thermography cameras. Road surface hazard detection (B-2) and building status detection (C-1) services ranked the lowest, ranging from 16th to 21st. These services cannot cover the control blind areas with fixed equipment. Although mobile equipment, like drones, may be utilized, the results may be insufficient and not reliable enough for evaluators.

4.3.2. Comparison of Analysis Results between Developers and Local Governments Operators

Here, we compare the priority analysis results between the thermography image analysis of the developer (n = 13), and local governments’ operator groups (hereinafter, local governments, n = 12) (Table 12). First, the two groups had different number one and two service models. Pedestrian detection at night (A-1-1) and nighttime mobility detection (A-1-2) were ranked first and second by local governments, but third and fourth by developers. Detection service of precursor fire events within buildings (C-2-1, first priority) and within facilities (C-2-2, second priority) were ranked the highest by developers, but relatively lower (eighth and fifth, respectively) by local governments. Thus, developers highly evaluated the importance and practical application of service models regarding temperature changes in heat, which are the core principles of thermography cameras, especially pre-fire phenomenon detection. In contrast, local governments highly evaluated the importance of control over citizen safety within the thermography camera service area. An additional example is that intrusion status detection service at night (A-2-3) was ranked as the third most important by local governments but as the 20th by developers. Second, developers evaluated the importance of road weather risk detection services more highly than local governments (e.g., road ice detection (B-1-3) and road surface temperature detection (B-1-1)). This is also a service model that clearly reflects the temperature change detection features of a thermography camera.
Third, local governments evaluated the importance of Natural disaster prevention detection services more highly than developers (e.g., detection of precursor forest fire (D-1-1) and river flooding detection (D-1-2)). This appears to be the result of local governments highlighting the importance of control in disaster prevention; poor response can lead to larger disasters. This is characterized by local or spatial control rather than the detection of specific objects. Heat island detection, which is in the same disaster prevention field but does not lead to an urgent disaster, was ranked higher by developers. The urgency of disasters appears to be an important factor in local governments’ selected priorities. Meanwhile, developers put a high priority on analyzing differences in heat due to temperature changes; specifically, they seek to maximize the service that can only be provided by thermal cameras and is difficult to measure with other measurement sensors.

5. Discussion

This study selected abnormal situation detection services using thermography cameras and selected priorities via service model evaluation via expert interviews. Thermography cameras enable preemptive detection of abnormal situations at night, abnormal weather, and pre-disaster phenomena that cannot be easily and immediately confirmed with regular CCTVs. This is because thermography cameras have a higher object recognition reliability at night, which is a limitation of CCTV RGB images, and clearly display temperature differences. However, it has the disadvantages of short object detection distance due to lower resolution than regular CCTVs and high equipment cost. Considering these factors, the installation locations of thermography cameras should be carefully selected. For instance, to prevent traffic accidents at night, thermography cameras should be selectively installed in intersections and roundabouts where pedestrian traffic accidents frequently occur and traffic volume is high. This approach can be also applied to areas where freezing and flooding frequently occur to ensure road safety. For crime prevention purposes, good locations to install cameras can include places with a high crime rate or that are desolate (e.g., parks and trails). Clearly, a spatial analysis is needed to select thermography camera locations in line with each installation’s purpose and regional features.
Technical discussions are also needed to implement and upgrade services. In some situations, maximizing service effects may be easy by merely utilizing thermography cameras. For instance, better object recognition and analysis effects can be achieved by using a thermal image combined with general CCTV images. Similarly, the reliability of temperature changes can be ensured by combining it with weather sensors (e.g., temperature, humidity, and atmospheric pressure), while temperature estimation and interpolation can be used to extend the detection distance of thermography cameras. After recognizing abnormal situations analyzed by thermography cameras, it is also necessary to contemplate how and to whom to convey it to. For instance, systems can alert or give an alarm to control managers, or even citizens around thermography cameras; if information is delivered to citizens, effective delivery methods such as text, image, and sound information should be considered. The utilization of active mobile equipment can be also considered. Thermography CCTVs, which are fixed at one location, have extremely limited control areas. Recent research and technology developments have been made in using mobile equipment, such as robots and drones, equipped with such cameras. For instance, a technology supporting autonomous driving via thermography modules can become a powerful tool to complement existing vision sensors [86,87]. Mobile equipment-based thermography cameras can be utilized to minimize control blind spots and detect urban abnormalities more clearly.
Government policy support is required to make thermography cameras widespread and ensure citizens’ life safety. As mentioned earlier, thermography cameras require higher construction costs than general CCTV. Local governments, which are regarded as the main source of demand for the service model, spend substantial money to maintain existing control services; as such, they face difficulties in securing the budget for implementing new services. In particular, small and medium cities with poor financial conditions may need support from the central government, such as government matching funds and public offering projects. This service model, which can directly monitor citizen safety, requires economic efficiency analysis considering public benefits rather than a financial review that simply examines construction and operation costs. This is because expanding infrastructure for citizen safety and securing a social safety net are essential public goods. Even in the field, as suggested here, service model evaluation and economic efficiency analysis are required based on appropriate cross-reviews by developers and local government operators.
Our findings have important implications. First, this study suggested a thermography camera-based abnormal situation detection service encompassing night situations, pre-fire phenomena, and status changes in objects caused by temperature changes. Combined with IoT sensors, this service model can help in overcoming the limitations of CCTVs. Studies have typically focused on Thermography/IR image analysis separately; by contrast, this study suggests a service model selection/evaluation process from a comprehensive perspective to implement control services in urban areas. Our insights can be utilized as a reference in decision-making situations for selecting relevant service models. Second, as a service model selection/evaluation technique, this study presents an extension of the classical AHP method: the integrated Fuzzy AHP/TOPSIS model. While we performed the Fuzzy AHP-based relative importance analysis for the evaluation criteria, which is often utilized in previous studies, we fuzzified the TOPSIS technique. This provides more precise criteria weights while considering questionnaire fatigue in a questionnaire with more than 20 alternative evaluations, and using best and worst alternatives. This study also has methodological significance because TFN, a fuzzified evaluation criteria weight, was applied to the fuzzy decision matrix while deriving FPIS and FNIS of Fuzzy TOPSIS. This allowed us to combine the strengths of Fuzzy AHP based on relative importance and Fuzzy TOPSIS based on absolute importance.
Third, the implications of the major service model evaluation results are as follows. The night pedestrian and mobility detection services were ranked first and second overall, respectively, which is considered to indicate a greater importance attributed to the object recognition utilization and reliability of thermal cameras than to general RGB images that are vulnerable at night. When implementing this service, it is essential to consider not only object recognition but also the method of transmitting it to pedestrians and drivers (e.g., using LED electronic boards and pictograms). Precursor fire event detection services were also ranked at the top (ranking third to fifth). To operate the service, it is necessary for operators to select thermal cameras suitable for the service area range. For example, to detect a precursor fire inside a building, several entry-level types with short detection distances should be installed in dangerous facilities (e.g., electric charging stations and distribution pipes). To identify fires outside buildings and forest fires, products with long detection distances should be selected, even if they are expensive. It is also a good idea to use mobile equipment, such as drones. Fourth, the difference in perception between the developer and the service operator is as follows. We found that developers highly evaluated services that can identify early signs of dangerous situations by detecting temperature changes in heat, which is the core principle of thermography cameras (e.g., pre-fire phenomenon), while local governments highly evaluated control services related to citizen safety (e.g., pedestrian detection at night). Clearly, while selecting an effective service model, the opinions of experts with a high understanding of the technology itself and operators who actually manage services should be appropriately reflected.
This study has some limitations. First, while a service model was evaluated from a comprehensive perspective to detect life safety abnormalities based on thermography cameras, actual regional and spatial analysis was not conducted. One must carefully explore the current status, such as regional features, before introducing thermography cameras and control infrastructure levels and performing spatial analysis. Second, this study focused on an expert group consisting of operators of local governments’ control centers, which are the source of demand. These centers are practical sources of public demand that actually decide the introduction of thermography cameras for citizens’ life safety. However, their opinions may not completely reflect more detailed and specific opinions from sites, such as construction sites, park management, road management, and facility management. Follow-up studies should consider opinions from the field in each service sector. In future research, it will be necessary to set service items and modify the analysis model according to the detailed field opinions of the service field. Third, as this study suggested abnormal situation detection services based on general thermography cameras, specific service implementation measures, such as required data per service, service scope, expected cost, and connection system, were not considered. Clearly, economic efficiency analysis and commercialization strategies for service implementation are necessary. In a follow-up study, it will be necessary to prepare specific service model scenarios and operational guidelines to establish services from the perspectives of practitioners.

6. Conclusions

This study selected and evaluated abnormal situation detection services based on thermography cameras, which enable facile detection of objects at night, and the identification of early signs of dangerous situations according to temperature changes in heat; the goal was to overcome the limitations of CCTV images, which is the current main tool. We conducted in-depth interviews with experts to select abnormal situation detection services based on thermography cameras. We also suggested an integrated Fuzzy AHP/TOPSIS model, which induces a more reasonable selection to support the decision-making of the demand for introducing thermography cameras. We used this model to evaluate our proposed service model and then conducted a comparative analysis of preference differences between developers and local governments.
Our main findings are as follows. The analysis results of all respondents indicate that services related to traffic accident prevention, which threaten pedestrian safety at night when CCTV video monitoring is not effective, had the highest evaluation. This is in the context of previous research, such as day and night pedestrian detection on the road and autonomous-driving-related object detection [19,20,21,32]. Meanwhile, services related to nighttime crime prevention ranked the lowest. This is because it became easier to detect objects at night due to the higher illuminance performance of newer CCTV for crime prevention, while more traffic accidents have happened due to pedestrians in dark clothing and dark vehicles at places with insufficient road lighting. Pre-fire phenomenon detection services inside buildings, facilities, and forests also ranked high, as the identification of pre-fire phenomena can help avoid larger disasters. The comparison of analysis results of developers and local governments’ operators shows that developers highly evaluated temperature changes in heat, which is the core principle of thermography cameras. This was particularly understandable by their preference for the detection service of the pre-fire phenomenon. This is consistent with previous studies of such areas as fire propagation, volcanic monitoring, and building thermal bridge diagnosis [22,40,41,42,43,44,46,47]. Meanwhile, local governments highly evaluated the controlling situations related to citizens’ safety within the thermography camera service area. Service decisions must consider the opinions of both developers with a high understanding of technology, and operators who actually introduce and manage services.
This study contributes to the literature and provides the basic foundation for the development of services utilizing thermography cameras by presenting thermography camera-based abnormal situation detection services and selection methods. Importantly, our study considers involving both developers and operators in decision-making when using sensors. We expect that the effectiveness and practicality of services using thermography cameras will further increase through future studies, such as those on spatial analysis, service implementation plans, and test bed demonstration. Together, these efforts can result in a stronger social safety net that allows citizens to lead safe lives.

Author Contributions

Conceptualization, W.C.; methodology, W.C., B.J. and I.J.; software, W.C. and H.S.; validation, B.J. and I.J.; formal analysis, W.C. and Y.J.; investigation, W.C.; resources, W.C.; data curation, H.S. and Y.J.; writing—original draft preparation, W.C.; writing—review and editing, W.C. and B.J.; visualization, W.C. and I.J.; supervision, W.C.; project administration, W.C. All authors have read and agreed to the published version of the manuscript.

Funding

Research for this paper was carried out under the KICT Research Program (project no. 20230117-001) funded by the Ministry of Science and ICT.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Levine, E.S.; Tisch, J.; Tasso, A.; Joy, M. The New York City Police Department’s domain awareness system. Interfaces 2017, 47, 70–84. [Google Scholar] [CrossRef]
  2. Li, Z.; Guo, Y.; Yarime, M.; Wu, X. Policy designs for adaptive governance of disruptive technologies: The case of facial recognition technology (FRT) in China. Policy Des. Pract. 2023, 6, 27–40. [Google Scholar] [CrossRef]
  3. Choi, W.; Kim, T.; Na, J.; Youn, J. Assessment of Dynamic Object Information Utilization Service in a Control Center for Each Urban Scale via Fuzzy AHP. Systems 2023, 11, 368. [Google Scholar] [CrossRef]
  4. Ahmed, S.; Bhatti, M.T.; Khan, M.G.; Lövström, B.; Shahid, M. Development and Optimization of Deep Learning Models for Weapon Detection in Surveillance Videos. Appl. Sci. 2022, 12, 5772. [Google Scholar] [CrossRef]
  5. Ratcliffe, J. Video Surveillance of Public Places; US Department of Justice, Office of Community Oriented Policing Services: Washington, DC, USA, 2006. Available online: https://www.ojp.gov/ncjrs/virtual-library/abstracts/video-surveillance-public-places (accessed on 20 June 2023).
  6. Park, E.S.; Kim, K.Y.; Seong, D.S.; Lee, K.B. Implementation of CCTV security services using GPS precision improvement. J. Korean Inst. Inf. Technol. 2014, 12, 187–202. [Google Scholar] [CrossRef]
  7. Lee, W.S. A study on legal issues of crime prevention with advanced science technology. Korean Criminol. Rev. 2016, 27, 231–262. [Google Scholar]
  8. Nam, G.H.; Shim, H.S. Technology and its effect on policing in South Korea. J. Police Policies 2017, 31, 1–42. [Google Scholar]
  9. You, J.D. Study on construction safe smart city from crime. Korean Assoc. Police Sci. Rev. 2017, 19, 199–222. [Google Scholar]
  10. Sultana, T.; Wahid, K. IoT-Guard: Event-driven fog-based video surveillance system for real-time security management. IEEE Access 2019, 7, 134881–134894. [Google Scholar] [CrossRef]
  11. Park, M.; Lee, H. Smart city crime prevention services: The Incheon Free Economic Zone case. Sustainability 2020, 12, 5658. [Google Scholar] [CrossRef]
  12. Socha, R.; Kogut, B. Urban video surveillance as a tool to improve security in public spaces. Sustainability 2020, 12, 6210. [Google Scholar] [CrossRef]
  13. Khan, P.; Byun, Y.; Park, N. A data verification system for CCTV surveillance cameras using blockchain technology in smart cities. Electronics 2020, 9, 484. [Google Scholar] [CrossRef]
  14. Carreno, P.; Gutierrez, F.; Ochoa, S.; Fortino, G. Supporting personal security using participatory sensing. Concurr. Comput.-Pract. Exp. 2015, 27, 2531–2546. [Google Scholar] [CrossRef]
  15. Lee, H. A study on social issue solutions using the “Internet of Things” (Focusing on a crime prevention camera system). Int. J. Distrib. Sens. Netw. 2015, 11, 747593. [Google Scholar] [CrossRef]
  16. Cha, J.H.; Lee, J.Y.; Lee, J.H. Secure smart safety system using streetlight infrastructure. J. Korean Inst. Commun. Inf. Sci. 2015, 40, 851–856. [Google Scholar]
  17. Hwang, W.; Oh, S.B.; Nam, Y.J.; Ro, S. A Study on the Development of Night Vision Thermal Camera Using Deep Learning. JKICS 2019, 44, 2332–2338. [Google Scholar] [CrossRef]
  18. Park, S.H.; Park, H.M.; Kim, C.B. Design and Implementation of Low-Power RFID System Using Infrared Ray Sensor. J. Korea Inst. Inf. Technol. 2011, 9, 41–48. [Google Scholar]
  19. González, A.; Fang, Z.; Socarras, Y.; Serrat, J.; Vázquez, D.; Xu, J.; López, A.M. Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison. Sensors 2016, 16, 820. [Google Scholar] [CrossRef] [PubMed]
  20. Batchuluun, G.; Nguyen, D.T.; Pham, T.D.; Park, C. Action Recognition From Thermal Videos. IEEE Access 2019, 7, 103893–103917. [Google Scholar] [CrossRef]
  21. Lee, Y.; Kim, S.; Lim, H.; Lee, H.K.; Choo, H.G.; Seo, J.; Yoon, K. Performance Analysis of Object Detection Neural Network According to Compression Ratio of RGB and IR Images. J. Broadcast Eng. 2021, 26, 155–166. [Google Scholar] [CrossRef]
  22. Mozaffari, H.M.; Li, Y.; Ko, Y. Detecting Flashover in a Room Fire Based on the Sequence of Thermal Infrared Images using Convolutional Neural Networks. In Proceedings of the 35th Canadian Conference on Articial Intelligence, Toronto, ON, Canada, 30 May–3 June 2022. [Google Scholar] [CrossRef]
  23. Wong, W.K.; Tan, P.N.; Loo, C.K.; Lim, W.S. An Effective Surveillance System Using Thermal Camera. In Proceedings of the 2009 International Conference on Signal Acquisition and Processing, Kuala Lumpur, Maylaysia, 3–5 April 2009; IEEE: Toulouse, France; p. 10763554. [Google Scholar] [CrossRef]
  24. Wong, W.K.; Chew, Z.Y.; Lim, H.L.; Loo, C.K.; Lim, W.S. Omnidirectional Thermal Imaging Surveillance System Featuring Trespasser and Faint Detection. Int. J. Image Process. 2011, 4, 518–538. [Google Scholar]
  25. Ivašić-Kos, M.; Krišto, M.; Pobar, M. Human Detection in Thermal Imaging Using YOLO. ICCTA. ’19. In Proceedings of the 2019 5th International Conference on Computer and Technology Applications, Istanbul, Turkey, 16–17 April 2019; pp. 20–24. [Google Scholar] [CrossRef]
  26. Teju, V.; Sowmya, K.V.; Kandula, S.R.; Stan, A.; Stan, O.P. A Hybrid Retina Net Classifier for Thermal Imaging. Appl. Sci. 2023, 13, 8525. [Google Scholar] [CrossRef]
  27. Pavez, V.; Hermosilla, G.; Pizarro, F.; Fingerhuth, S.; Yunge, D. Thermal Image Generation for Robust Face Recognition. Appl. Sci. 2022, 12, 497. [Google Scholar] [CrossRef]
  28. Wang, B.; Jiang, X.; Dong, Z.; Li, J. Behavioral Parameter Field for Human Abnormal Behavior Recognition in Low-Resolution Thermal Imaging Video. Appl. Sci. 2022, 12, 402. [Google Scholar] [CrossRef]
  29. Lee, J.M.; An, Y.E.; Bak, E.S.; Pan, S. Improvement of Negative Emotion Recognition in Visible Images Enhanced by Thermal Imaging. Sustainability 2022, 14, 15200. [Google Scholar] [CrossRef]
  30. Rezaee, K.; Khosravi, M.R.; Neshat, N. Deep Transfer Learning-Based Fall Detection Approach Using IoMT-Enabled Thermal Imaging-Assisted Pervasive Surveillance and Big Health Data. J. Circuits Syst. Comput. 2022, 31, 2240005. [Google Scholar] [CrossRef]
  31. Rezaee, K.; Khosravi, M.R.; Moghimi, M.K. Intelligent elderly people fall detection based on modified deep learning deep transfer learning and IoT using thermal imaging-assisted pervasive surveillance. Intell. Healthc. 2022, 113–132. [Google Scholar] [CrossRef]
  32. Munir, F.; Azam, S.; Rafique, M.A.; Sheri, A.M.; Jeon, M.; Pedrycz, W. Exploring thermal images for object detection in underexposure regions for autonomous driving. Appl. Soft Comput. 2022, 121, 108793. [Google Scholar] [CrossRef]
  33. Aparna; Bhatia, Y.; Rai, R.; Gupta, V.; Aggarwal, N.; Akula, A. Convolutional neural networks based potholes detection using thermal imaging. J. King Saud Univ.—Comput. Inf. Sci. 2022, 34, 578–588. [Google Scholar] [CrossRef]
  34. Mowen, D.; Munian, Y.; Alamaniotis, M. Improving Road Safety during Nocturnal Hours by Characterizing Animal Poses Utilizing CNN-Based Analysis of Thermal Images. Sustainability 2022, 14, 12133. [Google Scholar] [CrossRef]
  35. Munian, Y.; Martinez-Molina, A.; Miserlis, D.; Hernandez, H.; Alamaniotis, M. Intelligent System Utilizing HOG and CNN for Thermal Image-Based Detection of Wild Animals in Nocturnal Periods for Vehicle Safety. Appl. Artif. Intell. 2022, 36, 2031825. [Google Scholar] [CrossRef]
  36. Stypułkowski, K.; Gołda, P.; Lewczuk, K.; Tomaszewska, J. Monitoring System for Railway Infrastructure Elements Based on Thermal Imaging Analysis. Sensors 2021, 21, 3819. [Google Scholar] [CrossRef]
  37. Glavaš, H.; Barić, T.; Karakašić, M.; Keser, T. Application of Infrared Thermography in E-Bike Battery Pack Detail Analysis—Case Study. Appl. Sci. 2022, 12, 3444. [Google Scholar] [CrossRef]
  38. Na, S.; Paik, I. Application of Thermal Image Data to Detect Rebar Corrosion in Concrete Structures. Appl. Sci. 2019, 9, 4700. [Google Scholar] [CrossRef]
  39. Kim, C.; Kang, S.; Chung, Y.; Kim, O.; Kim, W. Quantification of the Effective Detectable Period for Concrete Voids of CLP by Lock-In Thermography. Appl. Sci. 2023, 13, 8247. [Google Scholar] [CrossRef]
  40. Kylili, A.; Fokaides, P.A.; Christou, P.; Kalogirou, S.A. Infrared thermography (IRT) applications for building diagnostics: A review. Appl. Energy 2014, 134, 531–549. [Google Scholar] [CrossRef]
  41. Asdrubali, F.; Baldinelli, G.; Bianchi, F.; Costarelli, D.; Rotili, A.; Seracini, M.; Vinti, G. Detection of thermal bridges from thermographic images by means of image processing approximation algorithms. Appl. Math. Comput. 2018, 317, 160–171. [Google Scholar] [CrossRef]
  42. Kirimtat, A.; Krejcar, O. A review of infrared thermography for the investigation of building envelopes: Advances and prospects. Energy Build. 2018, 176, 390–406. [Google Scholar] [CrossRef]
  43. Masri, Y.E.; Rakha, T. A scoping review of non-destructive testing (NDT) techniques in building performance diagnostic inspections. Constr. Build. Mater. 2020, 265, 120542. [Google Scholar] [CrossRef]
  44. Kim, C.; Choi, J.S.; Jang, H.; Kim, E.J. Automatic Detection of Linear Thermal Bridges from Infrared Thermal Images Using Neural Network. Appl. Sci. 2021, 11, 931. [Google Scholar] [CrossRef]
  45. Howell, K.; Dudek, K.; Soroko, M. Thermal Camera Performance and Image Analysis Repeatability in Equine Thermography. Infrared Phys. Technol. 2020, 110, 103447. [Google Scholar] [CrossRef]
  46. Spampinato, L.; Calvari, S.; Oppenheimer, C.; Boschi, E. Volcano surveillance using infrared cameras. Earth-Sci. Rev. 2011, 106, 63–91. [Google Scholar] [CrossRef]
  47. Nazari, R.; Alfergani, H.; Haas, F.; Karimi, M.E.; Fahad, M.G.R.; Sabrin, S.; Everett, J.; Bouaynaya, N.; Peters, R.W. Application of Satellite Remote Sensing in Monitoring Elevated Internal Temperatures of Landfills. Appl. Sci. 2020, 10, 6801. [Google Scholar] [CrossRef]
  48. Gusso, A.; Cafruni, C.; Bordin, F.; Veronez, M.R.; Lenz, L.; Crija, S. Multi-Temporal Patterns of Urban Heat Island as Response to Economic Growth Management. Sustainability 2015, 7, 3129–3145. [Google Scholar] [CrossRef]
  49. Loveday, J.; Loveday, G.; Byrne, J.J.; Ong, B.; Morrison, G.M. Seasonal and Diurnal Surface Temperatures of Urban Landscape Elements. Sustainability 2019, 11, 5280. [Google Scholar] [CrossRef]
  50. Song, B.; Park, K. Analysis of Spatiotemporal Urban Temperature Characteristics by Urban Spatial Patterns in Changwon City, South Korea. Sustainability 2019, 11, 3777. [Google Scholar] [CrossRef]
  51. Zhu, R.; Dong, X.; Wong, M.S. Estimation of the Urban Heat Island Effect in a Reformed Urban District: A Scenario-Based Study in Hong Kong. Sustainability 2022, 14, 4409. [Google Scholar] [CrossRef]
  52. Du, C.; Song, P.; Wang, K.; Li, A.; Hu, Y.; Zhang, K.; Jia, X.; Feng, Y.; Wu, M.; Qu, K.; et al. Investigating the Trends and Drivers between Urbanization and the Land Surface Temperature: A Case Study of Zhengzhou, China. Sustainability 2022, 14, 13845. [Google Scholar] [CrossRef]
  53. Zhao, Z.; Sun, B.; Cheng, G.; Wang, C.; Yang, N.; Wang, H.; Tang, X. A Multitemporal and Multilevel Land Surface Temperature Regional Attribute Change Analysis in Henan, China, Using MODIS Imagery. Sustainability 2022, 14, 10071. [Google Scholar] [CrossRef]
  54. Mantau, A.J.; Widayat, I.W.; Leu, J.S.; Köppen, M. A Human-Detection Method Based on YOLOv5 and Transfer Learning Using Thermal Image Data from UAV Perspective for Surveillance System. Drones 2022, 6, 290. [Google Scholar] [CrossRef]
  55. Kannadaguli, P. YOLO v4 Based Human Detection System Using Aerial Thermal Imaging for UAV Based Surveillance Applications. In Proceedings of the 2020 International Conference on Decision Aid Sciences and Application, Sakheer, Bahrain, 8–9 November 2020; IEEE: Toulouse, France; p. 20364322. [Google Scholar] [CrossRef]
  56. Akshatha, K.R.; Karunakar, A.K.; Shenoy, S.B.; Pai, A.K.; Nagaraj, N.H.; Rohatgi, S.S. Human Detection in Aerial Thermal Images Using Faster R-CNN and SSD Algorithms. Electronics 2022, 11, 1151. [Google Scholar] [CrossRef]
  57. Kannadaguli, P. FCOS Based Human Detection System Using Thermal Imaging for UAV Based Surveillance Applications. In Proceedings of the 2020 IEEE Bombay Section Signature Conference, Virtual, 4–6 December 2020; IEEE: Toulouse, France; p. 20400706. [Google Scholar] [CrossRef]
  58. Gil-Docampo, M.; Sanz, J.O.; Guerrero, I.C.; Cabanas, M.F. UAS IR-Thermograms Processing and Photogrammetry of Thermal Images for the Inspection of Building Envelopes. Appl. Sci. 2023, 13, 3948. [Google Scholar] [CrossRef]
  59. Lee, G.; Kim, G.; Min, G.; Kim, M.; Jung, S.; Hwang, J.; Cho, S. Vegetation Classification in Urban Areas by Combining UAV-Based NDVI and Thermal Infrared Image. Appl. Sci. 2023, 13, 515. [Google Scholar] [CrossRef]
  60. Hasan, M.; Dipto, A.Z.; Islam, M.S.; Sorwar, A.; Alam, S. A Smart Semi-Automated Multifarious Surveillance Bot for Outdoor Security Using Thermal Image Processing. Adv. Netw. 2019, 7, 21–28. [Google Scholar] [CrossRef]
  61. Saaty, T. Priority setting in complex problems. IEEE Trans. Eng. Manag. 1983, 30, 140–155. [Google Scholar] [CrossRef]
  62. Zadeh, L. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef]
  63. Thengane, S.; Hoadley, A.; Bhattacharya, S.; Mitra, S.; Bandyopadhyay, S. Cost-benefit analysis of different hydrogen production technologies using AHP and Fuzzy AHP. Int. J. Hydrogen Energy 2014, 39, 15293–15306. [Google Scholar] [CrossRef]
  64. Tozan, H. Fuzzy AHP based decision support system for technology selection in abrasive water jet cutting processes. Teh. Vjesn.-Tech. Gaz. 2011, 18, 187–191. [Google Scholar]
  65. Jaganathan, S.; Erinjeri, J.; Ker, J. Fuzzy analytic hierarchy process-based group decision support system to select and evaluate new manufacturing technologies. Int. J. Adv. Manuf. Technol. 2007, 32, 1253–1262. [Google Scholar] [CrossRef]
  66. Lee, S.; Mogi, G.; Kim, J. Decision support for prioritizing energy technologies against high oil prices: A fuzzy analytic hierarchy process approach. J. Loss Prev. Process Ind. 2009, 22, 915–920. [Google Scholar] [CrossRef]
  67. Lee, S.; Mogi, G.; Kim, J. Prioritizing the weights of hydrogen energy technologies in the sector of the hydrogen economy by using a fuzzy AHP approach. Int. J. Hydrogen Energy 2011, 36, 1897–1902. [Google Scholar] [CrossRef]
  68. Laarhoven, P.; Pedrycz, W. A fuzzy extension of Saaty’s priority theory. Fuzzy Sets Syst. 1983, 11, 229–241. [Google Scholar] [CrossRef]
  69. Kim, J. Development of CTP Selection Methodology of Equipment Line Using AHP and Fuzzy Decision Model. Ph.D. Thesis, Kumoh National Institute of Technology, Gumi-si, Republic of Korea, 2019. [Google Scholar]
  70. Park, Y. Analyzing the Efficiency of SCM Using Fuzzy-AHP/DEA. Ph.D. Thesis, Yonsei University, Seoul, Republic of Korea, 2013. [Google Scholar]
  71. Chang, D. Applications of extent analysis method on fuzzy AHP. Eur. J. Oper. Res. 1996, 95, 649–655. [Google Scholar] [CrossRef]
  72. Kahraman, C.; Cebeci, U.; Ruan, D. Multi-attribute comparison of catering service companies using fuzzy AHP: The case of Turkey. Int. J. Prod. Econ. 2003, 87, 171–184. [Google Scholar] [CrossRef]
  73. Lee, G. Multiple-Criteria Decision-Making Approach for Flood Vulnerability Assessment Considering the Uncertainties. Ph.D. Thesis, Sungkyunkwan University, Seoul, Republic of Korea, 2013. [Google Scholar]
  74. Lai, Y.; Liu, T.; Hwang, C. TOPSIS for MCDM. Eur. J. Oper. Res. 1994, 76, 486–500. [Google Scholar] [CrossRef]
  75. Chu, T. Selecting Plant Location via a Fuzzy TOPSIS Approach. Int. J. Adv. Manuf. Tech. 2002, 20, 859–864. [Google Scholar] [CrossRef]
  76. Jun, K.; Chung, E.; Sung, J.; Lee, K. Development of Spatial Water Resources Vulnerability Index Considering Climate Change Impact. Sci. Tot. Environ. 2011, 409, 5228–5242. [Google Scholar] [CrossRef]
  77. Kim, G.; Park, C.S.; Yoon, K.P. Identifying investment opportunities for advanced manufacturing systems with comparative-integrated performance measurement. Int. J. Prod. Econ. 1997, 50, 23–33. [Google Scholar] [CrossRef]
  78. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications; Springer: New York, NY, USA, 1981. [Google Scholar]
  79. Sorin, N.; Simona, D.; Ioan, D. Fuzzy TOPSIS: A General View. Procedia Comput. Sci. 2016, 91, 823–831. [Google Scholar]
  80. Chen, C. Extensions of the TOPSIS for group decision-making under fuzzy environment. Fuzzy Sets Syst. 2000, 114, 1–9. [Google Scholar] [CrossRef]
  81. Park, S.W.; Lee, W.J. Improvement of management of long-term care facilities through FGI. J. Korea Contents Assoc. 2019, 19, 587–597. [Google Scholar] [CrossRef]
  82. Kim, C.B.; Hong, W.H.; Jo, Y.B.; Kim, J.D. Extraction of evaluation criteria on technology and service related to smart grid and analysis of relative importance among evaluation criteria by AHP method. Korea Environ. Policy Adm. Soc. 2013, 21, 127–144. [Google Scholar] [CrossRef]
  83. Smart City Korea. Available online: http://smartcitysvc.kict.re.kr (accessed on 10 June 2023).
  84. Korea Technology Finance Corporation. 2021 KOTEC Annual Report; Korea Technology Finance Corporation: Busan, Republic of Korea, 2021. [Google Scholar]
  85. Ministry of Land, Infrastructure and Transport. 2021 Manual for Technology Valuation; Ministry of Land, Infrastructure and Transport: Sejong, Republic of Korea, 2021.
  86. Bieszczad, G.; Krupiński, M.; Madura, H. Thermal Camera for Autonomous Mobile Platforms. Vis. Based Syst. UAV Appl. 2013, 481, 95–114. [Google Scholar] [CrossRef]
  87. Aydin, S.; Yiğit, K.S.; Savci, İ.H. Survey of air battery thermal management on the autonomous mobile robots. Therm. Sci. 2023, 00, 22. [Google Scholar] [CrossRef]
Figure 1. Triangular fuzzy number M.
Figure 1. Triangular fuzzy number M.
Applsci 13 11591 g001
Figure 2. Hierarchy structure.
Figure 2. Hierarchy structure.
Applsci 13 11591 g002
Table 1. Fuzzy scale.
Table 1. Fuzzy scale.
Linguistic ScaleF-No.Fuzzy Triangular ScaleReciprocal F-T-Scale
Equal importance1(1, 1, 1)(1, 1, 1)
Moderate importance3(2/3, 1, 3/2)(2/3, 1, 3/2)
Strong importance5(3/2, 2, 5/2)(2/5, 1/2, 2/3)
Very strong importance7(5/2, 3, 7/2)(2/7, 1/3, 2/5)
Extreme importance9(7/2, 4, 9/2)(2/9, 1/4, 2/7)
Table 2. Service model.
Table 2. Service model.
Major ClassificationMedium ClassificationMinor Classification (Service Model)
A. Object anomaly detectionA-1. Preventive detection of traffic accidents at nightA-1-1. Pedestrian detection at night
A-1-2. Nighttime mobility detection
A-2. Detection of night security incidentsA-2-1. Violence-related incident detection
A-2-2. Detection of crime fear
A-2-3. Intrusion status detection
A-3. Pedestrian hazard detectionA-3-1. Detection of dense pedestrian situation
A-3-2. Calculation of cluster population
B. Road anomaly detectionB-1. Road weather hazard detectionB-1-1. Road surface temperature detection
B-1-2. Road flooding detection
B-1-3. Road ice detection
B-2. Road surface hazard detectionB-2-1. Road pothole detection
B-2-2. Detection of fallen objects on road
C. Facility anomaly detectionC-1. Building status detectionC-1-1. Detection of building exterior damage
C-1-2. Building structure diagnostics
C-2. Detection of precursor fire events in facilityC-2-1. Detection of precursor fire events within buildings
C-2-2. Detection of precursor fire events within facilities
C-3. Detection of facility accident signsC-3-1. Detection of emissions in business places
C-3-2. Detection of explosives at large events
D. Disaster prevention anomaly detectionD-1. Natural disaster prevention detectionD-1-1. Detection of precursor forest fire
D-1-2. River flooding detection
D-2. Heat island detectionD-2-1. Heat island detection
Table 3. Evaluation criteria.
Table 3. Evaluation criteria.
Main CategorySub-CategoryDetails of Evaluation Criteria
Purpose compliancePrevention of life safetyThe level of conforming with the purpose of life safety prevention
CCTV replacement/complementary effectsDetecting anomalies that cannot be solved solely by CCTV images
Conformity to relevant policiesThe level of compliance with government and local government policies
Service suitabilityService performances/effectsPerformance and effectiveness of the service itself (e.g., accident prevention and incident reaction)
Competitiveness versus existing servicesLevel of competitiveness compared with existing facilities/technologies/services
Market demand growthTechnology trends and the extent of market potential in the corresponding service sector
Service feasibilityEconomic efficiencyEffectiveness compared to investment, economic efficiency, and commercialization potential
Field applicabilityRisk management (e.g., personal information and administrative delays) and levels of cooperation with related organizations (e.g., 112, 119, and local governments)
Operational sustainabilityEfficient management operation and long-term service sustainability
Table 4. The second class TFN and weight calculation results.
Table 4. The second class TFN and weight calculation results.
The Second ClassPurpose ComplianceService SuitabilityService Feasibility
Purpose compliance(1, 1, 1)(0.76, 0.97, 1.24)(0.82, 1.04, 1.32)
Service suitability(0.80, 1.03, 1.31)(1, 1, 1)(0.75, 0.98, 1.27)
Service feasibility(0.76, 0.96, 1.21)(0.79, 1.02, 1.33)(1, 1, 1)
TFN ( S 1 ,   S 2 ,   S 3 ) (0.24, 0.34, 0.46)(0.24, 0.33, 0.47)(0.24, 0.33, 0.46)
Weight (normalization)0.3370.3330.331
Table 5. The third class TFN and weight calculation results.
Table 5. The third class TFN and weight calculation results.
Third ClassPrevention of Life SafetyCCTV Replacement/Complementary EffectsConformity to Relevant Policies
Purpose compliancePrevention of life safety(1, 1, 1)(0.93, 1.17, 1.47)(1.17, 1.47, 1.83)
CCTV replacement/complementary effects(0.68, 0.85, 1.07)(1, 1, 1)(0.97, 1.30, 1.72)
Conformity to relevant policies(0.55, 0.85, 1.07)(0.58, 0.77, 1.03)(1, 1, 1)
TFN ( S 1 ,   S 2 ,   S 3 ) (0.28, 0.39, 0.54)(0.24, 0.34, 0.48)(0.19, 0.26, 0.37)
Weight (normalization)0.3940.3370.270
Third ClassService Performances/EffectsCompetitiveness versus Existing ServicesMarket Demand Growth
Service suitabilityService performances/effects(1, 1, 1)(1.13, 1.41, 1.79)(1.03, 1.36, 1.76)
Competitiveness versus existing services(0.56, 0.70, 0.88)(1, 1, 1)(0.77, 0.98, 1.26)
Market demand growth(0.57, 0.73, 0.97)(0.79, 1.02, 1.30)(1, 1, 1)
TFN ( S 1 ,   S 2 ,   S 3 ) (0.29, 0.41, 0.58)(0.21, 0.29, 0.40)(0.22, 0.30, 0.41)
Weight (normalization)0.4020.2950.299
Third ClassEconomic EfficiencyField ApplicabilityOperational Sustainability
Service feasibilityEconomic efficiency(1, 1, 1)(0.69, 0.87, 1.11)(0.74, 0.94, 1.20)
Field applicability(0.90, 1.15, 1.45)(1, 1, 1)(1.22, 1.59, 2.05)
Operational sustainability(0.83, 1.06, 1.35)(0.49, 0.63, 0.82)(1, 1, 1)
TFN ( S 1 ,   S 2 ,   S 3 ) (0.22, 0.30, 0.42)(0.28, 0.40, 0.57)(0.21, 0.29, 0.40)
Weight (normalization)0.3080.3960.295
Table 6. Analysis results of evaluation criteria.
Table 6. Analysis results of evaluation criteria.
Class AClass B
Major ClassificationWeight (Rank)Minor ClassificationWeight (Rank)Final Importance (Rank)
Purpose compliance0.337 (1)Prevention of life safety0.394 (1)0.132 (2)
CCTV replacement/complementary effects0.337 (2)0.113 (4)
Conformity to relevant policies0.270 (3)0.091 (9)
Service suitability0.333 (2)Service performances/effects0.402 (1)0.134 (1)
Competitiveness versus existing services0.295 (3)0.098 (7)
Market demand growth0.299 (2)0.099 (6)
Service feasibility0.331 (3)Economic efficiency0.308 (2)0.102 (5)
Field applicability0.396 (1)0.131 (3)
Operational sustainability0.295 (3)0.097 (8)
Table 7. Triangular fuzzification.
Table 7. Triangular fuzzification.
ClassificationPurpose ComplianceService SuitabilityService Feasibility
Prevention of Life SafetyCCTV Replacement/Complementary EffectsConformity to Relevant PoliciesService Performances/EffectsCompetitiveness versus Existing ServicesMarket Demand GrowthEconomic EfficiencyField ApplicabilityOperational Sustainability
A-1-1.(0.7, 3.1, 4.5)(0.7, 3.1, 4.5)(0.7, 2.8, 4.5)(1.5, 3.6, 4.5)(1.5, 2.9, 4.5)(0.7, 3.0, 4.5)(0.7, 2.8, 4.5)(1.5, 3.2, 4.5)(0.7, 3.0, 4.5)
A-1-2.(0.7, 3.0, 4.5)(0.7, 3.0, 4.5)(0.7, 2.8, 4.5)(1.5, 3.4, 4.5)(1.5, 2.9, 4.5)(0.7, 2.8, 4.5)(0.7, 2.8, 4.5)(1.5, 3.1, 4.5)(0.7, 3.0, 4.5)
A-2-1.(0.7, 2.6, 4.5)(0.7, 2.4, 4.5)(0.7, 2.6, 4.5)(0.7, 2.7, 4.5)(0.7, 2.2, 4.5)(0.7, 2.3, 4.5)(0.7, 1.9, 4.5)(0.7, 2.3, 4.5)(0.7, 2.2, 4.5)
A-2-2.(0.7, 2.4, 4.5)(0.7, 2.0, 4.5)(0.7, 2.2, 4.5)(0.7, 2.5, 4.5)(0.7, 1.9, 4.5)(0.7, 2.2, 4.5)(0.7, 1.7, 4.5)(0.7, 1.9, 4.5)(0.7, 1.8, 4.5)
A-2-3.(0.7, 2.6, 4.5)(0.7, 2.6, 4.5)(0.7, 2.8, 4.5)(0.7, 2.7, 4.5)(0.7, 2.4, 4.5)(0.7, 2.1, 4.5)(0.7, 2.2, 4.5)(0.7, 2.4, 4.5)(0.7, 2.1, 4.5)
A-3-1.(0.7, 2.8, 4.5)(0.7, 2.4, 4.5)(0.7, 2.8, 4.5)(0.7, 3.0, 4.5)(0.7, 2.6, 4.5)(0.7, 2.7, 4.5)(0.7, 2.5, 4.5)(0.7, 2.8, 4.5)(0.7, 2.4, 4.5)
A-3-2.(0.7, 2.8, 4.5)(0.7, 2.2, 4.5)(0.7, 2.8, 4.5)(0.7, 2.8, 4.5)(0.7, 2.4, 4.5)(0.7, 2.5, 4.5)(0.7, 2.4, 4.5)(0.7, 2.8, 4.5)(0.7, 2.4, 4.5)
B-1-1.(0.7, 2.4, 4.5)(0.7, 2.3, 4.5)(0.7, 2.4, 4.5)(0.7, 2.5, 4.5)(0.7, 2.3, 4.5)(0.7, 2.4, 4.5)(0.7, 2.0, 4.5)(0.7, 2.3, 4.5)(0.7, 2.0, 4.5)
B-1-2.(0.7, 2.9, 4.5)(0.7, 2.3, 4.5)(0.7, 2.7, 4.5)(0.7, 2.7, 4.5)(0.7, 2.1, 4.5)(0.7, 2.5, 4.5)(0.7, 2.2, 4.5)(0.7, 2.6, 4.5)(0.7, 2.1, 4.5)
B-1-3.(0.7, 3.0, 4.5)(0.7, 3.0, 4.5)(0.7, 2.7, 4.5)(0.7, 2.8, 4.5)(0.7, 2.8, 4.5)(0.7, 2.7, 4.5)(0.7, 2.4, 4.5)(0.7, 2.7, 4.5)(0.7, 2.3, 4.5)
B-2-1.(0.7, 2.4, 4.5)(0.7, 2.0, 4.5)(0.7, 2.4, 4.5)(0.7, 2.3, 4.5)(0.7, 1.9, 4.5)(0.7, 2.3, 4.5)(0.7, 1.9, 4.5)(0.7, 2.1, 4.5)(0.7, 2.0, 4.5)
B-2-2.(0.7, 2.4, 4.5)(0.7, 1.7, 4.5)(0.7, 2.1, 3.5)(0.7, 2.2, 4.5)(0.7, 2.0, 4.5)(0.7, 2.0, 4.5)(0.7, 2.0, 4.5)(0.7, 2.0, 4.5)(0.7, 2.0, 3.5)
C-1-1.(0.7, 2.0, 4.5)(0.7, 1.9, 4.5)(0.7, 1.8, 3.5)(0.7, 2.3, 4.5)(0.7, 2.1, 4.5)(0.7, 2.2, 4.5)(0.7, 1.8, 4.5)(0.7, 2.0, 4.5)(0.7, 2.0, 4.5)
C-1-2.(0.7, 1.9, 3.5)(0.7, 1.9, 4.5)(0.7, 1.9, 4.5)(0.7, 2.3, 4.5)(0.7, 2.0, 4.5)(0.7, 2.2, 4.5)(0.7, 1.9, 4.5)(0.7, 2.1, 4.5)(0.7, 2.0, 4.5)
C-2-1.(0.7, 3.1, 4.5)(0.7, 3.0, 4.5)(0.7, 2.7, 4.5)(1.5, 3.3, 4.5)(0.7, 3.1, 4.5)(0.7, 2.8, 4.5)(0.7, 2.7, 4.5)(0.7, 3.2, 4.5)(0.7, 2.8, 4.5)
C-2-2.(0.7, 3.3, 4.5)(0.7, 3.3, 4.5)(0.7, 2.8, 4.5)(1.5, 3.4, 4.5)(0.7, 3.2, 4.5)(0.7, 2.8, 4.5)(0.7, 2.8, 4.5)(0.7, 3.1, 4.5)(0.7, 2.8, 4.5)
C-3-1.(0.7, 2.4, 4.5)(0.7, 2.6, 4.5)(0.7, 2.6, 4.5)(0.7, 2.7, 4.5)(0.7, 2.6, 4.5)(0.7, 2.2, 4.5)(0.7, 2.1, 4.5)(0.7, 2.4, 4.5)(0.7, 2.2, 4.5)
C-3-2.(0.7, 2.4, 4.5)(0.7, 2.5, 4.5)(0.7, 2.2, 4.5)(0.7, 2.5, 4.5)(0.7, 2.3, 4.5)(0.7, 1.9, 3.5)(0.7, 1.9, 4.5)(0.7, 2.2, 4.5)(0.7, 2.0, 4.5)
D-1-1.(0.7, 3.2, 4.5)(0.7, 3.1, 4.5)(1.5, 3.1, 4.5)(0.7, 3.1, 4.5)(0.7, 2.8, 4.5)(0.7, 2.6, 4.5)(0.7, 2.8, 4.5)(0.7, 2.8, 4.5)(0.7, 2.6, 4.5)
D-1-2.(0.7, 2.6, 4.5)(0.7, 2.2, 4.5)(0.7, 2.8, 4.5)(0.7, 2.5, 4.5)(0.7, 2.1, 4.5)(0.7, 2.1, 3.5)(0.7, 2.2, 4.5)(0.7, 2.4, 4.5)(0.7, 2.2, 4.5)
D-2-1.(0.7, 2.4, 4.5)(0.7, 3.1, 4.5)(0.7, 2.4, 4.5)(0.7, 3.0, 4.5)(0.7, 2.7, 4.5)(0.7, 2.2, 4.5)(0.7, 2.3, 4.5)(0.7, 2.2, 4.5)(0.7, 2.2, 4.5)
Table 8. FPIS and FNIS analysis results.
Table 8. FPIS and FNIS analysis results.
ClassificationPurpose ComplianceService SuitabilityService Feasibility
Prevention of Life SafetyCCTV Replacement/Complementary EffectsConformity to Relevant PoliciesService Performances/EffectsCompetitiveness versus Existing ServicesMarket Demand GrowthEconomic EfficiencyField ApplicabilityOperational Sustainability
Weights of the evaluation criteria(0.28, 0.39, 0.54)(0.24, 0.34, 0.48)(0.19, 0.26, 0.37)(0.29, 0.41, 0.58)(0.21, 0.29, 0.40)(0. 2, 0.30, 0.41)(0. 2, 0.30, 0.42)(0.28, 0.40, 0.57)(0.21, 0.29, 0.40)
A-1-1.(0.04, 0.27, 0.54)(0.04, 0.24, 0.48)(0.03, 0.17, 0.37)(0.10, 0.32, 0.58)(0.07, 0.19, 0.40)(0.03, 0.20, 0.41)(0.03, 0.30, 0.42)(0.09, 0.28, 0.57)(0.03, 0.20, 0.40)
A-1-2.(0.04, 0.27, 0.54)(0.04, 0.23, 0.48)(0.03, 0.16, 0.37)(0.10, 0.31, 0.58)(0.07, 0.19, 0.40)(0.03, 0.19, 0.41)(0.03, 0.30, 0.42)(0.09, 0.28, 0.57)(0.03, 0.20, 0.40)
A-2-1.(0.04, 0.23, 0.54)(0.04, 0.19, 0.48)(0.03, 0.16, 0.37)(0.04, 0.24, 0.58)(0.03, 0.14, 0.40)(0.03, 0.15, 0.41)(0.03, 0.20, 0.42)(0.04, 0.21, 0.57)(0.03, 0.14, 0.40)
A-2-2.(0.04, 0.21, 0.54)(0.04, 0.15, 0.48)(0.03, 0.13, 0.37)(0.04, 0.23, 0.58)(0.03, 0.12, 0.40)(0.03, 0.15, 0.41)(0.03, 0.18, 0.42)(0.04, 0.17, 0.57)(0.03, 0.12, 0.40)
A-2-3.(0.04, 0.23, 0.54)(0.04, 0.20, 0.48)(0.03, 0.16, 0.37)(0.04, 0.25, 0.58)(0.03, 0.15, 0.40)(0.03, 0.14, 0.41)(0.03, 0.23, 0.42)(0.04, 0.21, 0.57)(0.03, 0.14, 0.40)
A-3-1.(0.04, 0.24, 0.54)(0.04, 0.18, 0.48)(0.03, 0.17, 0.37)(0.04, 0.28, 0.58)(0.03, 0.17, 0.40)(0.03, 0.18, 0.41)(0.03, 0.27, 0.42)(0.04, 0.26, 0.57)(0.03, 0.16, 0.40)
A-3-2.(0.04, 0.24, 0.54)(0.04, 0.17, 0.48)(0.03, 0.17, 0.37)(0.04, 0.26, 0.58)(0.03, 0.16, 0.40)(0.03, 0.16, 0.41)(0.03, 0.25, 0.42)(0.04, 0.26, 0.57)(0.03, 0.15, 0.40)
B-1-1.(0.04, 0.21, 0.54)(0.04, 0.18, 0.48)(0.03, 0.14, 0.37)(0.04, 0.23, 0.58)(0.03, 0.15, 0.40)(0.03, 0.16, 0.41)(0.03, 0.21, 0.42)(0.04, 0.21, 0.57)(0.03, 0.13, 0.40)
B-1-2.(0.04, 0.25, 0.54)(0.04, 0.18, 0.48)(0.03, 0.16, 0.37)(0.04, 0.24, 0.58)(0.03, 0.13, 0.40)(0.03, 0.16, 0.41)(0.03, 0.24, 0.42)(0.04, 0.23, 0.57)(0.03, 0.14, 0.40)
B-1-3.(0.04, 0.26, 0.54)(0.08, 0. 2, 0.48)(0.03, 0.16, 0.37)(0.04, 0.26, 0.58)(0.03, 0.18, 0.40)(0.03, 0.18, 0.41)(0.03, 0.25, 0.42)(0.04, 0.24, 0.57)(0.03, 0.15, 0.40)
B-2-1.(0.04, 0.21, 0.54)(0.04, 0.15, 0.48)(0.03, 0.14, 0.37)(0.04, 0.21, 0.58)(0.03, 0.12, 0.40)(0.03, 0.15, 0.41)(0.03, 0.21, 0.42)(0.04, 0.19, 0.57)(0.03, 0.13, 0.40)
B-2-2.(0.04, 0.21, 0.54)(0.04, 0.13, 0.48)(0.03, 0.12, 0.28)(0.04, 0.20, 0.58)(0.03, 0.13, 0.40)(0.03, 0.13, 0.41)(0.03, 0.21, 0.42)(0.04, 0.18, 0.57)(0.03, 0.13, 0.31)
C-1-1.(0.04, 0.17, 0.54)(0.04, 0.14, 0.48)(0.03, 0.10, 0.28)(0.04, 0.21, 0.58)(0.03, 0.13, 0.40)(0.03, 0.15, 0.41)(0.03, 0.19, 0.42)(0.04, 0.18, 0.57)(0.03, 0.13, 0.40)
C-1-2.(0.04, 0.17, 0.42)(0.04, 0.15, 0.48)(0.03, 0. 1, 0.37)(0.04, 0.21, 0.58)(0.03, 0.13, 0.40)(0.03, 0.15, 0.41)(0.03, 0.20, 0.42)(0.04, 0.19, 0.57)(0.03, 0.13, 0.40)
C-2-1.(0.04, 0.27, 0.54)(0.08, 0.23, 0.48)(0.03, 0.16, 0.37)(0.10, 0.30, 0.58)(0.03, 0.20, 0.40)(0.03, 0.18, 0.41)(0.03, 0.29, 0.42)(0.04, 0.28, 0.57)(0.03, 0.18, 0.40)
C-2-2.(0.04, 0.29, 0.54)(0.08, 0.25, 0.48)(0.03, 0.16, 0.37)(0.10, 0.31, 0.58)(0.03, 0.21, 0.40)(0.03, 0.19, 0.41)(0.03, 0.30, 0.42)(0.04, 0.28, 0.57)(0.03, 0.18, 0.40)
C-3-1.(0.04, 0.21, 0.54)(0.04, 0.20, 0.48)(0.03, 0.15, 0.37)(0.04, 0.25, 0.58)(0.03, 0.17, 0.40)(0.03, 0.15, 0.41)(0.03, 0. 2, 0.42)(0.04, 0. 2, 0.57)(0.03, 0.14, 0.40)
C-3-2.(0.04, 0.21, 0.54)(0.04, 0.19, 0.48)(0.03, 0.13, 0.37)(0.04, 0.23, 0.58)(0.03, 0.15, 0.40)(0.03, 0.13, 0.32)(0.03, 0.21, 0.42)(0.04, 0.19, 0.57)(0.03, 0.13, 0.40)
D-1-1.(0.04, 0.28, 0.54)(0.04, 0.23, 0.48)(0.06, 0.18, 0.37)(0.04, 0.28, 0.58)(0.03, 0.18, 0.40)(0.03, 0.17, 0.41)(0.03, 0.30, 0.42)(0.04, 0.25, 0.57)(0.03, 0.17, 0.40)
D-1-2.(0.04, 0.23, 0.54)(0.04, 0.16, 0.48)(0.03, 0.16, 0.37)(0.04, 0.23, 0.58)(0.03, 0.13, 0.40)(0.03, 0.14, 0.32)(0.03, 0.23, 0.42)(0.04, 0. 2, 0.57)(0.03, 0.14, 0.40)
D-2-1.(0.04, 0.21, 0.54)(0.04, 0.24, 0.48)(0.03, 0.14, 0.37)(0.04, 0.28, 0.58)(0.03, 0.17, 0.40)(0.03, 0.15, 0.41)(0.03, 0.25, 0.42)(0.04, 0.20, 0.57)(0.03, 0.14, 0.40)
A*(0.04, 0.27, 0.54)(0.08, 0.25, 0.48)(0.06, 0.18, 0.37)(0.10, 0.32, 0.58)(0.07, 0.21, 0.40)(0.03, 0.20, 0.41)(0.03, 0.30, 0.42)(0.09, 0.28, 0.57)(0.03, 0.20, 0.40)
A(0.04, 0.17, 0.42)(0.04, 0.13, 0.47)(0.03, 0.10, 0.28)(0.04, 0.20, 0.58)(0.03, 0.12, 0.40)(0.03, 0.13, 0.32)(0.03, 0.18, 0.42)(0.04, 0.17, 0.57)(0.03, 0.12, 0.31)
Table 9. Results of FPIS distance analysis per alternative.
Table 9. Results of FPIS distance analysis per alternative.
ClassificationPurpose ComplianceService SuitabilityService FeasibilityFPIS
( d i * )
Prevention of Life SafetyCCTV Replacement/Complementary EffectsConformity to Relevant PoliciesService Performances/EffectsCompetitiveness versus Existing ServicesMarket Demand GrowthEconomic EfficiencyField ApplicabilityOperational Sustainability
A-1-1.0.0080.0270.0230.0000.0100.0000.0020.0000.0000.071
A-1-2.0.0120.0290.0230.0080.0120.0050.0050.0020.0000.096
A-2-1.0.0320.0450.0260.0560.0450.0240.0590.0530.0310.373
A-2-2.0.0420.0620.0380.0650.0530.0290.0690.0710.0450.473
A-2-3.0.0340.0380.0240.0540.0390.0320.0420.0510.0340.349
A-3-1.0.0260.0460.0230.0410.0330.0110.0200.0350.0220.257
A-3-2.0.0260.0520.0230.0490.0360.0180.0300.0350.0250.295
B-1-1.0.0420.0490.0320.0630.0400.0230.0520.0530.0390.393
B-1-2.0.0200.0490.0260.0560.0470.0180.0400.0440.0340.334
B-1-3.0.0160.0140.0260.0510.0280.0110.0300.0390.0270.241
B-2-1.0.0420.0600.0330.0720.0540.0240.0570.0620.0390.444
B-2-2.0.0440.0730.0620.0800.0500.0350.0540.0670.0650.532
C-1-1.0.0670.0670.0690.0740.0470.0270.0670.0690.0400.528
C-1-2.0.0980.0650.0460.0720.0500.0290.0590.0640.0390.522
C-2-1.0.0080.0110.0250.0130.0230.0080.0070.0300.0100.135
C-2-2.0.0000.0000.0230.0110.0230.0060.0000.0310.0100.104
C-3-1.0.0470.0390.0280.0540.0320.0270.0470.0480.0310.354
C-3-2.0.0470.0420.0380.0650.0410.0660.0570.0600.0370.453
D-1-1.0.0020.0270.0000.0400.0260.0120.0020.0370.0160.164
D-1-2.0.0320.0550.0240.0650.0470.0620.0420.0500.0300.408
D-2-1.0.0440.0270.0310.0410.0300.0290.0320.0570.0310.323
Table 10. Results of FNIS distance analysis per alternative.
Table 10. Results of FNIS distance analysis per alternative.
ClassificationPurpose ComplianceService SuitabilityService FeasibilityFNIS
( d i )
Prevention of Life SafetyCCTV Replacement/Complementary EffectsConformity to Relevant PoliciesService Performances/EffectsCompetitiveness versus Existing ServicesMarket Demand GrowthEconomic EfficiencyField ApplicabilityOperational Sustainability
A-1-1.0.0930.0610.0590.0800.0450.0660.0670.0710.0680.611
A-1-2.0.0900.0560.0590.0720.0440.0640.0640.0690.0680.586
A-2-1.0.0790.0320.0560.0270.0100.0550.0100.0210.0530.343
A-2-2.0.0750.0120.0490.0170.0010.0540.0000.0000.0520.260
A-2-3.0.0780.0400.0580.0290.0180.0540.0270.0230.0530.380
A-3-1.0.0820.0300.0590.0460.0250.0610.0490.0480.0560.457
A-3-2.0.0820.0230.0590.0360.0210.0570.0400.0480.0550.420
B-1-1.0.0750.0260.0520.0190.0160.0560.0170.0210.0520.334
B-1-2.0.0850.0260.0560.0270.0070.0570.0300.0330.0530.375
B-1-3.0.0870.0600.0560.0340.0330.0610.0400.0390.0550.464
B-2-1.0.0750.0140.0510.0080.0000.0550.0120.0100.0520.278
B-2-2.0.0740.0000.0110.0000.0040.0530.0150.0040.0040.166
C-1-1.0.0700.0070.0000.0060.0070.0550.0020.0020.0520.201
C-1-2.0.0000.0090.0470.0080.0040.0540.0100.0080.0520.193
C-2-1.0.0930.0630.0570.0680.0450.0620.0620.0640.0620.576
C-2-2.0.0980.0730.0590.0700.0490.0630.0690.0620.0620.606
C-3-1.0.0730.0390.0540.0290.0270.0550.0220.0270.0530.379
C-3-2.0.0730.0350.0490.0170.0150.0000.0120.0120.0520.266
D-1-1.0.0970.0600.0690.0480.0360.0600.0670.0440.0590.538
D-1-2.0.0790.0190.0580.0170.0070.0080.0270.0250.0540.294
D-2-1.0.0740.0610.0520.0460.0300.0540.0370.0170.0530.425
Table 11. Fuzzy TOPSIS analysis results for all respondents.
Table 11. Fuzzy TOPSIS analysis results for all respondents.
Major ClassificationMedium ClassificationMinor Classification (Service Model)FPIS
( d i * )
FNIS
( d i )
C C i Rank
A.
Object anomaly detection
A-1. Preventive detection of traffic accidents at nightA-1-1. Pedestrian detection at night0.0710.6110.8961
A-1-2. Nighttime mobility detection0.0960.5860.8592
A-2. Detection of night security incidentsA-2-1. Violence-related incident detection0.3730.3430.47913
A-2-2. Detection of crime fear0.4730.2600.35518
A-2-3. Intrusion status detection0.3490.3800.52111
A-3. Pedestrian hazard detectionA-3-1. Detection of dense pedestrian situation0.2570.4570.6407
A-3-2. Calculation of cluster population0.2950.4200.5888
B.
Road anomaly detection
B-1. Road weather hazard detectionB-1-1. Road surface temperature detection0.3930.3340.45914
B-1-2. Road flooding detection0.3340.3750.52910
B-1-3. Road ice detection0.2410.4640.6586
B-2. Road surface hazard detectionB-2-1. Road pothole detection0.4440.2780.38516
B-2-2. Detection of fallen objects on road0.5320.1660.23821
C.
Facility anomaly detection
C-1. Building status detectionC-1-1. Detection of building facade damage0.5280.2010.27619
C-1-2. Building structure diagnostics0.5220.1930.27020
C-2. Detection of precursor fire events in facilityC-2-1. Detection of precursor fire events within buildings0.1350.5760.8104
C-2-2. Detection of precursor fire events in facilities0.1040.6060.8543
C-3. Detection of facility accident symptomsC-3-1. Detection of emissions in business places0.3540.3790.51712
C-3-2. Detection of explosives at large events0.4530.2660.37017
D.
Disaster prevention anomaly detection
D-1. Natural disaster prevention detectionD-1-1. Detection of precursor forest fire0.1640.5380.7675
D-1-2. River flooding detection0.4080.2940.41915
D-2. Heat island detectionD-2-1. Heat island detection0.3230.4250.5689
Table 12. Comparison of analysis results between developers and local government operators.
Table 12. Comparison of analysis results between developers and local government operators.
Major ClassificationMedium
Classification
Minor
Classification (Service Model)
Developer
(n = 13)
Local Government
Operator (n = 12)
C C i Rank C C i Rank
A.
Object anomaly detection
A-1. Preventive detection of traffic accidents at nightA-1-1. Pedestrian detection at night0.80630.8721
A-1-2. Nighttime mobility detection0.80540.8212
A-2. Detection of night security incidentsA-2-1. Violence-related incident detection0.512110.55212
A-2-2. Detection of crime fear0.463140.42519
A-2-3. Intrusion status detection0.367200.7993
A-3. Pedestrian hazard detectionA-3-1. Detection of dense pedestrian situation0.59880.7107
A-3-2. Calculation of cluster population0.58390.6479
B.
Road anomaly detection
B-1. Road weather hazard detectionB-1-1. Road surface temperature detection0.500120.43918
B-1-2. Road flooding detection0.531100.58111
B-1-3. Road ice detection0.63650.59710
B-2. Road surface hazard detectionB-2-1. Road pothole detection0.435160.51816
B-2-2. Detection of fallen objects on road0.168210.48317
C.
Facility anomaly detection
C-1. Building status detectionC-1-1. Detection of building facade damage0.375180.40820
C-1-2. Building structure diagnostics0.439150.32221
C-2. Detection of precursor fire events in facilityC-2-1. Detection of precursor fire events within buildings0.89210.7068
C-2-2. Detection of precursor fire events in facilities0.83620.7435
C-3. Detection of facility accident symptomsC-3-1. Detection of emissions in business places0.465130.7116
C-3-2. Detection of explosives at large events0.388170.54615
D.
Disaster prevention anomaly detection
D-1. Natural disaster prevention detectionD-1-1. Detection of precursor forest fire0.63360.7684
D-1-2. River flooding detection0.369190.54714
D-2. Heat island detectionD-2-1. Heat island detection0.60170.54913
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Choi, W.; Jang, B.; Jung, I.; Sung, H.; Jang, Y. Evaluation of Preferences for a Thermal-Camera-Based Abnormal Situation Detection Service via the Integrated Fuzzy AHP/TOPSIS Model. Appl. Sci. 2023, 13, 11591. https://doi.org/10.3390/app132011591

AMA Style

Choi W, Jang B, Jung I, Sung H, Jang Y. Evaluation of Preferences for a Thermal-Camera-Based Abnormal Situation Detection Service via the Integrated Fuzzy AHP/TOPSIS Model. Applied Sciences. 2023; 13(20):11591. https://doi.org/10.3390/app132011591

Chicago/Turabian Style

Choi, Woochul, Bongjoo Jang, Intaek Jung, Hongki Sung, and Younmi Jang. 2023. "Evaluation of Preferences for a Thermal-Camera-Based Abnormal Situation Detection Service via the Integrated Fuzzy AHP/TOPSIS Model" Applied Sciences 13, no. 20: 11591. https://doi.org/10.3390/app132011591

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop