Next Article in Journal
Evaluation of the Seismic Retrofitting of Mainshock-Damaged Reinforced Concrete Frame Structure Using Steel Braces with Soft Steel Dampers
Previous Article in Journal
Additive Manufacturing in Industry
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 11
Issue 2
10.3390/app11020842
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Meta-Analysis and Systematic Review of the Application of Machine Learning Classifiers in Biomedical Applications of Infrared Thermography

by
Carolina Magalhaes
1,2,
Joaquim Mendes
1,2 and
Ricardo Vardasca
1,2,3,4,*
1
LABIOMEP, INEGI, R. Dr. Roberto Frias 400, 4200-465 Porto, Portugal
2
Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias S/N, 4200-465 Porto, Portugal
3
ISLA Santarém, Largo Cândido dos Reis, 2000-241 Santarém, Portugal
4
Faculty of Computing, Engineering and Science, University of South Wales, Pontypridd CF37 1DL, UK
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(2), 842; https://doi.org/10.3390/app11020842
Submission received: 15 December 2020 / Revised: 13 January 2021 / Accepted: 14 January 2021 / Published: 18 January 2021
(This article belongs to the Section Applied Biosciences and Bioengineering)
Browse Figures
Review Reports Versions Notes

Abstract

:
Atypical body temperature values can be an indication of abnormal physiological processes associated with several health conditions. Infrared thermal (IRT) imaging is an innocuous imaging modality capable of capturing the natural thermal radiation emitted by the skin surface, which is connected to physiology-related pathological states. The implementation of artificial intelligence (AI) methods for interpretation of thermal data can be an interesting solution to supply a second opinion to physicians in a diagnostic/therapeutic assessment scenario. The aim of this work was to perform a systematic review and meta-analysis concerning different biomedical thermal applications in conjunction with machine learning strategies. The bibliographic search yielded 68 records for a qualitative synthesis and 34 for quantitative analysis. The results show potential for the implementation of IRT imaging with AI, but more work is needed to retrieve significant features and improve classification metrics.

1. Introduction

Atypical body temperature value can be an indication of an abnormal physiological process or health issue. Infrared thermal (IRT) imaging, also known as thermography, is an imaging modality capable of capturing the natural thermal radiation emitted by an object through the use of an IR camera [1]. In the case of the human body, this heat emission is mainly dependent on underlying skin structures pertaining to the vascular and nervous system [2]. This technique can be implemented in a static manner, by performing image capture without stressing the skin area under assessment, or dynamically, through the application in advance of a chemical, thermal or mechanical stimulus [3]. In either case, the international accepted guidelines should be followed to assure proper measurements and reproducibility of temperature readings [4,5]. Its innocuousness, combined with the ability to perform a fast analysis at a low price, have made it an interesting method to assist physicians in diagnosis and/or treatment monitoring of several pathologies, such as diabetic foot [6], rheumatoid arthritis [7], breast tumors [8] and skin cancer [9].
The acquired temperature matrixes, i.e., thermograms, require image processing strategies to retrieve temperature data that is meaningful for the condition under analysis. Still, the understanding of this information is a challenging and time-consuming task that can be eased through the implementation of artificial intelligence (AI) computational methods based on machine learning (ML) algorithms [10]. Over the last years, thermal information has been applied by researchers as input features for AI classifiers to give physicians a second opinion [11]. Support Vector Machines (SVM), Artificial Neural Networks (ANN), Decision Trees (DT), AdaBoost, k-Nearest Neighbour (k-NN), Naïve Bayes (NB), Fuzzy methods and Random Forest (RF) are some of the examples of this type of learning. The success of its implementation is commonly based on the metrics of accuracy (ACC), sensitivity (SN) and specificity (SP), with higher rates being an indication of a good ML algorithm. It is, however, of great importance to examine performance metrics as a whole and not individually in order to avoid drawing poor conclusions [12].
The aim of this work was to assess the biomedical applications of thermal image data with ML strategies. A systematic review is performed along with a meta-analysis based on reported classification metrics.

2. Materials and Methods

The systematic review and meta-analysis were conducted following the PRISMA (Preferred Reporting Items for Systematic re-views and Meta-Analyses) guidelines [13,14]. Relevant studies concerning the use of biomedical thermal data in conjunction with ML algorithms were carefully selected and appraised, being part of the retrieved data used to summarize their findings through statistical methods.

2.1. Information Sources

The reference sources Scopus, PubMed, ISI Web of Science and IEEE Xplore were used for the bibliographic search. The syntax (TITLE- ABS- KEY (machine learning OR (machine classification) OR (artificial intelligence))) AND TITLE- ABS- KEY (thermography OR (infrared imaging) OR (thermal imaging))) AND TITLE- ABS- KEY (biomedical)), ((machine learning[Title/Abstract] OR (machine classification[Title/ Abstract]) OR (artificial intelligence[Title/Abstract])) AND (thermography[Title/Abstract] OR (infrared imaging[Title/ Abstract]) OR (thermal imaging[Title/Abstract]) AND (biomedical[Title/Abstract]), TOPIC: (machine learning OR (machine classification) OR (artificial intelligence)) AND TOPIC: (thermography OR (infrared imaging) OR (thermal imaging)) AND TOPIC: (biomedical) and (((((((“All Metadata”:thermography) OR “All Metadata”:infrared imaging) OR “All Metadata”:thermal imaging) AND “All Metadata”:machine learning) OR “All Metadata”:machine classification) OR “All Metadata”:artificial intelligence) AND “All Metadata”:biomedical) was used, respectively. The database Google Scholar was also searched with the same keywords combination. To guarantee that the highest possible number of articles was found, the operator OR was employed to include articles that applied alternative terms and/or expressions to “thermography” and “machine learning”. No time restriction was imposed, and publications were included from the first available date to March 2020. A duplicate removal was performed after the bibliographic search.

2.2. Eligibility Criteria and Screening

After article selection, the title and abstracts were analyzed to exclude publications that did not refer to the use of biomedical thermal data for ML applications. The first eligibility criterion excluded publications that applied infrared imaging in the near and/or medium IR section of the spectrum. Articles where artificial intelligence strategies were not used to the assessment of a given pathology and/or treatment were also removed. The third eligibility criterion kept only papers written in English. Reviews and opinion articles were eliminated, as well as publications where the full text was unavailable online and was still unavailable upon contact with authors—these constituted the fourth and fifth criterion, respectively. Lastly, papers that did not present any quantitative metrics of classification were also disregarded.

3. Results

The bibliographic research yielded 429 records after the removal of duplicates. The title and abstract screening excluded an additional 305 publications that did not mention the application of artificial intelligence algorithms to biomedical thermal data. From the remaining 124 articles, 27 were excluded due to the application of near and/or medium infrared spectroscopy and 13 due to the implementation of learners on tasks non-related to pathology appraisal. The third criterion discarded four articles, and nine reviews were removed based on criterion number four. Finally, three papers were not considered for this systematic review and meta-analysis, since no performance metrics were reported. The remaining 68 articles underwent a qualitative synthesis, being divided by assessed disease: breast cancer (39), skin neoplasm (3), diabetes disease (6) and other conditions (19). The latter included studies concerning pathologies mentioned in less than three publications. The quantitative synthesis was performed for 34 of the 68 encountered publications and individually for breast cancer (22 studies) and diabetic foot (three studies), since not all records reported the necessary metrics for the meta-analysis process and/or focused on the diagnosis of a given pathology. A flow-diagram describing the phases of this systematic-review and meta-analysis is presented in Figure 1 [13].
The information retrieved from the encountered studies is summarized in Appendix A.

3.1. Qualitative Synthesis

3.1.1. Breast Cancer

Breast cancer diagnosis is the topic with the highest number of publications for the application of thermal imaging in conjunction with ML algorithms. Ng et al. presented the first studies using ANN to detect breast malignancies, but with low performance metrics of ACC, SN and SP (58.5%, 54% and 67%) [15,16]. These results were improved further along with a more elaborate approach focused on the use of biostatistical methods (SN = 81.2%, SP = 88.2%) [17]. The main author also collaborated in the work of Tan et al. where fuzzy rules were implemented for the construction of hidden layers in a neural network, yielding good SN results (100%) in the proposed classification tasks [18]. The improvement of classification parameters when fuzzy-neural networks are selected in place of single neural networks was also confirmed by another author’s research [19,20]. Schaefer et al. studied the number of ideal partitions in a fuzzy-rule based classification system for breast cancer, reaching ACC, SN and SP values of 97.95%, 93.10% and 99.15%, respectively, as the number of partitions increased [21,22]. A fuzzy model based on C-means clustering by Lashkari et al. showed accuracy values of 75% for the screening of suspicious breast areas, a lower performance when compared to the supervised AdaBoost algorithm developed in their previous work (88%) [23,24]. Other strategies to improve neural networks’ performance in breast cancer classification refer to the extraction of features with wavelet transform [25], or to numerical simulations conducted on various breast tissue composition models [26].
The preferred learner for breast cancer detection studies is based on Support Vector Machines (SVM). Francis et al. studied the extraction of statistical and textural features in the curvelet domain, classifying 91% of instances correctly [27]. The use of Principle Component Analysis (PCA) in subsequent work to reduce the number of features, along with dynamic thermal image acquisition, led to lower values of ACC (83.3%) [28]. A similar approach, using only four clinically significant features, had previously achieved good performance metrics with a larger dataset (ACC = 88.10%, SN = 85.71%, SP = 90.48%) [29]. The Radial Basis Function (RBF) kernel appears to be the best option for the implementation of SVM in breast cancer research, as proven in [30,31,32,33], as well as the use of textural features retrieved from thermograms, all with high performance metrics. Gogoi et al. focused its work on the gathering of the best possible inputs that characterize healthy, benign and malignant cases through temperature and intensity analysis (ACC = 83.2%, SN = 85.5%, SP = 73.2%) [34]. SVM was the chosen learner after comparison with other AI algorithms [35]. Satish et al. tried a different strategy, using local energy features of wavelet sub-bands after temperature matrix normalization to classify normal and abnormal breast thermograms with ACC = 91%, SN = 87.23% and SP = 94.34% with a SVM-Gaussian [8].
Apart from neural networks and support vector machines, studies with equally high classification metrics can be found using AdaBoost [36], Bayesian classifiers [37] and k-NN [38,39] classifiers. With the goal of achieving the best possible outcome, some authors choose to test different AI learners, with SVM [40,41], k-NN [42,43], ANN [44,45] and Decision tree [46] algorithms being the best options among the tested ones.
The use of ensemble classifiers has been frequent in recent years. Most work of Krawczyk et al. is based on a pool of learners, which are as distinct as possible from each other, in order to reduce complexity and avoid redundant information during the classification stage [47,48,49]. A first approach used different feature inputs to train the different classifiers and address class imbalance, evolving later for the one-class classification strategy to increase learners’ sensitivity to minority classes [47,48]. Another way to surpass data shortage is suggested by the author through feature space clustering, assigning each cluster to the most competent classifier from the ensemble [50]. This last method gave the best results with ACC = 90.02%, SN = 82.55% and SP = 91.89%.
Few implementations of ML non-related to breast cancer diagnosis were found. Zadeh et al. used fuzzy active contours to segment suspected breast tumor areas (ACC = 91.89%) and Saednia et al. developed a supervised ML algorithm based on Random Forest to assess dermatitis caused by radiation therapy, reporting thermal markers indicative of radiation-induced skin toxicity with an ACC of 87% [51].

3.1.2. Diabetes Disease

The combination of IR imaging with AI algorithms has proven its usefulness in diabetic foot detection. Hernandez-Contreras et al. retrieved features representative of 3D morphological patterns and position of the foot, which achieved ACC values of 94.33% with an ANN classifier [52]. High classification rates (91%) were attained more recently with a learner, also based on neural networks, fed with inputs collected after the analysis of surface temperature distribution of the foot [6]. Adam et al. decomposed images of left, right and bilateral foot to calculate entropy and texture features. The best performance was achieved by a k-NN learner (ACC = 93.16%, SN = 90.31%; SP = 98.04%) [53]. Good classification metrics for k-nearest neighbor were also presented by Vardasca et al. The authors first used steady-state thermal images and built a computational tool for automatic image processing and classification with a success rate of 92.5% [54]. Later on, the use of dynamically acquired thermograms was tested by the same research group, achieving ACC = 81.25%, SP = 80% and SN = 100% with k-NN [55]. Great results were also recently achieved by Cruz-Vega et al. [56] for the detection of changes in the plantar temperature of the foot. The authors were the first to apply a deep learning structure for diabetic foot assessment, retrieving ACC and SN values of 85.3% and 91.67%, respectively. Thirunavukkarasu et al. presented the only study pertaining to diabetes disease that did not report on the evaluation of diabetic foot [57]. Instead, thermal data and artificial intelligence were used as a pre-screening tool for type II diabetes, through the analysis of tongue thermograms. The ANN classifier was the top learner with an ACC of 94.28%.

3.1.3. Skin Cancer

The encountered studies that focused on the use of ML with thermal data for skin cancer detection were conducted by the same group of authors. Magalhaes et al. first tried to distinguish benign from malignant lesions with static IR imaging, reaching a low ACC value of 60% with k-NN classifiers [58]. The overall results were slightly improved when dynamic thermal information was added to the feature input set [59]. Recently, the distinction of melanomas and nevi lesions was successfully performed, reaching ACC and SN values of 84.2% and 91.3% [9].

3.1.4. Other Conditions

The assessment of other pathological conditions is also reported on the literature, yet with less broadening. Papez et al. diagnosed carpal tunnel syndrome with artificial neural network classifiers and IR data with an ACC of 82.2% [60]. The authors identified relevant segments of the hand for improved classification, having tested this finding in a larger data set further along. Nonetheless, the classifier performance decreased, making the authors recommend their methodology only for the screening of severe cases [61]. Acharya et al. focused their research on dry eye disease, first for treatment monitoring (k-NN: ACC = 99.88%; SN = 99.7%; SP = 100%) and then for diagnosis (k-NN: ACC = 99.8%; SN = 99.8%; SP = 99.8%) [62,63]. The presence of rheumatoid arthritis disease and particular areas of interest for improved diagnosis were identified by Frize et al. (Decision Trees: SN = 96%; SP = 92%) [64], while Umapathy et al. used k-means methods to segment hands, aiding in the evaluation of this health problem [7]. Other applications of infrared thermography and ML algorithms include the evaluation of whiplash injury extent [65], back pain [66], thyroid tumors [67], breathing [68], drunkenness state [69], exercise-induced fatigue [70], hypertension [71], schizophrenia [72], burn wounds [73], facial recognition [74], cardiovascular disease [75], hemodynamic shock [76] and stress recognition [77], with the preferred classifiers being based on ANN [65,68,70,71,74,77], SVM [66,69,70,72], Decision Trees [67] and Naïve-Bayes [75].

3.2. Quantitative Synthesis

The meta-analysis process required the number of True Positives (TP), False Negatives (FN), True Negatives (TN) and False Positives (FP) for each study. Thus, studies that did not report this information, nor supplied data sufficient to retrieve it, were not included in the quantitative synthesis.
The software R was used with the “meta” package to perform univariate analysis and retrieve sensitivity (SN), specificity (SP) and log of diagnostic odds ratio (DOR) forest plots and the “mada” package to plot the summary receiver operating characteristic (SROC) curve of all studies [78]. The plots were constructed with the information retrieved from all studies included in the quantitative analyses process, independently of the studied pathology. Thus, a better visual assessment of the distribution of the different metrics among studies is possible.
The test for heterogeneity retrieved a high chi-square (χ2) value for the analysis of all records, suggesting high heterogeneity between studies (Appendix B). The high values of I2 encountered also support this idea [79]. Thus, a random effects model was used to summarize the effects and different analyses that were made, separating studies focused on breast cancer and diabetic foot. Still, the values of χ2 and I2 remained high for breast cancer studies, while being low for diabetic foot ones (Appendix B).
The values of sensitivity and specificity differed substantially among all studies with SN ranging in [0.36–1] (Figure 2) and SP in [0.10–1] (Figure 3). The estimator of the random effect model was 3.70 (95%CI, 3.10–4.21) (Figure 4.). The SROC curve presented some scattering, with only few studies inside the 95% confidence interval region. Still, most publications are located on the top left corner of the plot (Figure 5). For breast cancer, SN and SP were in the ranges of [0.36–1] and [0.49–1] with a DerSimonian and Laird (DSL) estimator of 3.74 (95%CI, 3.12–4.36). For diabetic foot, SN and SP were in the ranges of [0.89–1] and [0.79–0.98] with a DSL estimator of 5.23 (95%CI, 4.02–6.45). The separate SN, SP and log (DOR) estimates along with 95% confidence intervals (CIs) for breast cancer and diabetic foot studies are included in Appendix C and Appendix D.

4. Discussion

The literature search yielded 68 studies concerning the application of biomedical thermal data with ML strategies. More than half of these were focused on breast cancer diagnosis, yet its usage as a primary screening method is not recommended [80].
There is a clear preference for the use of ANN (19 studies), SVM (17 studies) and k-NN (9 studies) classifiers to achieve the best possible classification metrics, with only some authors choosing ensemble approaches (9) (Appendix A). The large number of successful implementation strategies already published, and the availability of pre-written algorithms based on these learners in development and computing environment tools such as WEKA, Spyder and Matlab, could be a justification for this choice. However, in some cases, this leads to a poor or inexistent description of classification parameters, training/testing conditions and/or number/type of input features selected, hindering the reproducibility of such studies. The inexistence of staple classification parameters is also clear and hampers a thorough meta-analysis.
When looking for breast cancer results, maximum performance was achieved by [39], while [53] showed the best approach to aid diabetic foot diagnosis. Still, additional work can be done to better the current methodologies.
For future work, most authors emphasize the need for improved feature selection strategies to guarantee the inclusion of significant features, while keeping the number of classification inputs as low as possible, thus reducing processing time [9,15,17,18,29,37,39,41,53,63,65]. To improve classification metrics, IR data could be complemented with information collected from other imaging modalities and/or biological tests [9,26,33,64,70]. The availability of a larger data sample is also mentioned by several studies across the different pathologies, in order to perform more complete testing and ease the implementation of such methodologies in daily practices [9,16,25,26,31,44,46,52,53,54,58,60,63,65,67]. Apart from the mentioned suggestions, the implementation of parameter optimization during the construction of the learner may yield better classification results, as well as the use of strategies to deal with potential class imbalance problems. In addition, the construction of user interfaces/dashboards designed to be utilized by health-care professionals would simplify the introduction of ML aiding tools in day-to-day clinical activities.
From the visual verification of the metastatistics results, it is possible to conclude that there is some heterogeneity between studies. The log DOR forest plot displays the highest statistical heterogeneity, followed by the sensitivity and specificity ones, with the latter showing a larger number of confidence intervals overlapping. This factor is also reflected in the SROC curve since slight scattering is visible. This variability may partially be caused by the comparison of studies that implement different learners for the classification of distinct pathologies. Nonetheless, there is a good distribution of studies on the top left corner of the SROC curve, showing a good balance between sensitivity measurements and false positive rate, with the majority of SN and SP values at the highest range of the scale (SN > 0.82, SP > 0.76). Thus, the interest of ML models for biomedical thermal applications is indicated. The low χ2 value attained for diabetic foot studies, with a high p-value, reveals that heterogeneity is insignificant (Appendix B). This fact, allied to the good sensitivity and specificity of the encountered estimators, leads to the conclusion that the use of biothermal data for diabetic foot assessment with artificial intelligence learners could be an interesting diagnostic aiding tool. Still, this inference should be taken with caution, due to the low number of studies included in the quantitative analysis. The similarity between the metastatistic results for all studies, and for breast cancer separately, favors the conclusion that thermographic data cannot be singularly used for breast tumor assessment. The heterogeneity visible in the SN, SP and log (DOR) forest plots, with the summary (DSL) being excluded by the error bars of some studies, is also an indication of this assumption. Nonetheless, clinical heterogeneity could also have an impact on the high statistical heterogeneity found, due to differences in population enrolled, disease severity and followed methodology.

5. Conclusions

IR thermal imaging is a valuable imaging modality to assess physiological changes with pathological causes, such as the ones encountered during neoplasm development and rheumatic diseases. The use of artificial intelligence models for thermogram interpretation has showed positive results, combining a fast analysis with efficient classification. Still, more work is needed to promote its usage on a daily basis in clinical scenarios. The search for more discriminant features and larger data sets are the most important topics to improve performance metrics. The use of IRT imaging with ML alone for breast cancer assessment should be avoided, and additional metastatic studies for diabetic foot diagnosis are of interest to strongly prove the usefulness of these tools. The combined use of IRT imaging and ML methods should also be further extended to other pathological conditions, such as skin cancer, in the hopes of easing detection and decreasing treatment costs.

Author Contributions

C.M. and R.V.: conceptualization; C.M. and R.V.: investigation; C.M. and R.V.: data organization and analysis; C.M.: Writing—Original draft; J.M. and R.V.: Writing—Review and editing; J.M. and R.V.: supervision; All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Project LAETA–UIDB/50022/2020, UIDP/50022/2020 and the PhD Scholoarship from Fundação para a Ciência e a Tecnologia–SFRH/BD/144906/2019.

Informed Consent Statement

Not applicable.

Data Availability Statement

It can be made available upon request to the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table
Table A1. Biomedical thermal applications and respective implemented base classifier, sample size, accuracy (ACC), sensitivity (SN) and specificity (SP), chronologically ordered from oldest to latest.
Table A1. Biomedical thermal applications and respective implemented base classifier, sample size, accuracy (ACC), sensitivity (SN) and specificity (SP), chronologically ordered from oldest to latest.
Year (Ref.)Biomedical ApplicationBest Overall ClassifierSample SizeACC (%)SN (%)SP (%)
1990 [65]Whiplash injuryANN209080100
2001 [15]Breast cancerANN8358.55467
2002 [16]Breast cancerANN200406267
2007 [18]Breast cancerANN + fuzzy rules78 (28 healthy, 43 benign, 7 cancer)-10060
2007 [21]Breast cancerFuzzy-rules146 thermograms98
2008 [17]Breast cancerANN90 81.288.2
2008 [60]Carpal tunnel syndromeANN56 patients (34 hands)82.5
2009 [22]Breast cancerFuzzy-rules146 (29 malignant, 117 benign)97.993.199.1
2009 [61]Carpal tunnel syndromeANN251 hands (132 healthy, 119 pathological)72.2
2012 [29]Breast cancerSVM50 (25 healthy, 25 malignant)88.185.790.4
2012 [40]Breast cancerSVM34 thermograms88.2
2012 [20]Breast cancerANN200 (15 abnormal cases)705075
2012 [19]Breast cancerANN200 (15 abnormal cases)97.59397
2012 [64]Rheumatoid arthritisDecision Trees52 (26 healthy, 26 ill) 9692
2013 [36]Breast cancerAdaBoost32 (9 malignant, 12 benign, 11 normal)95
2013 [30]Breast cancerSVM63 (29 healthy, 34 malignant)100
2013 [47]Breast cancerEnsemble classifier146 (29 malignant, 117 benign)88.781.395.5
2013 [48]Breast cancerEnsemble classifier146 (29 malignant, 117 benign)87.482.188.7
2013 [50]Breast cancerEnsemble classifier146 (29 malignant, 117 benign)90.082.591.8
2013 [37]Breast cancerNaive-Bayes98 (77 malignant, 21 healthy)71.88237
2014 [42]Breast cancerEuclidean Distance50 (14 malignant, 19 benign, 17 cysts)8485.786.5
2014 [27]Breast cancerSVM22 (11 normal, 11 abnormal)90.981.8100
2014 [28]Breast cancerSVM36 (24 normal, 12 malignant)83.3
2014 [49]Breast cancerEnsemble classifier146 (29 malignant, 117 benign)89.081.990.8
2014 [43]Breast cancerk-NN40 (26 normal, 14 abnormal)92.078.6100
2014 [62]Dry-eye diseasek-NN81 (40 responded and 41 not responded)99.899.7100
2015 [66]Back PainSVM1000 (300 healthy, 200 faulty posture and 500 lateral spinal curvature) 9088
2015 [31]Breast cancerSVM63 (29 healthy, 34 malignant)100
2015 [52]Diabetic FootANN60 (30 non-diabetic, 30 diabetes)94.397.391.3
2015 [38]Breast cancerk-NN18 (9 normal, 9 abnormal)99.4
2015 [41]Breast cancerSVM50 (37 normal, 13 abnormal)8876.991.9
2015 [25]Breast cancerANN306 thermograms of 102 individuals (123 unhealthy, 183 healthy)90.487.689.7
2015 [44]Breast cancerANN22 (11 normal, 11 cancer)90.981.8100
2015 [26]Breast cancerANN24092.8
2015 [63]Dry-eye diseasek-NN104 (83 ill, 21 healthy)99.899.899.8
2016 [23]Breast cancerAdaBoost6787.4
2016 [24]Breast cancerFuzzy C-Means6775
2016 [46]Breast cancerDecision Trees50 (25 malignant, 25 normal)9896.6100
2016 [32]Breast cancerSVM80 (40 normal, 40 abnormal)9087.592.5
2016 [39]Breast cancerk-NN80 (40 normal, 40 abnormal)100100100
2016 [81]Breast cancerFuzzy active contours6091.885
2016 [67]Thyroid tumourDecision Trees51 (30 abnormal, 21 normal)989599
2017 [68]Breath analysisANN1 subject (25 experiments)100
2017 [69]Drunkenness stateSVM41100
2017 [33]Breast cancerSVM240 (100 normal, 66 Benign, 78 Cancer)94.8
2017 [70]Exercise induced-fatigueEnsemble classifier1981.5
2017 [71]HypertensionANN28 (14 hypertensive, 14 normal)8985.792.9
2017 [7]Rheumatoid arthritisk-means60 (30 arthritis rheumatoid, 30 healthy)8386.679
2017 [72]SchizophreniaSVM35 (18 moderately ill, 17 markedly ill)94.3
2018 [73]Burn woundsRandom Forest3485.3
2018 [74]Facial recognitionANN3561 from 22 subjects (2124 positive, 1437 negative)85.5
2018 [53]Diabetic Footk-NN107 (51 healthy, 66 diabetic-33 neuropathic, 33 non-neuropathic)93.190.398.0
2018 [35]Breast cancerSVM407 (117 abnormal, 290 normal)989898
2018 [58]Skin neoplasmsk-NN85 (68 malignant, 17 benign)60
2018 [45]Breast cancerEnsemble classifier725 (219 cyst, 371 benign, 235 malignant)73.37888
2018 [82]Diabetic Footk-NN5692.5
2019 [75]Cardiovascular Disease (CVD)Naive-Bayes20 (10 non-CDV, 10 CVD)908090
2019 [76]Hemodynamic shockRF539 (253 continuous intra-arterial blood pressure)736582
2019 [77]Stress RecognitionANN93 sets of data from 17 (9 males, 8 females)78.3
2019 [34]Breast cancerSVM60 (25 healthy, 23 benign, 12 malignant)83.285.573.2
2019 [6]Diabetic FootANN246 (150 diabetics without complications, 36 with complications, 60 healthy)91
2019 [59]Skin neoplasmsSVM320 (51 SCC, 118 BCC, 16 melanomas, 29 AK, 30 nevi, 14 SK, 11 cysts, 29 other benign neoplasia and 22 scar tissue lesions)8491
2019 [9]Skin neoplasmsSVM46 (16 melanoma, 30 nevi) cooling84.291.311
2019 [51]Breast cancer treatment sequelsRF90 (75 training, 15 testing)87
2019 [8]Breast cancerSVM100 (47 abnormal, 53 normal)9187.294.3
2019 [55]Diabetic Foot Ulcer (DFU)k-NN39 (15 DFU ischemic or infected)81.280100
2020 [57]Type II Diabetes Mellitus (DM)ANN140 (70 normal, 70 type 2 DM)94.2
2020 [56]Diabetic FootANN11085.391.67

Appendix B

Table
Table A2. Test for heterogeneity with χ2, respective p-values and I2 for all studies combined, as well as for breast cancer and diabetic foot separately.
Table A2. Test for heterogeneity with χ2, respective p-values and I2 for all studies combined, as well as for breast cancer and diabetic foot separately.
χ2p-ValueI2 (%)
All studies185.559<0.00186.66
Breast cancer96.474<0.00182.81
Diabetic foot0.5200.7710.00

Appendix C

Applsci 11 00842 g0a1 550
Figure A1. Forest plots for sensitivity for breast cancer studies.
Figure A1. Forest plots for sensitivity for breast cancer studies.
Applsci 11 00842 g0a1
Applsci 11 00842 g0a2 550
Figure A2. Forest plots for specificity for breast cancer studies.
Figure A2. Forest plots for specificity for breast cancer studies.
Applsci 11 00842 g0a2
Applsci 11 00842 g0a3 550
Figure A3. Forest plots for log diagnostic odd ratios for breast cancer studies.
Figure A3. Forest plots for log diagnostic odd ratios for breast cancer studies.
Applsci 11 00842 g0a3

Appendix D

Applsci 11 00842 g0a4 550
Figure A4. Forest plots for sensitivity for diabetic foot studies.
Figure A4. Forest plots for sensitivity for diabetic foot studies.
Applsci 11 00842 g0a4
Applsci 11 00842 g0a5 550
Figure A5. Forest plots for specificity for diabetic foot studies.
Figure A5. Forest plots for specificity for diabetic foot studies.
Applsci 11 00842 g0a5
Applsci 11 00842 g0a6 550
Figure A6. Forest plots for log diagnostic odd ratios for diabetic foot studies.
Figure A6. Forest plots for log diagnostic odd ratios for diabetic foot studies.
Applsci 11 00842 g0a6

References

  1. Ring, F.; Ammer, K. Infrared thermal imaging in medicine. Physiol. Meas. 2012, 33, R33–R46. [Google Scholar] [CrossRef] [PubMed]
  2. Arens, E.; Zhang, H.A. The Skin’s Role in Human Thermoregulation and Comfort. In Thermal and Moisture Transport in Fibrous Materials; Pan, N., Gibson, P., Eds.; Woodhead Publishing Limited: Cambridge, UK, 2006; pp. 560–602. [Google Scholar]
  3. Vardasca, R.; Ramalhão, C.; Gabriel, J. Bases da Termografia: Enquadramento da Termografia Com Outras Técnicas de Imagem Médica. In Termografia: Imagem Médica e Síndromes Dolorosas, 1st ed.; Lidel: Lisbon, Portugal, 2016; pp. 3–10. [Google Scholar]
  4. Ring, F.; Ring, E.F.J.; Ammer, K. The technique of infrared imaging in medicine. Infrared Imaging 2014, 2015, 1–10. [Google Scholar] [CrossRef]
  5. Ammer, K. The Glamorgan Protocol for recording and evaluation of thermal images of the human body. Thermol. Int. 2008, 18, 125–129. [Google Scholar]
  6. Gururajarao, S.B.; Venkatappa, U.; Shivaram, J.M.; Sikkandar, M.Y.; Al Amoudi, A. Infrared Thermography and Soft Computing for Diabetic Foot Assessment; Elsevier BV: Amsterdam, The Netherlands, 2019; pp. 73–97. [Google Scholar]
  7. Umapathy, S.; Vasu, S.; Gupta, N. Computer Aided Diagnosis Based Hand Thermal Image Analysis: A Potential Tool for the Evaluation of Rheumatoid Arthritis. J. Med. Biol. Eng. 2017, 38, 666–677. [Google Scholar] [CrossRef]
  8. Sathish, D.; Kamath, S.; Prasad, K.; Kadavigere, R. Role of normalization of breast thermogram images and automatic classification of breast cancer. Vis. Comput. 2017, 35, 57–70. [Google Scholar] [CrossRef]
  9. Magalhaes, C.; Vardasca, R.; Rebelo, M.; Valenca-Filipe, R.; Ribeiro, M.; Mendes, J. Distinguishing melanocytic nevi from melanomas using static and dynamic infrared thermal imaging. J. Eur. Acad. Dermatol. Venereol. 2019, 33, 1700–1705. [Google Scholar] [CrossRef] [PubMed]
  10. Deo, R.C. Machine Learning in Medicine. Circulation 2015, 132, 1920–1930. [Google Scholar] [CrossRef] [Green Version]
  11. Vardasca, R.; Magalhaes, C.; Mendes, J. Biomedical Applications of Infrared Thermal Imaging: Current State of Machine Learning Classification. Proceedings 2019, 27, 46. [Google Scholar] [CrossRef] [Green Version]
  12. Catal, C. Performance Evaluation Metrics for Software Fault Prediction Studies. Acta Polytech. Hungarica 2012, 9, 193–206. [Google Scholar]
  13. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
  14. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med. 2009, 6, e1000100. [Google Scholar] [CrossRef] [PubMed]
  15. Ng, E.Y.K. Statistical analysis of healthy and malignant breast thermography. J. Med. Eng. Technol. 2001, 25, 253–263. [Google Scholar] [CrossRef] [PubMed]
  16. Ng, E.Y.K.; Fok, S.C.; Peh, Y.; Ng, F.; Sim, L. Computerized detection of breast cancer with artificial intelligence and thermograms. J. Med. Eng. Technol. 2002, 26, 152–157. [Google Scholar] [CrossRef] [PubMed]
  17. Ng, E.Y.K.; Kee, E.C. Advanced integrated technique in breast cancer thermography. J. Med. Eng. Technol. 2008, 32, 103–114. [Google Scholar] [CrossRef]
  18. Tan, T.; Quek, C.; Ng, G.; Ng, E. A novel cognitive interpretation of breast cancer thermography with complementary learning fuzzy neural memory structure. Expert Syst. Appl. 2007, 33, 652–666. [Google Scholar] [CrossRef]
  19. Zadeh, H.G.; Pakdelazar, O.; Haddadnia, J. Diagnosing Breast Cancer with the Aid of Fuzzy Logic Based on Data Mining of a Genetic Algorithm in Infrared Images. Middle East J. Cancer 2012, 3, 119–129. [Google Scholar]
  20. Zadeh, H.G.; Haddadnia, J.; Hashemian, M.; Hassanpour, K. Diagnosis of breast cancer using a combination of genetic algo-rithm and artificial neural network in medical infrared thermal imaging. Iran J Med. Phys. 2012, 9, 265–274. [Google Scholar] [CrossRef]
  21. Schaefer, G.; Nakashima, T.; Zavisek, M.; Yokota, Y.; Drastich, A.; Ishibuchi, H. Breast Cancer Classification Using Statistical Features and Fuzzy Classification of Thermograms. In Proceedings of the 2007 IEEE International Fuzzy Systems Conference, London, UK, 23–26 July 2007; pp. 1–5. [Google Scholar] [CrossRef]
  22. Schaefer, G.; Závišek, M.; Nakashima, T. Thermography based breast cancer analysis using statistical features and fuzzy classification. Pattern Recognit. 2009, 42, 1133–1137. [Google Scholar] [CrossRef] [Green Version]
  23. Lashkari, A.; Pak, F.; Firouzmand, M. Full Intelligent Cancer Classification of Thermal Breast Images to Assist Physician in Clinical Diagnostic Applications. J. Med. Signals Sensors 2016, 6, 12–24. [Google Scholar] [CrossRef]
  24. Lashkari, A.E.; Firouzmand, M. Early breast cancer detection in thermogram images using AdaBoost classifier and fuzzy C-Means clustering algorithm. Middle East J. Cancer 2016, 7, 113–124. [Google Scholar]
  25. Pramanik, S.; Bhattacharjee, D.; Nasipuri, M. Wavelet Based Thermogram Analysis for Breast Cancer Detection. In Proceedings of the 2015 International Symposium on Advanced Computing and Communication (ISACC), Silchar, India, 14–15 September 2015; Volume 2016, pp. 205–212. [Google Scholar]
  26. Wahab, A.A.; Salim, M.I.M.; Yunus, J.; Aziz, M.N.C. Tumor localization in breast thermography with various tissue compositions by using Artificial Neural Network. In Proceedings of the 2015 IEEE Student Conference on Research and Development (SCOReD), Kuala Lumpur, Malaysia, 13–14 December 2015; Volume 2015, pp. 484–488. [Google Scholar]
  27. Francis, S.V.; Sasikala, M.; Saranya, S. Detection of Breast Abnormality from Thermograms Using Curvelet Transform Based Feature Extraction. J. Med. Syst. 2014, 38, 1–9. [Google Scholar] [CrossRef] [PubMed]
  28. Francis, S.V.; Sasikala, M.; Bharathi, G.B.; Jaipurkar, S.D. Breast cancer detection in rotational thermography images using texture features. Infrared Phys. Technol. 2014, 67, 490–496. [Google Scholar] [CrossRef]
  29. Acharya, U.R.; Ng, E.Y.K.; Tan, J.-H.; Sree, S.V. Thermography Based Breast Cancer Detection Using Texture Features and Support Vector Machine. J. Med. Syst. 2012, 36, 1503–1510. [Google Scholar] [CrossRef] [PubMed]
  30. Gaber, T.; Ismail, G.; Anter, A.; Soliman, M.; Ali, M.; Semary, N.; Hassanien, A.E.; Snasel, V. Thermogram Breast Cancer Prediction Approach based on Neutrosophic Sets Ther-mogram Breast Cancer Prediction Approach based on Neutrosophic Sets and Fuzzy C-Means Algorithm. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Milano, Italy, 25–29 August 2015. [Google Scholar]
  31. Ali, M.; Sayed, G.I.; Gaber, T.; Hassanien, A.E.; Snášel, V.; Silva, L.F. Detection of Breast Abnormalities of Thermograms based on a New Segmentation Method. In Proceedings of the 2015 Federated Conference on Computer Science and Information Systems, Lodz, Poland, 13–16 September 2015; Volume 5, pp. 255–261. [Google Scholar]
  32. Sathisha, K.; Kamath, S.; Prasad, K.; Kadavigere, R.; Martis, R.J. Asymmetry analysis of breast thermograms using automated segmentation and texture features. Signal Image Video Process. 2017, 11, 745–752. [Google Scholar] [CrossRef]
  33. Araujo, A.D.S.; Conci, A.; Resmini, R.; Montenegro, A.; Araujo, C.; Lebon, F. Computer Aided Diagnosis for Breast Diseases Based on Infrared Images. In Proceedings of the 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, Tunisia, 30 October–3 November 2017; pp. 172–177. [Google Scholar]
  34. Gogoi, U.R.; Majumdar, G.; Bhowmik, M.K.; Ghosh, A.K. Evaluating the efficiency of infrared breast thermography for early breast cancer risk prediction in asymptomatic population. Infrared Phys. Technol. 2019, 99, 201–211. [Google Scholar] [CrossRef]
  35. Gogoi, U.R.; Bhowmik, M.K.; Bhattacharjee, D.; Ghosh, A.K. Singular value based characterization and analysis of thermal patches for early breast abnormality detection. Australas. Phys. Eng. Sci. Med. 2018, 41, 861–879. [Google Scholar] [CrossRef]
  36. EtehadTavakol, M.; Chandran, V.; Ng, E.; Kafieh, R. Breast cancer detection from thermal images using bispectral invariant features. Int. J. Therm. Sci. 2013, 69, 21–36. [Google Scholar] [CrossRef]
  37. Nicandro, C.-R.; Efrén, M.-M.; Yaneli, A.-A.M.; Enrique, M.-D.-C.-M.; Gabriel, A.-M.H.; Nancy, P.-C.; Alejandro, G.-H.; De Jesús, H.-R.G.; Erandi, B.-M.R. Evaluation of the Diagnostic Power of Thermography in Breast Cancer Using Bayesian Network Classifiers. Comput. Math. Methods Med. 2013, 2013, 1–10. [Google Scholar] [CrossRef] [Green Version]
  38. Mejia, T.M.; Pérez, M.G.; Andaluz, V.H.; Conci, A. Automatic Segmentation and Analysis of Thermograms Using Texture Descriptors for Breast Cancer Detection. In Proceedings of the 2015 Asia-Pacific Conference on Computer Aided System Engineering, Quito, Ecuador, 14–16 July 2015; pp. 24–29. [Google Scholar]
  39. Silva, L.F.; Santos, A.A.S.; Bravo, R.S.; Silva, A.C.; Muchaluat-Saade, D.C.; Conci, A. Hybrid analysis for indicating patients with breast cancer using temperature time series. Comput. Methods Programs Biomed. 2016, 130, 142–153. [Google Scholar] [CrossRef]
  40. Resmini, R.; Borchartt, T.B.; Conci, A.; Lima, R.C.F. Auxílio ao Diagnóstico Precoce de Patologias da Mama Usando Imagens Térmicas Câncer de mama. Anais Comput. Beach 2013, 1, 305–314. [Google Scholar]
  41. Milosevic, M.; Jankovic, D.; Peulic, A. Comparative analysis of breast cancer detection in mammograms and thermograms. Biomed. Tech. Eng. 2014, 60, 49–56. [Google Scholar] [CrossRef] [PubMed]
  42. Araújo, M.C.; Lima, R.C.; De Souza, R.M. Interval symbolic feature extraction for thermography breast cancer detection. Expert Syst. Appl. 2014, 41, 6728–6737. [Google Scholar] [CrossRef]
  43. Milosevic, M.; Jankovic, D.; Peulic, A. Thermography based breast cancer detection using texture features and minimum variance quantization. EXCLI J. 2014, 13, 1204–1215. [Google Scholar] [PubMed]
  44. Silva, L.F.; Sequeiros, G.O.; Santos, M.L.O.; Fontes, C.A.P.; Muchaluat-Saade, D.C.; Conci, A. Thermal Signal Analysis for Breast Cancer Risk Verification. Stud. Health Technol. Inform. 2015, 216, 746–750. [Google Scholar] [PubMed]
  45. De Santana, M.A.; Pereira, J.M.S.; Da Silva, F.L.; De Lima, N.M.; De Sousa, F.N.; De Arruda, G.M.S.; Lima, R.D.C.F.D.; Da Silva, W.W.A.; Dos Santos, W.P. Breast cancer diagnosis based on mammary thermography and extreme learning machines. Res. Biomed. Eng. 2018, 34, 45–53. [Google Scholar] [CrossRef] [Green Version]
  46. Raghavendra, U.; Acharya, U.R.; Ng, E.Y.K.; Tan, J.-H.; Gudigar, A. An integrated index for breast cancer identification using histogram of oriented gradient and kernel locality preserving projection features extracted from thermograms. Quant. Infrared Thermogr. J. 2016, 13, 195–209. [Google Scholar] [CrossRef]
  47. Krawczyk, B.; Schaefer, G. A pruned ensemble classifier for effective breast thermogram analysis. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 7120–7123. [Google Scholar]
  48. Krawczyk, B.; Schaefer, G.; Woźniak, M. Combining one-class classifiers for imbalanced classification of breast thermogram features. In Proceedings of the 2013 Fourth International Workshop on Computational Intelligence in Medical Imaging (CIMI), Osaka, Japan, 3–7 July 2013; Volume 2013, pp. 36–41. [Google Scholar]
  49. Krawczyk, B.; Schaefer, G. A hybrid classifier committee for analysing asymmetry features in breast thermograms. Appl. Soft Comput. 2014, 20, 112–118. [Google Scholar] [CrossRef]
  50. Krawczyk, B.; Schaefer, G.; Zhu, S.Y. Breast Cancer Identification Based on Thermal Analysis and a Clustering and Selection Classification Ensemble. In Lecture Notes in Computer Science; Springer: Berlin, Germany, 2013; Volume 8211, pp. 256–265. [Google Scholar]
  51. Saednia, K.; Tabbarah, S.; Lagree, A.; Wu, T.; Klein, J.; Garcia, E.; Hall, M.; Chow, E.; Rakovitch, E.; Childs, C.; et al. Quantitative Thermal Imaging Biomarkers to Detect Acute Skin Toxicity From Breast Radiation Therapy Using Supervised Machine Learning. Int. J. Radiat. Oncol. 2020, 106, 1071–1083. [Google Scholar] [CrossRef]
  52. Hernandez-Contreras, D.; Peregrina-Barreto, H.; Rangel-Magdaleno, J.; Ramirez-Cortes, J.; Renero-Carrillo, F. Automatic classification of thermal patterns in diabetic foot based on morphological pattern spectrum. Infrared Phys. Technol. 2015, 73, 149–157. [Google Scholar] [CrossRef]
  53. Adam, M.; Ng, E.Y.; Oh, S.L.; Heng, M.L.; Hagiwara, Y.; Tan, J.-H.; Tong, J.W.; Acharya, U.R. Automated detection of diabetic foot with and without neuropathy using double density-dual tree-complex wavelet transform on foot thermograms. Infrared Phys. Technol. 2018, 92, 270–279. [Google Scholar] [CrossRef]
  54. Vardasca, R.; Vaz, L.; Magalhaes, C.; Seixas, A.; Mendes, J. Towards the Diabetic Foot Ulcers Classification with Infrared Thermal Images. In Proceedings of the 14th Quantitative Infrared Thermography Conference, Berlin, Germany, 25–29 June 2018. [Google Scholar] [CrossRef]
  55. Vardasca, R.; Magalhaes, C.; Seixas, A.; Carvalho, R.; Mendes, J. Diabetic foot monitoring using dynamic thermography and AI classifiers. In Proceedings of the 3rd Quantitative InfraRed Thermography Asia Conference (QIRT Asia 2019), Tokyo, Japan, 1–5 July 2019. [Google Scholar] [CrossRef]
  56. Cruz-Vega, I.; Hernandez-Contreras, D.; Peregrina-Barreto, H.; Rangel-Magdaleno, J.D.J.; Ramirez-Cortes, J.M. Deep Learning Classification for Diabetic Foot Thermograms. Sensors 2020, 20, 1762. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  57. Thirunavukkarasu, U.; Umapathy, S.; Krishnan, P.T.; Janardanan, K. Human Tongue Thermography Could Be a Prognostic Tool for Prescreening the Type II Diabetes Mellitus. Evid. Based Complement. Altern. Med. 2020, 2020, 1–16. [Google Scholar] [CrossRef] [PubMed]
  58. Magalhaes, C.; Vardasca, R.; Mendes, J. Classifying Skin Neoplasms with Infrared Thermal Images. In Proceedings of the 14th Quantitative InfraRed Thermography Conference (QIRT 2018), Berlin, Germany, 25–29 June 2018. [Google Scholar] [CrossRef]
  59. Magalhaes, C.; Mendes, J.; Filipe, R.V.; Vardasca, R. Skin neoplasms dynamic thermal assessment. In Proceedings of the 2019 IEEE 6th Portuguese Meeting on Bioengineering (ENBENG), Lisbon, Portugal, 22–23 February 2019; Volume 2019, pp. 1–4. [Google Scholar]
  60. Papež, B.J.; Palfy, M.; Turk, Z. Infrared Thermography Based on Artificial Intelligence for Carpal Tunnel Syndrome Diagnosis. J. Int. Med. Res. 2008, 36, 1363–1370. [Google Scholar] [CrossRef] [PubMed]
  61. Papež, B.J.; Palfy, M.; Mertik, M.; Turk, Z. Infrared Thermography Based on Artificial Intelligence as a Screening Method for Carpal Tunnel Syndrome Diagnosis. J. Int. Med. Res. 2009, 37, 779–790. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  62. Acharya, U.R.; Tan, J.-H.; Vidya, S.; Yeo, S.; Too, C.L.; Lim, W.J.E.; Chua, K.C.; Tong, L. Diagnosis of response and non-response to dry eye treatment using infrared thermography images. Infrared Phys. Technol. 2014, 67, 497–503. [Google Scholar] [CrossRef]
  63. Acharya, U.R.; Tan, J.H.; Koh, J.E.; Vidya, K.S.; Yeo, S.; Too, C.L.; Chua, C.K.; Ng, E.; Tong, L. Automated diagnosis of dry eye using infrared thermography images. Infrared Phys. Technol. 2015, 71, 263–271. [Google Scholar] [CrossRef]
  64. Frize, M.; Ogungbemile, A. Estimating rheumatoid arthritis activity with infrared image analysis. Stud. Health Technol. Inform. 2012, 180, 594–598. [Google Scholar] [PubMed]
  65. Egbert, D.; Kaburlasos, V.; Goodman, P. Neural Network Discrimination of Sublte Image Patterns. In Proceedings of the 1990 IJCNN International Joint Conference on Neural Networks, San Diego, CA, USA, 17–21 June 1990; pp. 517–524. [Google Scholar]
  66. Koprowski, R. Automatic analysis of the trunk thermal images from healthy subjects and patients with faulty posture. Comput. Biol. Med. 2015, 62, 110–118. [Google Scholar] [CrossRef]
  67. Gopinath, M.P.; Prabu, S. Classification of thyroid Abnormalities on thermal image: A study and approach. IIOAB J. 2016, 7, 41–57. [Google Scholar]
  68. Procházka, A.; Charvátová, H.; Vyšata, O.; Kopal, J.; Chambers, J.A. Breathing Analysis Using Thermal and Depth Imaging Camera Video Records. Sensors 2017, 17, 1408. [Google Scholar] [CrossRef]
  69. Koukiou, G.; Anastassopoulos, V. Fusion of Dissimilar Features from Thermal Imaging for Improving Drunk Person Identification. Int. J. Signal Process. Syst. 2017, 5, 106–111. [Google Scholar] [CrossRef]
  70. Lopez, M.B.; Del-Blanco, C.R.; Garcia, N. Detecting Exercise-Induced Fatigue Using Thermal Imaging and Deep Learning. In Proceedings of the 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA), Montreal, QC, Canada, 28 November–1 December 2017; Volume 2017, pp. 1–6. [Google Scholar]
  71. Thiruvengadam, J.; Mariamichael, A. A preliminary study for the assessment of hypertension using static and dynamic IR thermograms. Biomed. Tech. Eng. 2017, 63, 197–206. [Google Scholar] [CrossRef] [PubMed]
  72. Jian, B.-L.; Chen, C.-L.; Chu, W.-L.; Huang, M.-W. The facial expression of schizophrenic patients applied with infrared thermal facial image sequence. BMC Psychiatry 2017, 17, 1–7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  73. Martínez-Jiménez, M.A.; Ramirez-GarciaLuna, J.L.; Kolosovas-Machuca, E.S.; Drager, J.; González, F.J. Development and validation of an algorithm to predict the treatment modality of burn wounds using thermographic scans: Prospective cohort study. PLoS ONE 2018, 13, e0206477. [Google Scholar] [CrossRef]
  74. Wang, S.; Pan, B.; Chen, H.; Ji, Q. Thermal Augmented Expression Recognition. IEEE Trans. Cybern. 2018, 48, 2203–2214. [Google Scholar] [CrossRef]
  75. Jayanthi, T.; Anburajan, M. Model-based computer-aided method for diagnosis of cardiovascular disease using IR thermogram. Biomed. Res. 2019, 30, 95–101. [Google Scholar] [CrossRef] [Green Version]
  76. Nagori, A.; Dhingra, L.S.; Bhatnagar, A.; Lodha, R.; Sethi, T. Predicting Hemodynamic Shock from Thermal Images using Machine Learning. Sci. Rep. 2019, 9, 1–9. [Google Scholar] [CrossRef] [Green Version]
  77. Cho, Y.; Julier, S.J.; Bianchi-Berthouze, N. Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging. JMIR Ment. Health 2019, 6, e10140. [Google Scholar] [CrossRef] [Green Version]
  78. Shim, S.; Kim, S.-J.; Lee, J. Diagnostic test accuracy: Application and practice using R software. Epidemiol. Health 2019, 41, e2019007. [Google Scholar] [CrossRef]
  79. Higgins, J.P.T.; Thompson, S.G.; Deeks, J.J.; Altman, D.G. Measuring inconsistency in meta-analyses. BMJ 2003, 327, 557–560. [Google Scholar] [CrossRef] [Green Version]
  80. Gourd, E. Breast thermography alone no substitute for mammography. Lancet Oncol. 2017, 18, e713. [Google Scholar] [CrossRef]
  81. Zadeh, H.G.; Haddadnia, J.; Seryasat, O.R.; Isfahani, S.M.M. Segmenting breast cancerous regions in thermal images using fuzzy active contours. EXCLI J. 2016, 15, 532–550. [Google Scholar]
  82. Silva, P.; Vardasca, R.; Mendes, J.; Restivo, M.T. Towards an Automated Analysis of Forearm Thermal Images During Handgrip Exercise. In International Conference on Remote Engineering and Virtual Instrumentation; Springer: Cham, Germany, 2018; Volume 2, pp. 498–506. [Google Scholar]
Applsci 11 00842 g001 550
Figure 1. Flow of information through the different phases of the systematic review [13].
Figure 1. Flow of information through the different phases of the systematic review [13].
Applsci 11 00842 g001
Applsci 11 00842 g002 550
Figure 2. Forest plot for sensitivity: all studies combined.
Figure 2. Forest plot for sensitivity: all studies combined.
Applsci 11 00842 g002
Applsci 11 00842 g003 550
Figure 3. Forest plot for specificity: all studies combined.
Figure 3. Forest plot for specificity: all studies combined.
Applsci 11 00842 g003
Applsci 11 00842 g004 550
Figure 4. Forest plot for log diagnostic odd ratios: all studies combined.
Figure 4. Forest plot for log diagnostic odd ratios: all studies combined.
Applsci 11 00842 g004
Applsci 11 00842 g005 550
Figure 5. Summary receiver operating characteristic (SROC) curve for diagnostic test accuracy and the respective 95% confidence interval region (95% CI region).
Figure 5. Summary receiver operating characteristic (SROC) curve for diagnostic test accuracy and the respective 95% confidence interval region (95% CI region).
Applsci 11 00842 g005
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Magalhaes, C.; Mendes, J.; Vardasca, R. Meta-Analysis and Systematic Review of the Application of Machine Learning Classifiers in Biomedical Applications of Infrared Thermography. Appl. Sci. 2021, 11, 842. https://doi.org/10.3390/app11020842

AMA Style

Magalhaes C, Mendes J, Vardasca R. Meta-Analysis and Systematic Review of the Application of Machine Learning Classifiers in Biomedical Applications of Infrared Thermography. Applied Sciences. 2021; 11(2):842. https://doi.org/10.3390/app11020842

Chicago/Turabian Style

Magalhaes, Carolina, Joaquim Mendes, and Ricardo Vardasca. 2021. "Meta-Analysis and Systematic Review of the Application of Machine Learning Classifiers in Biomedical Applications of Infrared Thermography" Applied Sciences 11, no. 2: 842. https://doi.org/10.3390/app11020842

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop