Next Article in Journal
In Silico Protein Structure Analysis for SARS-CoV-2 Vaccines Using Deep Learning
Next Article in Special Issue
Artificial Intelligence in Bladder Cancer Diagnosis: Current Applications and Future Perspectives
Previous Article in Journal
Ultrasound Elastography: Basic Principles and Examples of Clinical Applications with Artificial Intelligence—A Review
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
BioMedInformatics
Volume 3
Issue 1
10.3390/biomedinformatics3010003
biomedinformatics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Reliability of Artificial Intelligence-Assisted Cephalometric Analysis. A Pilot Study

by
Anna Alessandri-Bonetti
1,2,
Linda Sangalli
3,*,
Martina Salerno
1 and
Patrizia Gallenzi
1
1
Department of Dental Clinic and Maxillofacial Surgery, A. Gemelli, University Policlinic IRCCS, Catholic University of Sacred Heart, 00168 Rome, Italy
2
Department of Oral Health Science, College of Dentistry, University of Kentucky, Lexington, KY 40506, USA
3
College of Dental Medicine—Illinois, Midwestern University, Downers Grove, IL 60515, USA
*
Author to whom correspondence should be addressed.
BioMedInformatics 2023, 3(1), 44-53; https://doi.org/10.3390/biomedinformatics3010003
Submission received: 14 December 2022 / Revised: 4 January 2023 / Accepted: 9 January 2023 / Published: 10 January 2023
(This article belongs to the Special Issue Computational Biology and Artificial Intelligence in Medicine)
Browse Figures
Versions Notes

Abstract

:
Recently, Artificial Intelligence (AI) has spread in orthodontics, in particular within cephalometric analysis, where computerized digital software is able to provide linear-angular measurements upon manual landmark identification. A step forward is constituted by fully automated AI-assisted cephalometric analysis, where the landmarks are automatically detected by software. The aim of the study was to compare the reliability of a fully automated AI-assisted cephalometric analysis with the one obtained by a computerized digital software upon manual landmark identification. Fully automated AI-assisted cephalometric analysis of 13 lateral cephalograms were retrospectively compared to the cephalometric analysis performed twice by a blinded operator with a computerized software. Intra- and inter-operator (fully automated AI-assisted vs. computerized software with manual landmark identification) reliability in cephalometric parameters (maxillary convexity, facial conicity, facial axis angle, posterior and lower facial height) was tested with the Dahlberg equation and Bland–Altman plot. The results revealed no significant difference in intra- and inter-operator measurements. Although not significant, higher errors were observed within intra-operator measurements of posterior facial height and inter-operator measurements of facial axis angle. In conclusion, despite the small sample, the cephalometric measurements of a fully automated AI-assisted cephalometric software were reliable and accurate. Nevertheless, digital technological advances cannot substitute the critical role of the orthodontist toward a correct diagnosis.

1. Introduction

In the past decades, the dental field has undergone a significant revolution thanks to the development of technological advances, which ultimately constitutes part of Artificial Intelligence (AI). AI is as a branch of computer science that uses technology to accomplish specific tasks, by mimicking human intelligence but without requiring human intervention and supervision [1,2,3,4,5,6,7]. It is based on data acquisition through the use of machine learning, deep learning applications, neural networks, and computational algorithms to aid in diagnostic process and prediction of prognoses, among others [3,4,7]. According to a recent estimate, the use of AI in the medical field might increase by an annual growth rate of 40%, from 1.30 to 10 billion by the year 2024 [8].
In the dental field, orthodontics is not exempt from this technological progress, where AI has been used for image analysis, prediction making, digital record keeping, and dental research, among others [8,9]. One orthodontic field of interest where AI has been widely implemented is the cephalometric analysis, considered a first important step for a correct diagnosis and treatment plan. A cephalometric radiograph assessment requires the exact identification of radiographic landmarks and the measurement of linear and angular distances. Traditionally, it has been performed manually. However, manual tracing is time-consuming, its accuracy is dependent on the experience of the operator, and has an inherent risk of subjective errors during landmark identification, measurements, and copying data to the server [9,10,11,12]. In addition, film deterioration has been identified as a reason of loss of information in craniofacial biology [11,13].
To minimize all these shortcomings, AI-assisted cephalometric analyses have been introduced and have gradually substituted the traditional manual tracing [14]. Generally, digital cephalometric analysis requires the operator to locate anatomic landmarks and provides the measurement of linear and angular values through the use of dedicated software. Thus far, several studies in the literature have assessed the validity and accuracy of digital cephalometry compared to manual measurements [15,16,17,18], thus demonstrating the superiority of the digital technique in reducing subjective errors and minimizing the time of the procedure.
A step forward in AI was the development of fully automated AI-assisted software adopted by specialized radiology centers. This software not only provides linear and angular measurements, but is also able to automatically locate landmarks on the cephalogram. As inaccurate identification of anatomic landmarks is currently the most common error of manual tracing, thus potentially leading to incorrect orthodontic treatment plan decisions [19], the possibility of relying on completely automatic digital cephalometric analysis would be preferrable, if these systems were found to be reliable and accurate.
As new fully automated AI-assisted cephalometric softwares are consistently spreading on the market, the accuracy of these technological advances needs to be tested to assist during the diagnosis and treatment plan. Therefore, the aim of this study was to compare the reliability and the accuracy of the cephalometric analysis provided by a dental specialized radiology lab through a fully automated AI-assisted cephalometric software (Novarad®, Novarad Medical Imaging Software, Provo, UT, USA) with the one obtained from a computerized cephalometric analysis program upon manual landmark identification. The latter was chosen as control because the literature has already proven the superiority of digital cephalometric analysis over those manually performed [18]. Although the ultimate diagnosis is a responsibility of the clinician, if fully automated AI-assisted cephalometric analyses were found to be reliable and valid, it could save time to the provider, expedite orthodontic diagnostic process, and reduce subjective errors.

2. Materials and Methods

This retrospective study was approved by the Institutional Review Committee of the university where it was conducted. As this constituted a pilot study, no sample size calculation was performed. A total of 13 lateral cephalograms were randomly selected from the archive of the Department of Pediatric Dentistry and Orthodontics at the Fondazione Policlinico Universitario “Agostino Gemelli” (Rome, Italy) and deidentified. On these, a total of 25 anatomic and cephalometric landmarks and 5 linear and angular cephalometric parameters were identified, in accordance with a similar study [20]. To be included, the cephalograms had to (1) present with a good quality, without any artifacts that may impede with the anatomical landmark localization; (2) have a calibration ruler for magnification assessment; (3) have a previously fully automated AI-assisted digital cephalometric analysis conducted from a specialized radiology lab available for review. No differentiation was made for sex, type of malocclusion, nor skeletal pattern.

2.1. Tracing Technique

The digital cephalometric analysis of the selected radiographs, previously performed by a specialized radiology lab according to Ricketts, were retrieved and kept blinded to the operator.
On those same selected cephalograms, an independent experienced orthodontist performed a cephalometric analysis according to Ricketts, using the digital cephalometric software Openceph (Openceph, v4.1.0, developed by Dr Bruno Oliva) [21]. Only one cephalometric analysis was selected for the current study, in order to reduce the potential confounding factor of variability and reproducibility of different landmark identification [19]. A maximum of five cephalometric tracings were performed per day, to minimize errors due to operator fatigue [18]. Each cephalometric trace was performed twice by the same blind operator, two weeks apart from each other and in random order. This was done to test the intra-operator accuracy in locating the landmarks: if a landmark is easily identified, the resulting cephalometric parameter should be more accurate and with few chances of error. The mean of the two measurements was then compared with the cephalometric analysis performed by fully automated AI-assisted software of the radiology lab. Then, the measurements were entered into the same Microsoft Office Excel spreadsheet and used for the analysis.

2.2. Cephalometric Measurements

A total of 25 anatomic and cephalometric landmarks were identified on each cephalogram according to Rickett’s cephalometric analysis, namely, Menton (Me), Nasion (N), Orbitale (Or), Basion (Ba), Pogonion (Pg), Condylion (Co), A point (A), B point (B), Pterygoid point (Pt), Gnathion (Gn), Xi point (Xi), Porion (P), Sella (S), protruberance menti (PM), condyle center point (DC), center of cranium point (CC), center of face point (CF), anterior nasal spine (ANS), posterior nasal spine (PNS), incisal edge of the upper central incisor (U1), incisal edge of the lower central incisor (L1), distal point of the upper first molar, distal point of the lower first molar, upper incisor root tip, and lower incisor root tip [22].
For the present study, the following five linear and angular cephalometric parameters were evaluated:
−
Maxillary Convexity: determined by the distance of A point to the facial plane N-Pg. The normal value is 2 ± 2 mm. This reflects the sagittal protrusion of the maxillary part of the face compared to the facial profile. A reduced angle indicates a maxillary retrusion within a normal facial plane, a mandibular protrusion with normal or retruded maxillary projection, or a brachycephalic facial profile. An increased angle identifies a maxillary protrusion with a mandible within normal limit, a mandibular retrusion with a maxilla within normal limit, a maxillary protrusion with retrognathia, or a dolichocephalic facial profile [22,23].
−
Angle of Facial Conicity: the angle formed by facial plane N-Pg with the mandibular plane Go-Gn. The normal value is 68 ± 4°. This reflects the sagittal and vertical position of the chin as well as the direction of the facial growth. A reduced angle suggests a clockwise mandibular growth and a dolichocephalic facial type; an increased angle indicates the tendency for a counterclockwise mandibular growth and a brachycephalic facial type [22].
−
Facial Axis Angle: identifies the posterior angle constituted by the intersection of the extension of facial axis Pt-Gn with the basal plane Ba-N. The normal value is 90 ± 3°. This angle reflects the vertical mandibular growth: a reduced angle indicates a dolichocephalic growth or retrognathic profile; an increased angle indicates a brachycephalic growth [22].
−
Posterior Facial Height: identified by the linear measurement that connects S-Go [22].
−
Lower Facial Height: the angle formed by mandibular axis Xi-PM and the line that connects the mandibular centroid and SNA (Xi-SNA). The normal value is 47 ± 4°. It identifies the position and the direction of the mandibular growth and the spatial location of the maxilla. A reduced angle suggests a horizontal growth, a downward inclination of the maxilla, or a counterclockwise rotation of the mandible. An increased angle indicates a vertical growth, an upward inclination of the maxilla, or a clockwise rotation of the mandible [22].

2.3. Statistical Analysis

A Shapiro–Wilk test was used to assess the normality of the data. In order to assess the intra-operator reproducibility of cephalometric parameters, the Dahlberg equation was utilized as follows:
S = √∑d2/2n
where S is the total error of each parameter, ∑d2 is the summation of the difference between the first (baseline) and the second measurement (at 2 weeks), and n is the total number of the radiographs. The Dahlberg equation was utilized because it provides a measurement of reproducibility of cephalometric parameters. Smaller errors identified more accurate cephalometric measurements.
For the analysis, the cephalometric analysis conducted by the radiologic lab (fully automated AI-assisted) was labeled as x1, and the cephalometric analysis performed by the digital orthodontic lab Orthoceph (considered as reference) was labeled as x2. In order to assess intra- and inter-operator reliability, a Bland–Altman analysis was utilized for each parameter, by calculating the mean of the difference (x1 − x2), the standard deviation (SD) of the difference, the superior limit x 1 − x 2 ¯ + 1.96 · D S , and the inferior limit x 1 − x 2 ¯ − 1.96 · D S of the 95% confidence interval (CI) of the difference. Bland–Altman graphs were created to display the difference between x1 and x2 for each parameter. The cephalometric analysis performed by fully automated AI-assisted cephalometric analysis (x1) was considered reliable in case the value of each parameter was included within the inferior and the superior limit of the CI of x2.
Statistical analyses were performed with IBM SPSS Statistics 23 (SPSS, Chicago, IL, USA).

3. Results

Measurements were performed on a total of 13 lateral cephalograms (9 females and 4 males, mean age 20 ± 17.5 years old, range 6.3–65.9).

3.1. Intra-Operator Measurements

Intra-operator measurements of each parameter as assessed at different time-points are presented in Table 1. The Bland–Altman analysis revealed that the value of each measurement performed by the same operator at 2 weeks was included within the CI of the measurement performed at baseline, with no statistically significant difference. The linear trend displayed by the Bland–Altman plots in Figure 1 are concordant with the results. A higher error was observed within intra-operator measurements of Posterior Facial Height (0.807, 95% CI −2.22, 2.42).

3.2. Inter-Operator Measurements

Inter-operator measurements of each parameter, as assessed by the two different types of cephalometric software, are presented in Table 2 and Figure 2. No statistically significant difference was found between the measurements conducted by a fully automated AI-assisted cephalometric analysis and those performed by a computerized digital software after manual landmark location. Although not significant, a higher error was observed within inter-operator measurements of the Facial Axis Angle (1.854, 95% CI −4.06, 5.93).

4. Discussion

An accurate cephalometric analysis is essential to make a correct diagnosis, establish the proper treatment plan, and monitor the stability of the orthodontic therapy. Due to the recent progress of AI in orthodontics and the important advantages that digital technologies could bring to the clinician, the aim of the study was to assess the reliability and accuracy of a fully automated AI-assisted cephalometric analysis compared to the one obtained by a computerized digital cephalometric software after manual landmark identification. The results of the present study suggest that the measurements obtained by the fully automated AI-assisted cephalometric analysis are reliable and do not differ from a well-established digitally conducted cephalometric tracing.
Performing an accurate cephalometric analysis is essential to obtain correct therapeutical indications; conversely, potential errors could result in incorrect treatment plans, with negative consequences on the patient’s outcomes. For example, the value of the incisor mandibular plane angle [24] has historically been weighed to determine if an orthodontic case required extraction. Several authors, such as Tweed [25] and Heiser [26], advocated an IMPA of 90° as cut-off for extractions, as this value was considered crucial to avoid gingival recession [27]. Nevertheless, despite the importance of precise cephalometric values, the ultimate treatment plan should carefully balance an equilibrium between hard and soft tissues towards a facial balance, as advocated first by Proffit [28,29].
The use of AI within dental field has become more and more popular to help achieve a correct diagnosis and proper treatment plan, especially in orthodontics. In the past decades, the orthodontic market has seen the spread of several types of digital cephalometric software and apps. Within the cephalometric analysis, AI technologies have been utilized in performing linear and angular measurements as well as, more recently, in identifying anatomic landmarks [30,31]. These are considered to be fully automated AI-assisted software (e.g., CephX®, CEFBOT, and WebCephTM, among other). Generally, an orthodontist that utilizes these technological tools in routine clinical practice relies on the values ultimately provided by the platform; therefore, assessing the accuracy of this software is fundamental. This is of critical importance considering that studies in the literature are discordant on the reliability of fully automated AI-assisted software, with some supporting good reliability [18,30,31,32,33,34] and some others revealing significant discrepancies when compared to manual landmarking methods [35,36]. Conversely, when compared to other digital computer software, the majority of the studies suggested a good accuracy of fully automated AI-assisted software [37,38]. However, some differences were found in specific parameters; therefore, supervision and a possible correction from the operator was necessary [36].
The results of the present study demonstrated a good reliability of the measurements of all selected parameters performed by a fully automated AI-assisted cephalometric analysis. Values that showed higher errors, i.e., 0.807 and 1.854, involved the measurement of Posterior Facial Height and Facial Axis Angle, respectively. The reason of this slight discrepancy in the intra- and inter-operator measurements can be attributed to the fact that some of the anatomic landmarks lay on blurred area of the craniofacial structures (e.g., Ba and Pt) [39,40], resulting in higher identification difficulties compared to others that lay on clear borders [35]. Moreover, the literature revealed inconsistent results in the exact location of certain landmarks, such as Gn and Go [41], which were used in the present investigation to measure the Posterior Facial Height and the Facial Axis Angle. Finally, a discrepancy in the measurements of the Facial Axis Angle was also observed by another study, suggesting that the measurement of these parameters present greater challenges [42]. It has to be noted that despite a higher discrepancy in the measurement of the Posterior Facial Height and Facial Axis Angle, good reliability was shown between fully automated AI-assisted cephalometric tracing and computerized cephalometric tracing upon manual landmark identification.
In the present study, cephalometric measurements were used for the analysis rather than anatomic landmarks, as linear and angular measurements are ultimately what drive the diagnosis and treatment plan. In general, a landmark identification is considered within acceptable levels of accuracy when the error is within 0.59 mm on the horizontal axis and within 0.56 mm on the vertical axis [35,43]. The fact that a good reliability was observed in linear and angular measurements underlines the strength of the potential clinical use of AI-assisted cephalometric tracings.
This study has some important strengths. First, only one expert investigator examined all radiographic images in order to minimize errors in landmark identification [11,41]; second, a maximum of five cephalograms per day were examined to reduce fatigue bias; third, repeated measurements were performed to assess intra-operator assessment. Nevertheless, this study is not exempt from some limitations. First, it was a pilot study based on a small sample size, with no sample size calculation; therefore, the current findings should be taken with caution until future studies with bigger samples can confirm these results. Another limitation is that the measurements were based on only one cephalometric analysis (i.e., Ricketts) and were derived from one specific specialized radiography lab, which limit the generalizability of the results to other fully automated AI-assisted cephalometric software. Future investigation should expand the number of observations to other software and cephalometric analyses, to increase the scientific relevance of the study.

5. Conclusions

These preliminary results suggest that a fully automated AI-assisted cephalometric tracing was accurate and reliable in obtaining cephalometric measurements compatible with computerized digital cephalometric tracing upon manual landmark identification. Despite being a reliable and expedited tool, digital technological advances cannot substitute the critical role of the orthodontist in integrating diagnostic records toward a definite and correct diagnosis.

Author Contributions

Conceptualization, A.A.-B. and P.G.; methodology, A.A.-B., P.G. and M.S.; data curation, A.A.-B. and M.S.; writing—original draft preparation, L.S. and A.A.-B.; writing—review and editing, L.S., A.A.-B. and P.G.; supervision, A.A.-B., L.S. and P.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Khanagar, S.B.; Al-Ehaideb, A.; Maganuretal, P.C. Developments, application, and performance of artificial intelligence in dentistry—A systematic review. J. Dent. Sci. 2021, 16, 508–522. [Google Scholar] [CrossRef]
  2. Subramanian, A.K.; Chen, Y.; Almalki, A.; Sivamurthy, G.; Kafle, D. Cephalometric Analysis in Orthodontics Using Artificial Intelligence-A Comprehensive Review. Biomed. Res. Int. 2022, 2022, 1880113. [Google Scholar] [CrossRef] [PubMed]
  3. Nguyen, T.T.; Larrivée, N.; Lee, A.; Bilaniuk, O.; Durand, R. Use of Artificial Intelligence in Dentistry: Current clinical trends and research advances. J. Can. Dent. Assoc. 2021, 87, 17. [Google Scholar] [CrossRef]
  4. Fatima, A.; Shafi, I.; Afzal, H.; Díez, I.D.L.T.; Lourdes, D.R.-S.M.; Breñosa, J.; Espinosa, J.C.M.; Ashraf, I. Advancements in Dentistry with Artificial Intelligence: Current clinical applications and future perspective. Healthcare 2022, 10, 2188. [Google Scholar] [CrossRef] [PubMed]
  5. Sangalli, L.; Savoldi, F.; Dalessandri, D.; Visconti, L.; Massetti, F.; Bonetti, S. Remote digital monitoring during the retention phase of orthodontic treatment: A prospective feasibility study. Kor. J. Orthod. 2022, 52, 123–130. [Google Scholar] [CrossRef]
  6. Sangalli, L.; Savoldi, F.; Dalessandri, D.; Bonetti, S.; Gu, M.; Signoroni, A.; Paganelli, C. Effects of remote digital monitoring on oral hygiene of orthodontic patients: A prospective study. BMC Oral Health 2021, 21, 435. [Google Scholar] [CrossRef]
  7. Menterubbianesi, R.; Tosco, V.; Vitiello, F.; Orilisi, G.; Fraccastoro, F.; Putignano, A.; Orsini, G. Augmented, virtual and mixed reality in dentristy: A narrative review on the existing platforms and future challenges. Appl. Sci. 2022, 12, 877. [Google Scholar] [CrossRef]
  8. Bichu, Y.M.; Hansa, I.; Bichu, A.Y.; Premjani, P.; Flores-Mir, C.; Vaid, N.R. Applications of artificial intelligence and machine learning in orthodontics: A scoping review. Progr. Orthod. 2021, 22, 18. [Google Scholar] [CrossRef]
  9. Kunz, F.; Stellzig-Eisenhauer, A.; Zeman, F.; Boldt, J. Artificial intelligence in orthodontics. J. Orofac. Orthop. Fortschr. Kieferorthopädie 2020, 81, 52–68. [Google Scholar] [CrossRef]
  10. Sandler, P.J. Reproducibility of cephalometric measurements. Brit. J. Orthod. 1988, 15, 105–110. [Google Scholar] [CrossRef]
  11. Sayinsu, K.; Isik, F.; Trakyali, G.; Arun, T. An evaluation of the errors in cephalometric measurements on scanned cephalometric images and conventional tracings. Eur. J. Orthod. 2007, 29, 105–108. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Dreyer, K.J.; Geis, J.R. When machines think: Radiology’s next frontier. Radiology 2017, 285, 713–718. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Melsen, B.; Baumrind, S.; Athanasiou, A. Clinical Research Application of Cephalometry, Orthodontic Cephalometry; Mosby-Wolfe: London, UK, 1995; pp. 181–202. [Google Scholar]
  14. Keim, R.G.; Gottlieb, E.L.; Vogels, I.; Vogels, P.B. 2014 JCO study of orthodontic diagnosis and treatment procedures, part 1: Results and trends. J. Clin. Orthod. 2014, 48, 607–630. [Google Scholar] [PubMed]
  15. Tsolakis, I.A.; Tsolakis, A.I.; Elshebiny, T.; Matthaios, S.; Palomo, J.M. Comparing a fully automated cephalometric tracing methods to a manual tracing method for orthodontic diagnosis. J. Clin. Med. 2022, 11, 6854. [Google Scholar] [CrossRef] [PubMed]
  16. Wang, S.; Li, H.; Li, J.; Zhang, Y.; Zou, B. Automatic analysis of lateral cephalograms based on multiresolution decision tree regression voting. J. Health Eng. 2018, 2018, 1797502. [Google Scholar] [CrossRef]
  17. Leonardi, R.; Giordano, D.; Maiorana, F.; Spampinato, C. Automatic cephalometric analysis. Angle Orhod. 2008, 78, 145–151. [Google Scholar] [CrossRef] [Green Version]
  18. Mahto, R.K.; Kafle, D.; Giri, A.; Luintel, S.; Karki, A. Evaluation of fully automated cephalometric measurements obtained from web-based artificial intelligence driven platform. BMC Oral Health 2022, 22, 132. [Google Scholar] [CrossRef]
  19. Durão, A.P.; Morosolli, A.; Pittayapat, P.; Bolstad, N.; Ferreira, A.P.; Jacobs, R. Cephalometric landmark variability among orthodontists and dentomaxillofacial radiologists: A comparative study. Imaging Sci. Dent. 2015, 45, 213–220. [Google Scholar] [CrossRef] [Green Version]
  20. AlBarakati, S.F.; Kula, K.S.; Ghoneima, A.A. The reliability and reproducibility of cephalometric measurements: A comparison of conventional and digital methods. Dentomaxillofac. Radiol. 2012, 41, 11–17. [Google Scholar] [CrossRef]
  21. Savoldi, F.; Del Re, F.; Tonni, I.; Gu, M.; Dalessandri, D.; Visconti, L. Appropriateness of standard cephalometric norms for the assessment of dentofacial characteristics in patients with cleidocranial dysplasia. Dentomaxillofac. Radiol. 2022, 51, 20210015. [Google Scholar] [CrossRef]
  22. Ricketts, R.M. A foundation for cephalometric communication. Am. J. Orthod. 1960, 46, 330–357. [Google Scholar] [CrossRef]
  23. Godt, A.; Muller, A.; Kalwitzki, M.; Goz, G. Angles of facial convexity in different skeletal Classes. Eur. J. Orthod. 2007, 29, 648–653. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Ellis, E.; McNamara, J.A. Cephalometric evaluation of incisor position. Angle Orthod. 1986, 56, 324–344. [Google Scholar] [PubMed]
  25. Tweed, C.H. Indications for the extraction of teeth in orthodontic procedure. Am. J. Orthod. Oral. Surg. 1944, 30, 405–428. [Google Scholar] [CrossRef]
  26. Heiser, W.; Niederwanger, A.; Bancher, B.; Bittermann, G.; Neunteufel, N.; Kulmer, S. Threedimensional dental arch and palatal form changes after extraction and nonextraction treatment, Part 1: Arch Length and Area. Am. J. Orthod. 2004, 126, 71–81. [Google Scholar] [CrossRef] [PubMed]
  27. Margolis, H. The axial inclination of the mandibular incisors. Am. J. Orthod. 1943, 29, 571–594. [Google Scholar] [CrossRef]
  28. Proffitt, W.R. The soft tissue paradigm in orthodontic diagnosis and treatment planning: A new view for a new century. J. Esthet. Dent. 2000, 12, 46–49. [Google Scholar]
  29. Sangalli, L.; Savoldi, F.; Dalessandri, D.; Visconti, L. Historical Development of the Planning of Incisal Position in Orthodontic Treatments: A Narrative Review of the Literature. Curr. Trends Dent. 2021, 1, 01. [Google Scholar]
  30. Park, J.H.; Hwang, H.W.; Moon, J.H.; Yu, Y.; Kim, H.; Her, S.B.; Srinivasan, G.; Aljanabi, M.N.; Donatelli, R.E.; Lee, S.J. Automated identification of cephalometric landmarks: Part 1—Comparisons between the latest deep-learning methods YOLOV3 and SS.D. Angle Orthod. 2019, 89, 903–909. [Google Scholar] [CrossRef] [Green Version]
  31. Li, C.-H.; Vandaele, R.; Mirzaalian, H.; Chen, C.; Li, W.-C.; Zheng, G.; Jodogne, S.; Chang, S.-W.; Wang, C.W.; Maree, R.; et al. Evaluation and comparison of anatomical landmark detection methods for cephalometric x-ray images: A grand challenge. IEEE Trans. Med. Imaging 2015, 34, 1890–1900. [Google Scholar]
  32. Nimkarn, Y.; Miles, P.G. Reliability of computer-generated cephalometrics. Int. J. Adult Orthodon. Orthognath. Surg. 1995, 10, 43–52. [Google Scholar] [PubMed]
  33. Mahto, R.K.; Kharbanda, O.P.; Duggal, R.; Sardana, H.K. A comparison of cephalometric measurements obtained from two computerized cephalometric softwares with manual tracings. J. Indian Orthod. Soc. 2016, 50, 162–170. [Google Scholar] [CrossRef]
  34. Lindner, C.; Wang, C.-W.; Huang, C.-T.; Li, C.-H.; Chang, S.-W.; Cootes, T.F. Fully Automatic System for Accurate Localisation and Analysis of Cephalometric Landmarks in Lateral Cephalograms. Sci. Rep. 2016, 6, 33581. [Google Scholar] [CrossRef] [PubMed]
  35. Anuwongnukroh, N.; Dewchkunakorn, S.; Damrongsri, S.; Nilwarat, C.; Pudpong, N.; Radomsutthisarn, W.; Kengern, S. Accuracy of Automatic Cephalometric Software on Landmark Identification. Mater. Sci. Eng. 2017, 265, 012028. [Google Scholar] [CrossRef]
  36. Meriç, P.; Naoumova, J. Web-based fully automated cephalometric analysis: Comparisons between app-aided, computerized, and manual tracings. Turk. J. Orthod. 2020, 33, 142–149. [Google Scholar] [CrossRef]
  37. Silva, T.P.; Hughes, M.M.; Menezes, L.D.; de Melo, M.D.; Takeshita, W.M.; Freitas, P.H. Artificial intelligence-based cephalometric landmark annotation and measurements according to Arnett’s analysis: Can we trust a bot to do that? Dentomaxillofac. Radiol. 2021, 51, 20200548. [Google Scholar] [CrossRef]
  38. Alqahtani, H. Evaluation of an online website-based platform for cephalometric analysis. J. Stomatol. Oral Maxillofac. Surg. 2020, 121, 53–57. [Google Scholar] [CrossRef]
  39. Broch, J.; Slagsvold, O.; Rosler, M. Error in landmark identification in lateral radiographic headplates. Eur. J. Orthod. 1981, 3, 9–13. [Google Scholar] [CrossRef]
  40. Baumrind, S.; Frantz, R.C. The reliability of head film measurements. Landmark identification. Am. J. Orthod. 1971, 60, 111–127. [Google Scholar] [CrossRef]
  41. Livas, C.; Delli, K.; Spijkervet, F.K.L.; Vissink, A.; Dijkstra, P.U. Concurrent validity and reliability of cephalometric analysis using smartphone apps and computer software. Angle Orhod. 2019, 89, 889–896. [Google Scholar] [CrossRef] [Green Version]
  42. Farooq, M.U.; Khan, M.; Imran, S.; Sameera, A.; Qureshi, A.; Ahmed, S.A.; Kumar, S.; Rahman, M.A. Assessing the Reliability of Digitalized Cephalometric Analysis in Comparison with Manual Cephalometric Analysis. J. Clin. Diagn. Res. 2016, 10, ZC20–ZC23. [Google Scholar] [CrossRef] [PubMed]
  43. Trpkova, B.; Major, P.; Prasad, N.; Nebbe, B. Cephalometric landmarks identification and reproducibility: A meta analysis. Am. J. Orthod. Dentofac. Orthop. 1997, 112, 165–170. [Google Scholar] [CrossRef] [PubMed]
Biomedinformatics 03 00003 g001 550
Figure 1. Bland–Altman plots of the 13 measurements for each parameter (Maxillary Convexity, Angle of Facial Conicity, Facial Axis Angle, Posterior Facial Height, and Lower Facial Height) assessing intra-operator variability. All the values are included within the upper and lower limit of the confidence interval of the mean.
Figure 1. Bland–Altman plots of the 13 measurements for each parameter (Maxillary Convexity, Angle of Facial Conicity, Facial Axis Angle, Posterior Facial Height, and Lower Facial Height) assessing intra-operator variability. All the values are included within the upper and lower limit of the confidence interval of the mean.
Biomedinformatics 03 00003 g001
Biomedinformatics 03 00003 g002a 550Biomedinformatics 03 00003 g002b 550
Figure 2. Bland–Altman plots of the 13 measurements for each parameter (Convexity, Angle of Facial Conicity, Facial Axis Angle, Posterior Facial Height, and Lower Facial Height) assessing inter-operator variability. All the values are included within the upper and lower limit of the confidence interval of the mean.
Figure 2. Bland–Altman plots of the 13 measurements for each parameter (Convexity, Angle of Facial Conicity, Facial Axis Angle, Posterior Facial Height, and Lower Facial Height) assessing inter-operator variability. All the values are included within the upper and lower limit of the confidence interval of the mean.
Biomedinformatics 03 00003 g002aBiomedinformatics 03 00003 g002b
Table
Table 1. Intra-operator variability according to each cephalometric parameter.
Table 1. Intra-operator variability according to each cephalometric parameter.
ParametersMeasurement 1 1
Mean (SD)		Measurement 2 2
Mean (SD)		Dahlberg95% CI of the Mean
Maxillary Convexity3.90 (3.43)4.15 (3.52)0.396−1.27, 0.78
Angle of Facial Conicity70.63 (4.63)70.80 (5.02)0.504−1.09, 1.18
Facial Axis Angle88.03 (4.53)87.98 (4.43)0.395−1.27, 0.78
Posterior Facial Height61.98 (6.21)61.88 (6.71)0.807−2.22, 2.42
Lower Facial Height42.15 (4.39)41.76 (4.65)0.667−1.37, 2.14
1 Measurements performed at baseline; 2 measurements performed after two weeks. CI: Confidence Interval; SD: Standard Deviation.
Table
Table 2. Inter-operator variability according to each cephalometric parameter.
Table 2. Inter-operator variability according to each cephalometric parameter.
Parametersx1 1
Mean (SD)		x2 2
Mean (SD)		Dahlberg95% CI of the Mean
Maxillary Convexity3.90 (3.43)4.20 (3.68)0.519−1.67, 1.07
Angle of Facial Conicity70.63 (4.63)70.41 (4.73)0.969−2.54, 2.98
Facial Axis Angle88.03 (4.53)87.09 (2.62)1.854−4.06, 5.93
Posterior Facial Height61.98 (6.21)63.51 (6.73)1.732−5.44, 2.38
Lower Facial Height42.15 (4.39)43.14 (4.17)1.455−4.67, 2.69
1 x1: Measurements performed by the fully automated AI-assisted cephalometric analysis; 2 x2: measurements performed by the computerized digital cephalometric software; CI: Confidence Interval; SD: Standard Deviation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Alessandri-Bonetti, A.; Sangalli, L.; Salerno, M.; Gallenzi, P. Reliability of Artificial Intelligence-Assisted Cephalometric Analysis. A Pilot Study. BioMedInformatics 2023, 3, 44-53. https://doi.org/10.3390/biomedinformatics3010003

AMA Style

Alessandri-Bonetti A, Sangalli L, Salerno M, Gallenzi P. Reliability of Artificial Intelligence-Assisted Cephalometric Analysis. A Pilot Study. BioMedInformatics. 2023; 3(1):44-53. https://doi.org/10.3390/biomedinformatics3010003

Chicago/Turabian Style

Alessandri-Bonetti, Anna, Linda Sangalli, Martina Salerno, and Patrizia Gallenzi. 2023. "Reliability of Artificial Intelligence-Assisted Cephalometric Analysis. A Pilot Study" BioMedInformatics 3, no. 1: 44-53. https://doi.org/10.3390/biomedinformatics3010003

APA Style

Alessandri-Bonetti, A., Sangalli, L., Salerno, M., & Gallenzi, P. (2023). Reliability of Artificial Intelligence-Assisted Cephalometric Analysis. A Pilot Study. BioMedInformatics, 3(1), 44-53. https://doi.org/10.3390/biomedinformatics3010003

Article Metrics

Back to TopTop