Discrepancies in Cephalometric Analysis Results between Orthodontists and Radiologists and Artificial Intelligence: A Systematic Review

Abstract

Cephalometry is a crucial examination in orthodontic diagnostics and during the planning of orthognathic surgical procedures. The objective of this article was to compare cephalometric measurements conducted by different specialists and systems tailored for such measurements, as well as to evaluate the capabilities of artificial intelligence in this field. In January 2024, we conducted electronic searches in the PubMed, Scopus, and Web of Science (WoS) databases. In the Scopus database, the results were refined to titles, abstracts, and keywords, while in PubMed, they were narrowed down to titles and abstracts. In WoS, the results were refined only to abstracts. The search criteria were based on the following terms: (cephalometric) AND (analysis) AND (discrepancy) AND ((orthodontic) OR (radiologist)). A total of 263 articles were identified, of which 17 met the criteria and were incorporated into the review. The review allowed us to conclude that the accuracy of cephalometric measurements relied on the expertise of the operator—specialists with more experience exhibited greater precision compared to novices or individuals not specialized in orthodontics. Cephalometric measurement computer programs yielded outcomes that streamlined work processes, minimized human errors, and enhanced precision. A novel aspect involved the application of artificial intelligence, which also demonstrated high precision and a substantial reduction in working time, although its utilization still necessitates further enhancements. Further research was required to address these limitations and to optimize the incorporation of technology in orthodontic and orthognathic surgery practices.

1. Introduction

Nowadays, orthodontic treatment relies significantly on precise radiographic diagnostics [1]. Traditional panoramic radiographs (dental X-ray of the upper and lower jaw) and lateral cephalograms (standardized lateral view X-ray of the head and neck for the evaluation of bony and soft tissue structures) serve as fundamental tools for assessing jaw, teeth, and dento-alveolar discrepancies [1]. Additionally, modern techniques, like CBCT (cone-beam computed tomography) and 3D facial evaluation, play supplementary roles in the comprehensive assessment of both facial soft and hard tissues, and become crucial in cases of doubt about the position of cephalometric points (landmarks, defined points found in the area of the human head) or a large defect that may require surgical intervention [1,2]. While some dento-alveolar discrepancies may stem solely from dental issues, others may involve skeletal abnormalities. In such cases, thorough and improved diagnostics become imperative [1,2]. CBCT is an advanced imaging modality that offers a radiation exposure dose that is 10 times less than that of conventional CT scans during maxillofacial exposure and enables three-dimensional imaging of hard tissue structures [3].

Despite the utilization of various diagnostic and orthodontic tools, certain anatomical reference points remain pivotal in establishing basic patient characteristics and, thus, in choosing the right treatment. Classic orthodontic approaches mostly use a standard lateral cephalometric radiograph to evaluate the patient’s profile, bite, and occlusion [1,2,3]. The so-called 3D individual approach for each patient is currently regarded as the method of choice for several reasons [4,5,6,7]. Due to improvements in digital diagnostics, orthodontic treatment is greatly influenced by these advancements. The use of 3D devices and the importance of cone-beam computed tomography (CBCT) significantly impact novel orthodontic and orthognathic surgery advances in the field of diagnostics, planning, and evaluation of the craniofacial skeleton. Currently, many reports describe improvements in CBCT, 3D imaging, artificial intelligence (AI) usage, and other devices to enhance overall facial and bite diagnostics. These advancements aim to establish the most accurate and up-to-date treatment plans for each case of dental and skeletal malocclusions [4,5,6,7]. Due to the numerous papers focusing on computer-enhanced planning and evaluation, it is essential to understand not only the benefits, limitations, strengths, and weaknesses of each classic 2D versus 3D evaluation but also how new techniques can assess the cephalometric image of each patient in computer-assisted and AI-improved studies [7,8,9,10,11].

In cases of severe dento-alveolar and skeletal discrepancies, a detailed evaluation of the 3D relationship of various cephalometric landmarks is crucial for planning the extent of the surgical operation. While orthodontic evaluations typically rely on individual landmark assessments by each clinician, specialized software can offer automatic features to find these reference points according to a predefined algorithm. During manual examination and evaluation of these cephalometric points, difficulties in finding their position and the angulation of selected variables may be observed [1,2,12,13]. Each orthodontist should enhance their treatment approach and leverage the benefits of new technologies, particularly the recent advances in 3D-CBCT. Software programs, in particular, hold significant potential to greatly enhance each diagnostic step [14,15]. Despite the plethora of available software, CBCTs, and other devices used in today’s dental, surgical, and orthodontic practices, there are still many underestimated and not fully evaluated aspects. One of these is the comparison between AI-derived and computer-enhanced automatic cephalometric analysis versus the classic 2D manual approach [14,15,16,17,18]. There seems to be a growing trend towards the integration of new technologies and advancements in both medicine and dentistry. This trend holds the promise of significantly enhancing patient diagnostics, therapies, and overall outcomes, thereby improving patients’ quality of life.

Selecting the most accurate cephalometric reference point is influenced by various factors. Foremost among these factors is the experience and expertise of each clinician. Additionally, the quality and clarity of the radiographs used for anthropometric purposes greatly affect the placement of selected points [2]. At the same time the distinction between the hard structures and the soft structures in their proximity is important to establish this position correctly. One potential solution to this issue could involve enhancing clinician training in selecting the most precise anatomical point positions and leveraging artificial intelligence (AI) for the development of automatic measurement protocols to achieve more reliable and accurate anatomical reference points [1,2,12,13,19,20,21,22,23,24]. The review on the usage of AI-driven and improved cephalometric analysis prevented herein is very important for the future of dentistry and orthodontics. This might be related to more accurate, 3D diagnostics, focused on each patient individual anatomy of the facial skeleton, jaws, teeth relations and might improve the reliability and reproducibility of three-dimensional cephalometric landmarks, similar to the findings presented in a review by Lisboa et al. [25].

Currently, cone-beam computed tomography (CBCT) stands as the standard for evaluating craniofacial features, encompassing both dento-alveolar and skeletal abnormalities within the facial skeleton [1,3]. Authors of this systematic review aim to emphasize the utilization of artificial intelligence for manual computer-driven cephalometric evaluations in patients undergoing orthodontic or combined orthodontic-surgical treatments. The purpose of this review is to compare and establish the most accurate and detailed method for the placement and tracing of cephalometric reference points as well as to determine whether the existing literature provides examples of potential solutions for the accurate landmark localization of individual patient characteristics, with a particular focus on artificial intelligence-based modelling. Given the absence of a systematic review on this specific topic in the searched databases, it is deemed essential to address this gap through a systematic approach.

2. Materials and Methods

2.1. Focused Question

This systematic review followed the PICO framework [26] as follows.

PICO question: In the case of cephalometric X-rays (population), will performing a cephalometric analysis by a radiologist or artificial intelligence (investigated condition) lead to a change in the measurements values (outcome) compared to the analysis results obtained by the orthodontist (comparison condition)?

2.2. Protocol

The article selection process for this systematic review was meticulously outlined according to the PRISMA flow diagram (see Figure 1). The systematic review was registered on the Open Science Framework under the following link: https://osf.io/2vyd7/ (accessed on 3 June 2024).

Figure 1. The PRISMA 2020 flow diagram [27].

2.3. Eligibility Criteria

All studies incorporated into the systematic review were required to adhere to the following criteria: they had to investigate discrepancies in cephalometric analysis results between orthodontists and radiologists, as well as between orthodontists and artificial intelligence (AI), encompassing both manual and digital methods of analysis. Additionally, studies needed to be published in English, with no restrictions on the publication date [26,28,29,30,31,32,33,34,35,36,37,38]. The authors of this review established the following exclusion criteria: studies written in languages other than English, comparisons solely between two orthodontists, clinical reports, opinions, editorial papers, review articles, and studies lacking a full-text version [26,28,29,30,31,32,33,34,35,36,37,38].

2.4. Information Sources, Search Strategy, and Study Selection

In January 2024, we conducted electronic searches in the PubMed, Scopus and Web of Science (WoS) databases. In the Scopus database, the results were refined to titles, abstracts and keywords, while in PubMed they were narrowed down to titles and abstracts. In WoS, the results were refined only to abstracts. The search criteria were based on the following keywords: (cephalometric) AND (analysis) AND (discrepancy) AND ((orthodontic) OR (radiologist)). All searches adhered to the established eligibility criteria, and only articles with available full-text versions were considered.

2.5. Data Collection and Data Items

Six researchers (K.W., J.K., N.S., K.N., J.K. and W.D.) meticulously curated the articles that met the predefined criteria. Subsequently, the pertinent data were gathered and recorded in a standardized Microsoft Excel 2013 (Microsoft, Redmond, WA, USA).

2.6. Assessing Risk of Bias in Individual Studies

In the initial phase of the study topic selection, the authors autonomously assessed the titles and abstracts of each paper to mitigate potential bias. The degree of consensus among researchers was evaluated using Cohen’s κ test. Any disparities regarding the inclusion or exclusion of a paper were resolved through collaborative discussions among the authors.

2.7. Quality Assessment

Two independent evaluators (J.M. and M.D.) assessed the procedural quality of each study included in the article. The assessment criteria focused on key aspects related to the methods of cephalometric analysis. The criteria for evaluating study design, implementation, and analysis included a minimum group size of 10 subjects, sample size calculation, presence of at least 5 landmarks, blinding, a minimum of triple measurements, and researchers’ experience. Studies were assigned scores ranging from 0 to 6 points, with higher scores indicating better study quality. The risk of bias was categorized as follows: 0–2 points denoted a high risk, 3–4 points denoted a moderate risk, and 5–6 points indicated a low risk. Any discrepancies in scoring were resolved through discussion until a consensus was reached [26,28,29,30,31,32,33,34,35,36,37,38].

3. Results

3.1. Study Selection

The initial database search across PubMed, Scopus, and WoS yielded 263 articles potentially relevant for the review. Following the removal of duplicates, 160 articles underwent screening. The initial screening of titles and abstracts resulted in the exclusion of 141 articles that did not involve a comparison of analysis between different specialists or AI/software. Subsequently, 19 articles underwent further full-text analysis, during which 2 articles were excluded for not meeting the inclusion criteria. Ultimately, a total of 17 articles were included in the qualitative synthesis of this review. The considerable heterogeneity among the included studies prevents the possibility of conducting a meta-analysis.

3.2. General Characteristics of the Included Studies

Selected studies compared the quality of cephalometric analysis performed by different specialists with varying levels of experience. The study conducted by Chen et al. [39] compared landmark detection and cephalometric analysis performed by an experienced specialist and a novice specialist. Similarly, Kuyl et al. [40] conducted a study comparing orthodontists, senior assistants, junior assistants, and non-orthodontist dentists. The results of both studies indicated that specialists with more experience demonstrated greater proficiency in performing cephalometric analysis. Furthermore, the accuracy of the analysis was found to be influenced by specialization, with orthodontists performing the analysis more thoroughly than non-orthodontist dentists. However, the level of orthodontists’ training did not impact the consistency of the measurements [39].

An alternative to the conventional method of cephalometric analysis involves the utilization of specialized software designed for this purpose. Among the included studies, eleven focused on comparing the accuracy of cephalometric analysis between clinicians and various software programs. The software programs used in these studies include CADCAS [41], CASS [42], PANN [43], PACS [44], Screenceph [45], Ceph X [46], Onyx Ceph (version 2.5.6.) [47], CephNinja (version 4.20) [48], and NemoCeph NX 2009 [49]. In a study conducted by Kumar M et al. [49], the last two software programs were compared with each other in terms of their accuracy. The majority of studies indicated that analysis performed by software was superior to traditional manually conducted analysis methods [39,42,43,45,46,50].

According to the authors of the included studies, the main benefits of using specialized software for cephalometric analysis are increased accuracy, reduced time, and minimized human error [39,43,46,50]. In their opinion, the use of cephalometric analysis systems makes it possible to reproduce intricate details and parameters that are essential for accurate diagnosis. For instance, the CASS software conducted an analysis on computed tomography, facilitating the recreation of Spee and Wilson curves [42]. However, certain studies did not identify significant differences between analysis conducted by software and that performed by a specialist [39,44,47,51]. Nonetheless, it is important to note that the selected software may not always serve as a flawless replacement. Errors can still occur despite a relatively well-conducted analysis. For instance, Zamrik et al. [52] demonstrated such a scenario in their study, where the measurement of the U1-A point was inaccurately performed. In terms of the efficacy of individual programs, a study by Kumar et al. [49] suggests that orthodontic analysis programs exhibit similar effectiveness.

The results of studies comparing artificial intelligence (AI) and traditional cephalometric analysis vary. Some authors highlight numerous advantages of AI, such as increased convenience, shorter analysis time, and high accuracy [48,53]. For instance, a study by Wang et al. [54] suggests that AI analysis demonstrates high precision, with deviations of up to 2 mm, and a detection rate of up to 80%. However, research conducted by Gupta et al. [55] did not find significant differences between traditional and AI-based analysis methods. This could be attributed to the fact that AI is still a relatively new tool in orthodontics and requires further refinement [53]. A general characteristic of the included studies has been demonstrated in Table 1.

Table 1. General characteristics of studies.

Study	Aim of the Study	Materials and Methods	Results	Conclusions
Chen et al. [39]	Evaluation of time needed by a clinician (expert and novice) and digital cephalometric analysis system (CADCAS) to perform a cephalometric analysis.	6 clinicians (3 experts and 3 novices) were asked to perform the cephalometric analysis (tracing, 19 landmark identification and measurements). The same analysis was conducted using CADCAS.	The time spent on analysis in the novice group was longer than in the expert group in cases of tracing and landmarks identification. The time needed for measurements was similar. The CADCAS showed the measurements results straight after landmarks identification.	The experience of a clinician can speed up the process of tracing anatomical structures and landmark identification but not the measurements process. The computer system CADCAS can reduce the time needed for measurements, and it can also reduce the number of errors committed in manual analysis.
Baker et al. [42]	Using the computer-assisted surgical simulation (CASS) software programs to assist in the planning of orthognathic surgeries.	Eleven patients qualified to perform orthognathic surgery were evaluated with traditional cephalometry and CASS.	In all cases that were simulated by CASS software, the surgery was successful, and all fabricated splints fit well. However, the CASS system turned out to be better in establishing the midline.	The CASS software proved to be an effective instrument for orthognathic surgery planning.
Mario et al. [43]	The use of a paraconsistent artificial neural network (PANN) in cephalometric analysis.	In total, 120 orthodontic patients were subjected to cephalometric analysis by 3 orthodontic experts and PANN using 3 units: anteroposterior, vertical, and dental discrepancy.	The results provided by the experts differed from those of PANN, with inconsistent results.	Manual cephalometric analysis is a subjective analysis by diagnosis that can be made between specialists. PANN treatment methods are more precise and eliminate the disadvantages of traditional analyses
Turner et al. [45]	A method of cephalometric analysis is described in which cephalometric X-rays were scanned using a flat-bed scanner and transparency hood. Then, the image was displayed on a computer monitor for point identification and sub- sequent cephalometric analysis using dedicated software. The reproducibility of point identification using this method was compared with two other, commonly used, methods.	The study material comprised 25 lateral skull X-rays taken as part of routine orthodontic assessment. Repeat cephalometric point identification was carried out on each X-ray using 3 methods: 1. On-screen digitization of the scanned bitmap image (Screenceph method); 2. Tracing followed by digitization of the identified points; 3. Direct digitization.	For the 8 angular and 4 linear cephalometric measurements examined, the Screenceph method compared favorably with the two conventional methods. The median difference between methods was 0.5 degrees and 0.2 mm. Using constructed Cartesian axes to examine the x, y discrepancy between repeat measurements and comparing Screenceph to tracing followed by digitization, there were significant differences in 3 instances at the 5% level and 2 instances at the 1% level. These differences represented median scores of 0.14 to 0.32 mm greater for Screenceph. Comparing Screenceph to direct digitization, 15 significant differences out of the 28 measurements were noted: 6 at the 5% level and 9 at the 1% level. The actual difference in median scores ranged from 0.2 mm to 0.53 mm	The results demonstrated that Screenceph is sufficiently accurate to use in a clinical setting but is not yet sufficiently exact for use in research projects owing to hardware limitations.
Kuyl et al. [40]	The aim of this study was twofold: (1) to evaluate the importance of the level of training in orthodontics when estimating skeletal configuration by visual inspection of the soft tissue profile, and (2) to evaluate a possible discrepancy between integumental profile (IP) and skeletal class (SC).	4 test groups comprising 10 orthodontists (0), 10 senior assistants (S), 10 junior assistants (J), and 10 dentists (0) assessed horizontal and vertical skeletal pattern from a series of slides of 100 patients. The assessments were repeated after a 1-month interval. Cephalometric analysis was also carried out by using a number of conventional analyses.	Analysis of the results with Levene’s test, two-factor mixed-design variance analysis, and Newman–Keuls’ multiple-range test showed that (1) orthodontists, independent of their level of training, are more consistent in assessing an IP than dentists; (2) assessments were more consistent for sagittal profile than for vertical profile; (3) sagittally, the Wits’ appraisal corresponds best with IP, and (4) vertically, the Steiner analysis corresponds best with IP.	1. Duplo-score indicates that sagittal discrepancies were scored more reliably (75%) than the vertical discrepancies (63%). Both scores show a considerable error, indicating that the consistency in scoring an underlying skeletal discrepancy by evaluating the soft tissue profile is not high. 2. Dentists scored less consistently than orthodontists in the repetition test. The level of advanced training in orthodontics had no influence on the consistency of scoring. 3. The recognition of the underlying skeletal discrepancy by evaluating the soft tissue profile was found to be difficult. Dentists had more problems doing so than orthodontists and orthodontists under training. 4. The sagittal profile score was similar between Sassouni’s analysis and the Steiner analysis for all test groups. A small difference was noticed between the Wits’ appraisal and the Steiner standards. The best reference was the Wits’ appraisal reflecting in 65% agreement between the soft tissue profile and the underlying skeletal growth pattern for advanced orthodontists. A percentage of 57% was found for the group of dentists. 5. The highest reflection of the underlying vertical skeletal pattern, evaluated by the soft tissue profile, was found by using the Steiner analysis. The Sassouni standards and the y-axis scored equally. Vertical scores were much lower than the sagittal ones. Differences between the four different test groups were negligible, indicating that advanced training did not contribute to an improvement in the vertical score. 6. The soft tissue profile does not reflect the underlying skeletal growth pattern very well. This does not mean that cephalograms are more accurate in determining the final treatment plan. More significance should be attached to soft tissue profile evaluation than to cephalometric analysis in orthodontic diagnosis and treatment planning.
Wang et al. [54]	To explore and compare automatic landmark detection methods in application to cephalometric X-ray images.	Anatomical landmarks were manually marked on cephalograms of 300 patients aged from 6 to 60 years as ground truth data, generated by 2 experienced doctors. Quantitative evaluation was performed to compare the results of a representative selection of current methods submitted to the challenge.	− Three methods are able to achieve detection rates greater than 80% using the 4 mm precision range. − Only one method achieves a detection rate greater than 70% using the 2 mm precision range, which is the acceptable precision range in clinical practice	Automated methods save time and manual costs and avoid problems caused by intra- and inter-observer variations or errors due to fatigue.
Kumar et al. [49]	To compare values of cephalometric analysis performed by CephNinja and NemoCeph for Downs’s analysis.	Diagnostic images were cropped, and scale image was placed on top. A laptop with a mouse-controlled cursor was used for NemoCeph, and an Android phone controlled with a finger touch screen was used for CephNinja.	The difference in mean values obtained using the two softwares showed no statistical significance for 70% of the variables. Y-axis, incisor occlusal plane angle, and the upper incisor to A-Pog showed a statistically significant difference.	CephNinja presented a satisfactory result with NemoCeph, and can be used interchangeably with confidence.
Gupta et al. [55]	To evaluate the accuracy of three-dimensional cephalometric measurements obtained through an automatic landmark detection algorithm compared to those obtained through manual identification.	A comparison of 51 cephalometric measurements (28 linear, 16 angles and 7 ratios) on 30 CBCT (cone-beam computed tomography) images. The analysis was performed to compare measurements based on 21 cephalometric landmarks detected automatically and those identified manually by 3 observers.	Inter-observer ICC for each landmark was found to be excellent ([Formula: see text]) among three observers. The unpaired t-test revealed that there was no statistically significant difference in the measurements based on automatically detected and manually identified landmarks. The difference between the manual and automatic observation for each measurement was reported as an error. The highest mean error in the linear and angular measurements was found to be 2.63 mm ([formula: see text] distance) and [formula: see text] ([formula: see text]-Me angle), respectively. The highest mean error in the group of distance ratios was 0.03 (for N-Me/N-ANS and [formula: see text]).	Cephalometric measurements computed from automatic detection of landmarks on 3D CBCT image were as accurate as those computed from manual identification.
Dot et al. [53]	Aimed to train and evaluate a deep learning (DL) pipeline based on SpatialConfiguration-Net for automatic localization of 3D cephalometric landmarks on computed tomography (CT) scans.	A retrospective sample of consecutive presurgical CT scans was randomly distributed between a training/validation set (n = 160) and a test set (n = 38). The reference data consisted of 33 landmarks, manually localized once by 1 operator (n = 178) or twice by 3 operators (n = 20, test set only). After inference on the test set, 1 CT scan showed “very low” confidence level predictions; we excluded it from the overall analysis but still assessed and discussed the corresponding results.	The model performance was evaluated by comparing the predictions with the reference data; the outcome set included localization accuracy, cephalometric measurements, and comparison to manual landmarking reproducibility. On the hold-out test set, the mean localization error was 1.0 ± 1.3 mm, while success detection rates for 2.0, 2.5, and 3.0 mm were 90.4%, 93.6%, and 95.4%, respectively. Mean errors were −0.3 ± 1.3° and −0.1 ± 0.7 mm for angular and linear measurements, respectively. When compared to manual reproducibility, the measurements were within the Bland–Altman 95% limits of agreement for 91.9% and 71.8% of skeletal and dentoalveolar variables, respectively.	To conclude, while our DL method still requires improvement, it provided highly accurate 3D landmark localization on a challenging test set, with a reliability for skeletal evaluation on par with what clinicians obtain.
Mosleh et al. [46]	This study utilizes some techniques to evaluate reliability, performance, and usability metrics using SUS methods of the developed cephalometric system, which has not been reported in previous studies.	A new system named Ceph-X was developed to computerize the manual cephalometric measurements. The system was developed by using image processing techniques, such as an enhanced X-ray image model, locating landmark model, and computation model. Ceph-X was then evaluated by using X-ray images of 30 subjects (male and female) obtained from the University of Malaya hospital. Three orthodontics specialists were involved in the evaluation of accuracy to avoid intra examiner error, and performance for Ceph-X, and twenty orthodontics specialists were involved in the evaluation of the usability and user satisfaction for Ceph-X by using the SUS approach.	Statistical analysis for the comparison between the manual and automatic cephalometric approaches showed that Ceph-X achieved a great accuracy of approximately 96.6%, with an acceptable error variation approximately less than 0.5 mm and 1°. Results showed that Ceph-X increased the specialist performance and minimized the processing time to obtain cephalometric measurements of the human skull. Furthermore, SUS analysis approach showed that Ceph-X has excellent usability per users’ feedback.	Ceph-X has proved its reliability, performance, and usability and can be used by orthodontists for the analysis, diagnosis, and treatment of cephalometric issues.
Chen et al. [41]	Investigating the difference in the positioning of cephalometric points on digital and original cephalometric X-rays.	Cephalometric points were marked on 27 X-rays and, using a computer program, on their digitized counterparts. The absolute difference between measurements was assessed, and statistical analysis was performed	A statistically significant difference (greater than 2 mm or 2 degrees) was seen in 7 of 27 cephalograms.	The difference between measurements is statistically significant but clinically acceptable.
Bruntz et al. [51]	Assessment of lateral cephalometric distortions by scanning and printing them, and assessing the accuracy of digital images to perform analysis.	8 measurement points were marked on 30 cephalometric X-rays, then they were scanned with an accuracy of 150 points per inch and printed with a laser printer. The difference in dot position on all 3 media was assessed. Statistical analysis was performed.	As a result of scanning, the images were enlarged by 0.8 mm vertically and reduced by 0.4 mm vertically. As a result of printing, vertical elongation of 1.1 mm and horizontal extension of 0.4 mm occurred. All differences are statistically significant.	The differences found are not clinically significant.
Zamrik et al. [52]	Assessment of the repeatability of cephalometric measurements performed using the traditional method and using the OnCeph Android application.	22 measurement points were marked on 30 cephalometric photos, and 26 parameters were measured twice for each method. Statistical analysis was performed.	A statistically significant difference was observed in 5 measurements (SNB and nasolabial angles and linear measurements: N I to Pog, U1-A and upper lip to S-line).	The clinically significant difference concerned only one linear U1-A measurement and resulted from an incorrect measurement by the application. The remaining differences are clinically significant.
Singh et al. [44]	Evaluation of cephalometric measurements performed by the PACS (picture archiving and communication system) compared to the traditional method.	6 measurements were made on 5 cephalometric X-rays. Statistical analysis was performed.	A statistically significant difference was demonstrated for 2 angles: SNB and lower incisors.	The differences demonstrated are not clinically significant.
Tsorovas et al. [50]	Evaluation of basic and advanced features of 5 different cephalometric analysis programs. Assessment of their compliance with the results obtained using the hand-tracking technique.	30 digital lateral radiographs comprised the material. 23 measurements were calculated by a single operator, both manually and with the use of 5 different software programs for cephalometric analysis.	Of the 23 measurements tested for each procedure, only 1 (Ii to NB (mm)) showed better agreement with hand-tracing when the advanced features were used. For the remaining 20 measurements, good agreement with hand-tracing was observed for both basic and advanced features. Two measurements (AB on FOP and Ii to A/Pog) showed poor intra-user reproducibility. Hand-tracing required significantly more time compared to both basic and advanced features. The basic features took less time to complete than the advanced features.	A computerized tracing technique, whether basic or advanced, can be considered as equally reliable to hand-tracing for cephalometric measurements, while also being less time-consuming.
Kılınç et al. [48]	Comparison and evaluation the reliability of five different cephalometric assessment methods: 1. Smartphone application 2. Tracing method CephNinja (SATM), 3. Web-based artificial intelligence (AI) 4. The conventional hand-tracing method (CHTM). 5. The driven tracing method with WebCeph (WATM).	The study enrolled 110 lateral cephalometric radiographs. One examiner measured 4 linear and 7 angular parameters using WebCeph, CephNinja, and conventional hand-tracing methods.	Statistically significant differences were found between the methods for SNA, SNB, SN-MP angle, U1-SN angle, L1-NB (mm), and E line–upper lip (mm) measurements.	Statistically and clinically significant differences were found among the groups in various measurements.
Swennen et al. [47]	Presentation of a modified hard and soft tissue lateral cephalometric cleft analysis to determine the accuracy, validity and reliability of this analysis for the future assessment of craniofacial morphology and growth in cleft patients.	Material comprised 40 conventional lateral cephalometric radiographs of non-cleft children, randomly selected. Lateral cephalometric radiographs were taken under standardized conditions. The study aimed to assess the accuracy, reliability, and validity of the modified cleft analysis. Linear and angular measurements of hard and soft tissues were recorded using 2 different methods: conventional and digital cephalometry.	Measurement error, as determined by the Bland and Altman method, was less than 1.00° and 1.00 mm. The squared correlation coefficients (r2), as determined by the Sackett et al. method, indicated high reliability.	The lateral cephalometric cleft analysis, which utilizes the Onyx Ceph software (version 2.5.6.), has been modified to analyze both hard and soft tissue. The results have demonstrated that this method is accurate, reliable, and suitable for future cleft research.

3.3. Main Study Outcomes

Studies selected in this systematic review varied in their comparison of cephalometric analysis conducted through different methodologies. Among them, one study distinguished itself by comparing analyses between three specialists with varying levels of experience [40]. Additionally, eleven studies focused on comparing analyses performed by clinicians and software programs [39,41,42,43,44,45,46,47,50,51,52]. Another study compared two different software analysis methods [49], while four studies examined the comparison between analyses conducted by clinicians and artificial intelligence [53,54,55]. Furthermore, one study investigated a comparison between clinicians, software, and artificial intelligence [48]. The studies were not homogeneous in terms of the type of the analysis. Some of the studies focused only on landmarks identification [45,53,54] while the others investigated linear and angular analysis as well [39,40,41,42,43,44,46,47,48,49,50,51,52,55]. The detailed characteristics of the included studies are shown in Table 2.

Table 2. Detailed characteristics of studies.

Authors	Comparison	Type of Analysis	Landmarks	Results
Chen et al. [39]	Clinicians vs. CADCAS	19 landmarks and 26 linear and angular measurements were assessed by 6 dentists (experts and novices) and the CADCAS program	N, S, Po, Or, Ar, Go, Me, Gn, Pog, B, A, ANS, PNS, UIA, UIE, LIA, LIE, UM, and LM	Experienced clinician performed better than novice while identifying landmarks, but measurement times were similar; CADCAS resulted in a reduction in human errors and a reduction in analysis duration
Baker et al. [42]	CASS vs. traditional analysis	Treatments of 11 orthognathic surgery patients were planned both with CASS and traditional cephalometry	Euler angles	CASS—performed similarly or better than traditional analysis; transverse maxillary cants were observed on the CT cephalometric analyses that were not observed on posterior–anterior radiographic analyses, Spee and Wilson curves were fabricated by software with an averaged occlusal plane what led to accurately corrected occlusal cant, mindfulness of discrepancies between soft tissue midline and bony midline is strongly advised
Mario et al. [43]	PANN vs. orthodontists	120 cases were examined by 3 orthodontists and PANN mathematical model	1. Anterior cranial base 2. Palatal plane (PP) 3. Occlusal plane (OP) 4. Mandibular plane (MP) 5. Cranial base 6. Y-axis 7. Posterior facial feight 8. Anterior facial height–median third 9. Anterior facial height–lower third 10. Anterior facial height 11. SNA 12. SNB 13. Long axis–upper incisor 14. Long axis—lower incisor 15. A point—Pogonion line Wits: distance between the projections of the A and B points on the occlusal plane.	PANN eliminates problems of traditional analysis; the model points out contradictions presented in the data that were not noticed by the orthodontists, precision of the system increases when more cephalometric variables are added at PANN
Turner et al. [45]	Software vs. clinician	On 25 skull X-rays, 14 landmarks were traced in the Screenceph program by mouse cursor, on tracing paper, and then digitized on graphic tablet and directly on graphic tablet; afterwards, 8 angular and 4 linear measurements were taken	S, Ar, Go, PNS, UI Apex, LI Tip, LI Apex, ANS, A, B, UI, Tip, Po, and Me	Screenceph method compared favorably with the two conventional methods; direct digitization of X-rays is the most accurate method of measurement
Kuyl et al. [40]	Orthodontists vs. senior assistants vs. junior assistants vs. dentists	100 cases assessed by 10 well-trained orthodontists, 10 senior postgraduate students, 10 junior postgraduate students, and 10 dentists using the analyses of Steiner, Wits, Sassouni, and Bjork	For the sagittal skeletal pattern, the analyses according to Steiner, Wits, and Sassouni were used, and for the vertical skeletal pattern, those of Steiner, Sassouni, and the y-axis of Bjork were used	Orthodontists perform better than dentists; sagittal profile was more consistently assessed than vertical
Wang et al. [54]	Orthodontists vs. AI	5 automatic landmark detection methods were compared with ground truth data based on landmarks marked manually by orthodontists	Sella turcica, nasion, orbitale, porion, subspinale, supramentale, pogonion, menton, gnathion, gonion, lower incisal incision, upper incisal incision, upper lip, lower lip, subnasale, soft tissue pogonion, posterior nasal spine, anterior nasal spine, articulate	Automated methods save time, intra- and inter-observer variations are eliminated; 3 methods—detection rates greater than 80%, 4 mm precision range; 1 method—detection rate greater than 70%, 2 mm precision range
Kumar et al. [49]	Software vs. software	Down’s cephalometric analysis was performed in the programs NemoCeph (landmarks marked on laptop with a mouse cursor) and CephNinja (Android phone controlled with finger touch)	Facial angle Angle of convexity A–B plane angle Mandibular plane angle Y-axis Cant of occlusal plane Inter-incisal angle Incisor occlusal plane angle Incisor mandibular plane angle U1 to A-Pog (linear)	Android-based CephNinja can be an alternative to the computer program NemoCeph
Gupta et al. [55]	Orthodontists vs. AI	3 orthodontists marked 21 landmarks on 30 CBCT images in the MIMICS software (Materialise, Belgium) and those same landmarks were marked in an automatic landmark detection program; afterwards, 51 cephalometric measurements were taken in both methods	Nasion Orbitale left Orbitale right A-point Anterior nasal spine Posterior nasal spine B-point Poronion Menton Gnathion Gonion left Gonion right Condylion left Condylion right #yeomatic point left #yeomatic point right Frontozyzomiatic left Frontozyzomatic right Sella Jugal point left Jugal point right	Both methods equally accurate
Dot et al. [53]	Clinicians vs. AI	33 landmarks were located by 1 orthodontist with 5 years of clinical experience or twice by 3 operators (2 trained orthodontists with 5 years of clinical experience, 1 final-year postgraduate maxillofacial surgeon) to create reference data; then, a deep learning-based landmarking model was created	11 Apex 11 Edge 16 Occlusal #1 Apex 21 Edge 26 Occlusal 31 Apex 31 Edge 36 Occlusal Ail apex Ai edge A6 Occlusal A point Anterior nasal spine B point Gnathion Gonion L Gonion R Infraorbital foramen L Infraorbital foramen R Internal acoustic foramen L Internal acoustic foramen R Mental foramen L Mental foramen R Menton Nasion Orbitale L Orbitale R Pogonion Porion L Porion R Posterior nasal spine Sella	Deep learning method provides highly accurate 3D landmark detection but still requires improvement
Mosleh et al. [46]	Orthodontists vs. software	Manual tracing of 30 cases was performed by an orthodontist, then digitalized radiograph samples were evaluated by 3 orthodontists by marking 12 landmarks both manually and on screen with the CephX system; the system automatically measures 6 angles and 12 lines	N: Nasion S: Sella Po: Porion Or: Orbitale Ar articulare Go: gonion Me: menton ANS: anterior nasal spine PNS: posterior nasal spine Point A: sub-spinal Point B: supramental	CephX reduces the time and effort of manual analysis, and it proved reliable
Chen et al. [41]	Orthodontists vs. software	7 orthodontic residents marked 19 landmarks both manually and on screen, and then 27 measurements were obtained and compared	SNA SNB ANB A-Nv Pog-Nv NAPog VVIts ab SN-FH SN-OP SN-MP UFA/LFA Ar-A Ar-Gn A-Gn Ar-A/Ar-Gn AArGn AGNAr ArAGn UI-SN UI-NPog U-L LI-OP LI-MP au DI u	Inter-observer errors in the manual method were comparable to the ones in digitized images; differences between data in both methods were statistically significant but clinically acceptable
Bruntz et al. [51]	Clinician vs. software vs. hard-copy	30 cases were manually traced using acetate tracing paper then digitalized (both initial and final cephalograms); digitalized versions were traced with 42 landmarks in Dolphin Imaging with the mouse cursor, and hard-copies made from digitalized X-ray photos were also traced manually; this study uses Downs, Steiner, Tweed and Riedel analyses	FP, facial plane; CON, angle of convexity; AB, A point–B point plane to Naison–Pogonion plane; Y, y-axis; OP, occlusal plane; INT, interncisal angle; L1OP, lower incisor to occlusal plane; L1MP, lower incisor to mandibular plane; U1AP, upper incisor to A point–Pogonion plane; U1FH, upper incisor to Frankfort horizontal plane; FH/NA, Frankfort horizontal plane to Nasion–A point plane; U1NA, upper incisor to Naison–A point plane; L1NB, lower incisor to Naison–B point plane; PONB, Pogonion to Naison B point plane; POL1NB, Pogonion–lower incisor plane to Naison–B point plane.	There is a difficulty in identifying certain landmarks (porion and orbitale) in computerized program and printed hard-copy; otherwise, all cephalometric analyses showed comparable accuracies
Zamrik et al. [52]	Clinician vs. smartphone software	30 cases were traced manually (22 landmarks, 7 planes, 26 linear and angular parameters) and digital versions were traced in the OneCeph program twice by the same investigator	Sella (S), nasion (N), anterior nasal spine (ANS), posterior nasal spine (PNS), A point (A), incisor superius (Is), incisor inferius (Ii), B point (B), pogonion (Pog), gnathion (Gn), menton (Me), gonion (Go), condylon (Cd), articulare (Ar), orbitale (Or), porion (Po), mid-point between molar superioris (Ms) and molar inferioris (Mi), (18) subnasal (Sn), (19) S point (Steiner analysis), (20) labial superius (LS), (21) labial inferius (LI), (22) soft tissue pogonion (Pog’). (A) SN plane, (B) Frankfort plane (Po-Or), (C) maxillary plane (ANS-PNS), (D) bisecting occlusal plane (BOP), (E) mandibular plane (Go-Gn), (F) mandibular plane (Go-Me), (G) mandibular plane (tangent to lower border of mandible)	Differences between the tracing methods were clinically insignificant except for the U1-A point measurement—the app incorrectly calculated the distance from the A line to the incisal edge of the upper central incisor rather than the facial surface of the upper incisor
Singh et al. [44]	Clinician vs. software	5 cases were traced digitally and manually, and 7 angular and 4 linear parameters were measured by 2 operators with 4 years of experience in the hand-tracing method and 1 year of experience in the digital method	SNA: angle between points S, N, and A; SNB: angle between points S, N, and B; ANB: angle between points A, N, and B; MMPA: angle between the maxillary plane (ANS to PNS) and the mandibular plane (Go to Me)	The PACS system could be an acceptable method for cephalometric analysis—no significant difference between the manual and digitalized methods
Tsorovas et al. [50]	Clinician vs. software	30 cases were traced manually by 1 observer (27 landmarks, 23 measurements) and afterwards were traced digitally in all 5 pieces of software with their basic and advanced features; the total time needed to trace the images was measured and compared	1: Sella (S), the midpoint of sella turcica; 2: nasion (N), junction of the frontal and nasal bones at the naso-frontal suture; 3: glabella (G′), the most anterior point on the forehead, in the region of the supra-orbital ridges; 4: pronasale (Pr′), the most anterior point on the nasal tip; 5: subnasale (Sn′), the junction of the columella of the nose with the philtrum of the upper lip; 6: Labrare Superios. (Ls), the muco-cutaneous junction of the upper lip and philtrum; 7: Labrare Inferios. (Li), the muco-cutaneous junction of the lower lip and philtrum; 8: soft Pogonion (Pg′), the most anterior point on the soft tissue chin; 9:mMenton (Me), the most inferior point on the bony chin; 10: Pogonion (Pg), the most anterior point on the bony chin; 11: point B, the deepest point in the concavity of the anterior mandible between the alveolar crest and pogonion; 12: lower incisor apex, the root apex of the lower central incisor; 13: lower incisor tip, the tip of the crown of the lower central incisor; 14: upper incisor tip, the tip of the crown of the upper central incisor; 15: upper incisor apex, the root apex of the upper central incisor; 16: point A, the deepest point in the concavity of the anterior maxilla between anterior nasal spine and the alveolar crest; 17: anterior nasal spine (ANS), the anterior limit of the floor of the nose, at the tip of anterior nasal spine; 18: posterior nasal spine (PNS), the posterior limit of the floor of the nose, at the tip of posterior nasal spine; 19: lower molar crown, the tip of the mesial cusp of the lower first molar; 20: lower first premolar tip, the tip of the crown of the lower first premolar; 21: inferior gonion, a mid-planed point at a tangent to the inferior border of the mandible near the gonion; 22: posterior gonion, a mid-planed point at a tangent to the posterior border of the mandible near gonion; 23: Ad1, a landmark located at the intersection of the line between PNS and basion with the posterior nasopharyngeal wall; 24: basion (Ba), the most inferior point on the anterior margin of the foramen magnum; 25: articulare (Ar), a mid-planed point located at the intersection of the posterior border of the ramus with the inferior surface of the cranial base; 26: porion (Po), the most superior point of the bony external auditory meatus; 27: orbitale (Or), the most inferior point on the infra-orbital margin	The computerized tracing method (either with basis or advanced features) takes less time and is equally reliable when compared to the manual method
Kılınç et al. [48]	Clinician vs. software vs. AI	110 cases were traced manually by 1 technician, on the smartphone application CephNinja and in WebCeph (an AI web-based orthodontic analysis platform)	SNA, SNB, SN-MP angle, U1-SN angle, L1-NB (mm), and E line–upper lip (mm)	Statistically and clinically significant differences were observed between three methods; however, the zoom function in applications gave much clearer images; AI software promises higher comfort, practicality, and speed
Swennen et al. [47]	Clinician vs. software	40 cases were traced manually by 2 investigators and in Onyx Ceph	Ba = basion, anterior lip of the foramen magnum; S = sella, estimated center of the hypophyseal fossa; R = registration point, point of crossing of the greater wing of the sphenoid and planum sphenoidale; N = nasion, junction of the nasal and frontal bone; NB = lip of nasal bone; ANS = anterior nasal spine; A = point of greatest concavity of the alveolar process of the maxilla; UI = upper incisor; UI-apex = upper incisor apex; LI = lower incisor; Li-apex = lower incisor apex; B = point of greatest concavity of the mandibular alveolar process; Pog = pogonion, most prominent point on the chin; Gn = gnathion, point on the symphysis between pogonion and menton farthest from the condyle; Men = menton, most inferior point on the symphysis; Co = condylion, posterior superior point on the outline of the condyle; PTM = pterygomaxillary fissure, the inferior point in the fissure; PNS = posterior nasal spine. Hard tissue landmarks necessary to construct other cephalometric reference points: TgA = mandibular body tangent; TgP = mandibular ramus tangent; Ar = articulare, point at the intersection between the contour of the mandibular ramus and occipital bone; UM-cusp = upper molar mesial cuspides; LM-cusp = lower molar mesial cuspides. Constructed landmarks: Go = gonion, a constructed point on the outline of the mandible by bisecting the ramus plane (Ar-TgP) and body plane (TgA-Men); PMP = posterior maxillary point, a constructed point created by dropping perpendicular to the maxillary plane (PNS-ANS) from the PTM; OccA = anterior point of the occlusal plane, a constructed point at the midline between UI and LI; OccP = posterior point of the occlusal plane, a constructed point at the midline between UM-cusp and LM-cusp. References lines: Ba-N = reference line to scale all linear measurements; S-N = anterior cranial base, line from S through N; MxPl = maxillary plane, line from PNS through ANS; OccPl = occlusal plane, line from OccP through OccA; MdPl = mandibular plane, line from Go through Men	Onyx Ceph software is an accurate and reliable method for lateral cephalometric cleft analysis; both traditional and digitized method proved to be highly accurate and reliable techniques for measuring hard and soft tissues in patients with clefts

N—nasion, S—sella, Po—porion, Or—orbitale, Ar—articulare, Go—gonion, Gn—gnathion, Me—menton, ANS—anterior nasal spine, PNS—posterior nasal spine, Point A—sub-spinal, Point B—supramental, SNA—angle between points S, N, and A, SNB—angle between points S, N, and B, ANB—angle between points A, N, and B, MMPA—angle between the maxillary plane (ANS to PNS) and the mandibular plane (Go to Me), S-N—linear distance from sella turcica (S) to nasion (N), S N-Ar—angle between anterior cranial base (S-N) and articulare (Ar) represents cranial base flexure, SNA—anteroposterior position of the maxilla relative to the anterior cranial base, N l to A—linear measurement from nasion perpendicular line to A point, Co-A—linear distance from the condylion to A point represents the effective mid-facial length, SNB—anteroposterior position of the mandible relative to the anterior cranial base, N I to Pog—linear measurement from nasion perpendicular line to pogonion (Pog), Co-Gn—linear distance from the condylion to the gnathion represents the effective mandibular length, Go-Gn—linear distance from gonion and gnathion represents mandibular body length, SN-Go Gn—angle between sella turcica–nasion (SN) line and the mandibular plane (Go-Gn), FMPA—angle between Frankfort (orbital–porion) and mandibular planes, Ar-Go-Me—angle between the mandibular plane (Go-Me) and ramal plane (Go-Ar), ANB—difference between SNA and SNB angles, ANS-Me—linear distance from the anterior nasal spine (ANS) to the menton (Me) represents the lower anterior facial height, S-Go/N Me—ratio between total posterior and anterior facial heights (sella–gonion and nasion–menton), U1-A point A—line is constructed through point A parallel to the nasion (perpendicular) and the distance measured to the facial surface of the upper incisor; it relates the upper incisor to the maxilla, LI-A Pog—distance from the facial surface of the lower incisor to the line drawn through point A and pogonion; it relates the lower incisor to the mandible, IMPA—angle between long axis of lower central incisor and the mandibular plane (tangent to lower border of mandible), U1-NA—angle between nasion–A point (NA) line and the long axis of upper incisor, U1-NA—linear measurement from the tip of upper central incisor to the NA line, L1-NB—angle between the nasion–B point (NB) line and the long axis of the lower incisor, L1-NB—the linear measurement from the tip of lower central incisor to NB line, UL to S line—linear measurement from most prominent point of upper lip to Steiner’s S line, LL to S line—linear measurement from most prominent point of the lower lip to Steiner’s S line.

3.4. Quality Assessment

Among the articles included in the review, one [49] was rated as high-quality, achieving a score of 5/6 points. Seven studies [41,42,44,45,47,51,52] were categorized as low-quality. Furthermore, nine studies [39,40,43,46,48,50,53,54,55] were identified as having a moderate risk of bias, scoring between 3 points (see Table 3).

Table 3. Quality assessment table.

	Group Size at Least 10 Subjects	Sample Size Calculation	Blind Study or Not	At Least 5 Landmarks Used	Triple Measurements	Experience of Researchers	Total Points	Risk of Bias
Chen et al. [39]	1	0	0	1	0	1	3	moderate
Baker et al. [42]	1	0	0	0	0	0	1	high
Mario et al. [43]	1	0	0	1	0	1	3	moderate
Turner et al. [45]	1	0	0	1	0	0	2	high
Kuyl et al. [40]	1	0	0	1	0	1	3	moderate
Wang et al. [54]	1	0	0	1	0	1	3	moderate
Kumar et al. [49]	1	1	1	1	0	1	5	low
Gupta et al. [55]	1	0	0	1	0	1	3	moderate
Dot et al. [53]	1	0	0	1	0	1	3	moderate
Mosleh et al. [46]	1	0	0	1	0	1	3	moderate
Chen et al. [41]	1	0	0	1	0	0	2	high
Bruntz et al. [51]	1	0	0	1	0	0	2	high
Zamrik et al. [52]	1	0	0	1	0	0	2	high
Singh et al. [44]	0	0	0	1	0	1	2	high
Tsorovas et al. [50]	1	0	0	1	0	1	3	moderate
Kılınç et al. [48]	1	0	0	1	0	1	3	moderate
Swennen et al. [47]	1	0	0	1	0	0	2	high

4. Discussion

This review looks at studies relevant to comparing human cephalometric assessments with assessments enhanced by artificial intelligence-enhanced software [23]. Although traditional methods of cephalometric assessment have prevailed for years, the advent of modern tools has made it possible to automatically refine such records based on computation or artificial intelligence [24]. Nevertheless, the number of articles on the subject is small or their availability remains limited, which makes it necessary to postulate that further similar studies should be carried out as widely as possible. AI-enhanced software is already reported in some papers from general dentistry, focusing on its role on dental feelings, occlusion, prosthodontics and general conservative dentistry [56,57,58,59,60]. Each orthodontic treatment focuses on achieving a good, balanced and functioning occlusion to improve patients bite, chewing and overall quality of life. It seems that AI and 3D-CBCT can improve orthodontics and lead to better and more accurate patient outcomes from orthodontic treatments.

Currently, a comprehensive evaluation of the facial skeleton in conjunction with soft tissue proportions, skeletal features, teeth, and occlusion holds paramount importance in contemporary orthodontic treatment and orthognathic surgery protocols. Numerous advancements in orthodontic and orthognathic studies have been reported, as noted by Prasad et al. [61] and Starch-Jensen et al. [1]. The main findings suggest that new technologies in cephalometric assessment are the future, but significant efforts are needed to fully optimize their performance.

In the past, the process of measuring angles and distances on X-ray film, as well as transferring reference points on tracing paper, was a time-consuming one [3]. The time required to manually draw points and create lines on tracing paper, take measurements with a ruler and protractor, and finally record cephalometric measurements was approximately 30 min. The digitization of X-rays and the ability to trace reference points with a computer mouse on the monitor screen represented a significant advance. The capacity to magnify individual structures on the monitor screen while marking successive reference points, coupled with the ongoing enhancement of digital cephalograms, has markedly enhanced the precision of measurements. Concurrently, the time required for the analysis using computer programs, which automatically count the individual parameters of the cephalometric analyses and place the results in tables and graphs after the doctor has entered the points, has decreased to approximately one to two minutes. It is crucial to recognize the benefits of new technologies and software that can enhance the work of dentists, particularly orthodontists. Over the past decade, advancements in virtual reality techniques, computer-enhanced technologies, cone-beam computed tomography (CBCT), and many others have significantly improved overall dental treatment. Today, 3D cephalometric measurements, 3D virtual treatment planning for skeletal, facial, jaw/bone, and teeth measurements, along with detailed 3D studies of superficial, skeletal, and skin anatomy, appear to greatly enhance success rates and reduce the occurrence of troublesome complications or limitations in treatment algorithms [62,63,64,65,66]. Despite the advancements and improvements in new techniques, it is always important to consider the potential for human and computer errors. Therefore, after each evaluation, it is crucial to double-check the results and ensure that the proposed treatment or diagnostic algorithm is sufficient and adequate for each patient’s case.

Some computer programs employ artificial intelligence to perform tasks previously reserved for medical professionals, such as identifying and marking reference points [67]. Nevertheless, the creators of these programs consistently advise medical professionals to verify that the program has correctly identified the relevant points. This is due to the fact that some bone and skin structures that overlap in an X-ray image may be misinterpreted by the computer program. Furthermore, the difficulty of computer programs in reading visual material is a common feature used to detect malicious bots employing CAPTCHA techniques.

The most illustrative example of this type of difficulty in orthodontic practice is the determination of Downs’ A-point [68]. In such instances, the bony structure may be erroneously identified as the shadow of the buccal fat pad. An erroneous determination of this point will result in a significant error in the diagnosis of the sagittal position of the maxilla in relation to the mandible, as evidenced by incorrect ANB and WITS parameters. It can be concluded that the reduction in time for the application of reference points achieved by the automatic procedure is a minor advantage, since at most two minutes of analysis time are saved. However, this does not guarantee the correctness of the diagnosis, especially in difficult cases. It is crucial to emphasize that orthodontic diagnosis is not solely dependent on the speed of the process.

In turn, other factors, such as patient positioning, maintaining a good NHP, and ensuring appropriate resting positions of the jaw, teeth, facial features, skull, and body posture during examination, are areas that still warrant attention [1,61]. The most critical aspect of evaluating facial photographs, CBCT scans, and 3D assessments lies in achieving a natural, physiologically balanced head position and patient silhouette. Many authors seem to align with this perspective [12,69]. Consequently, the utilization of craniometric studies in anatomical and radiological tracings can be more dependable. When the patient’s head position is secured in the most reliable manner, further cephalometric analysis can be conducted [68]. In the past, such patient positioning for evaluations was pivotal in ensuring the most accurate and precise placement of craniometric and anthropometric points within the assessed data. In today’s context, this scenario still holds significant importance. Even though automatic tracing software can identify precise anatomical landmarks, the accuracy of proportions, angles, and correlations between points may be compromised due to head positioning in the NHP (natural head position). Whether utilizing manual or automatic/computer-enhanced tracing methods, as well as AI software, the NHP and patient posture during the examination are crucial factors. Both the experience and expertise of clinicians, as well as the proficiency and knowledge of younger clinicians, can greatly influence cephalometric tracings and their accuracy [13]. While manual tracings rely on the clinician’s experience and knowledge to utilize craniometric reference points, automatic or AI-driven software evaluates images based on algorithms or other automated tools. It seems, however, that the issue of correct patient positioning remains outside the area that can be optimized with computer programs.

Regardless of the method used, the most crucial factor is to establish the most accurate, comprehensive, and sufficient analysis for each patient’s case. The precise placement of anatomical cephalometric landmarks to enhance the linear, angular, and planar comparison of selected landmarks is pivotal for planning both orthodontic and orthognathic surgery procedures [20]. Numerous authors, such as Chen et al. [23], Kuyl et al. [40] and others, have studied the skill of placing anatomical and cephalometric points and concluded that a skilled specialist with more years of active clinical practice demonstrates greater proficiency in accurately placing the necessary reference points for conducting a precise cephalometric analysis [19]. Furthermore, it is important to note that clinicians, dentists, surgeons, and orthodontists who perform such analyses routinely possess more experience compared to those who perform them sporadically or infrequently.

With the advancements in 3D software and the computer-enhanced evaluation of radiographs, CBCT studies, and facial photographs, new techniques used in both orthodontics and orthognathic surgery are being widely embraced for their rapid and precise analysis. Similar studies conducted by Chen et al. [39], Baker et al. [42], Mario et al. [43], Turner et al. [45], Mosley et al. [46], and Tsorovas et al. [50], among others, tend to corroborate these findings. Today’s software aims not only for precise and accurate measurements but also to mitigate any potential human errors or inaccuracies in craniometric evaluation [1,61]. From the author’s perspective, this situation is quite evident, but when patients’ head positioning in NHP during radiographs, facial photographs, and CBCT evaluation is misaligned, even with numerous new devices and software, the results may not be accurate. This situation holds particular importance in planning surgical interventions, such as in orthognathic surgery, where establishing the full scope of the soft and hard tissue contour and balance is imperative without any disruptions. When this criterion is not met, numerous disturbances in measurements are observed, irrespective of the computed-enhanced/AI or classic manual evaluation methods utilized in various studies. This aspect is considered one of the most crucial parts of each cephalometric evaluation and has been well-documented [19,20,70]. From the author’s perspective, regardless of the anthropometric reference points and techniques used, some cases of severe skeletal malocclusion surpass the capabilities of standard planning methods, necessitating a combined manual-approach and 3D/AI-driven software.

The discussion regarding the comparison between manual classic cephalometric evaluation and those conducted by AI-computed assisted software is steadily gaining traction in the existing literature. Both methods possess their respective advantages and disadvantages, and it is particularly noteworthy that some very complex and challenging cases still require traditional manual hand-made evaluation, cephalometric measurements, and estimation. Presently, the rapidly expanding integration of AI and computed CBCT evaluation holds promise for the future of next-generation orthodontic and orthognathic surgery treatments. However, there is still much work to be carried out, primarily because further improvements are necessary [6]. The most significant clinical observation is that contemporary orthognathic surgery relies heavily on 3D-CBCT patient evaluation, complemented by facial, bite, and occlusal photographs, which are integrated and amalgamated to estimate, measure, and predict the most effective treatment plan for each patient case. Without the aid of new technologies, such as CBCT and AI-automatic software tracings, achieving this level of precision would not be feasible. However, it is important to note that a few exceptionally complex cases of severe skeletal malocclusion may still necessitate manual-based evaluations, particularly those associated with significant skeletal jawbone disproportions and similar factors [71,72]. While the key conclusions drawn by the authors align partially with those of other relevant papers, it is crucial to acknowledge the variability in results across different studies, attributable to the diverse capabilities and resources of each tool and software program for cephalometric tracings.

The following limitations were encountered during the study: an insufficient number of papers comparing natural head position (NHP) and the usage of AI-driven software, leading to gaps in understanding; significant variability in the software and computer-enhanced programs utilized, along with their 3D/measurement capabilities, complicating comparisons and standardization; a limited number of studies on patients discussing and describing the preparation, positioning, and evaluation process before and during the study, which hinders reproducibility and generalizability; diverse cephalometric measurements utilized across various orthodontic protocols, making it challenging to establish consensus and comparability; limitations in the size, resolution, and accuracy of CBCT scans, traditional radiographs, scanners, and software utilized, affecting the quality and reliability of the data; and a plethora of radiographic software, programs, and companies involved in the dental market, contributing to heterogeneity and potential inconsistencies in methodologies and results. Significant heterogeneity among the included studies does not allow for performing a meta-analysis. However, further research should be conducted to enable proceeding with a meta-analysis.

5. Conclusions

The studies examined comparisons between specialists of varying experience levels, clinicians, software programs, and artificial intelligence (AI). In general, specialists with more experience demonstrated greater proficiency in cephalometric analysis. However, the accuracy of the analysis was influenced by specialization. The utilization of specialized software for cephalometric analysis demonstrated numerous advantages, including enhanced precision, reduced time requirements, and the minimization of human errors. Such errors may be caused by intra- and interobserver variations, or by operator fatigue. While some studies have indicated that software-based analysis is superior to traditional methods, others have not identified any significant differences. The efficacy of AI-based analysis was found to vary across studies. Some studies highlighted the advantages of AI-based analysis in terms of convenience, shorter analysis time, and high accuracy. However, other studies found no significant differences between AI-based analysis and traditional methods. It is, however, important to note that AI in orthodontics is still in a state of evolution and requires further refinement. There are still challenges that need to be addressed, including variations in software and methodologies, limitations in patient positioning protocols, and diverse cephalometric measurement standards across studies. This may indicate that optimization using new software and artificial intelligence is not feasible in all areas. Conversely, the benefits of automated referencing, such as reduced analysis time, are in fact minimal, as the advantage of automated referencing over human referencing is less than two minutes. While AI-driven enhancements offer potential for the future, manual evaluations remain necessary for complex cases. Further research is required to address these limitations and to optimize the incorporation of technology in orthodontic and orthognathic surgery practices.