An Efficient Hybrid 3D Computer-Aided Cephalometric Analysis for Lateral Cephalometric and Cone-Beam Computed Tomography (CBCT) Systems

Abstract

Lateral cephalometric analysis is commonly used in orthodontics for skeletal classification to ensure an accurate and reliable diagnosis for treatment planning. However, most current research depends on analyzing different type of radiographs, which requires more computational time than 3D analysis. Consequently, this study addresses fully automatic orthodontics tracing based on the usage of artificial intelligence (AI) applied to 2D and 3D images, by designing a cephalometric system that analyzes the significant landmarks and regions of interest (ROI) needed in orthodontics tracing, especially for the mandible and maxilla teeth. In this research, a computerized system is developed to automate the tasks of orthodontics evaluation during 2D and Cone-Beam Computed Tomography (CBCT or 3D) systems measurements. This work was tested on a dataset that contains images of males and females obtained from dental hospitals with patient-informed consent. The dataset consists of 2D lateral cephalometric, panorama and CBCT radiographs. Many scenarios were applied to test the proposed system in landmark prediction and detection. Moreover, this study integrates the Grad-CAM (Gradient-Weighted Class Activation Mapping) technique to generate heat maps, providing transparent visualization of the regions the model focuses on during its decision-making process. By enhancing the interpretability of deep learning predictions, Grad-CAM strengthens clinical confidence in the system’s outputs, ensuring that ROI detection aligns with orthodontic diagnostic standards. This explainability is crucial in medical AI applications, where understanding model behavior is as important as achieving high accuracy. The experimental results achieved an accuracy exceeding 98.9%. This research evaluates and differentiates between the two-dimensional and the three-dimensional tracing analyses applied to measurements based on the practices of the European Board of Orthodontics. The results demonstrate the proposed methodology’s robustness when applied to cephalometric images. Furthermore, the evaluation of 3D analysis usage provides a clear understanding of the significance of integrated deep-learning techniques in orthodontics.

1. Introduction

The global market of AI in healthcare, including dentistry, was valued at USD 20.9 billion in 2024, and is expected to reach USD 148.4 billion by 2029, growing at a compound annual growth rate (CAGR) of 48.1%. This rapid growth demonstrates the transformative potential of AI in healthcare, including dentistry [1]. Recently, much research has been undertaken to understand genetic mechanisms that control the development of dental pathologies. One of the main problems ahead is related to further elucidating how genes contribute to diseases with a complex inheritance pattern. Almost every aspect of oral health is affected to a degree by genes.

Population studies indicate that tooth agenesis can occur either as an isolated trait or as part of a broader syndrome. Isolated cases may arise sporadically or follow a familial pattern, with inheritance linked to dominant, recessive, or X-linked genetic mechanisms [2]. While the genetic basis of third molar agenesis is complex and often cannot be fully explained by a simple autosomal dominant model, research suggests that a polygenic mode of inheritance plays a significant role. Beyond the absence of certain teeth, tooth agenesis can contribute to malocclusion by disrupting occlusal harmony, affecting dental arch integrity, and leading to functional and aesthetic concerns. The absence of key teeth may result in spacing irregularities, altered eruption patterns, and compensatory dental shifts, all of which can influence bite alignment and overall craniofacial development. Grahnen [3] stated that tooth agenesis is typically transmitted as an autosomal dominant trait with incomplete penetrance and variable expressivity. Twin studies have been widely used to show the importance of the genetic component involved during tooth development to control both tooth size and form. There are numerous case reports suggesting concordance for tooth agenesis in monozygotic twins, and case reports where variation in the expressivity is observed.

Lateral cephalometry is a fundamental tool applied in orthodontics and oral health, and is widely utilized for evaluating dental, skeletal, and soft tissue structures [4]. Its role in orthodontic diagnosis is essential, particularly for skeletal classification and treatment planning. By analyzing cephalometric images, clinicians can assess jaw alignment, growth patterns, and facial proportions, guiding treatment decisions for malocclusions and other orthodontic concerns. Traditionally, this process requires the manual identification of key cephalometric landmarks, followed by complex linear and angular measurements. However, while precise landmark identification is crucial for accurate analysis, it is also highly time-consuming and prone to human errors and variability. Advancements in digital automation and AI-assisted cephalometric analysis aim to enhance accuracy and efficiency, reducing the potential for inconsistencies in orthodontic assessments [5]. Cephalometric analysis is essential for evaluating skeletal structures, dental alignment, and overall facial balance. This process relies on key anatomical landmarks to ensure precise measurements. For skeletal assessment, reference points such as Sella (S) and Nasion (N) help define the cranial base, while A Point and B Point provide insight into the positioning of the upper and lower jaws. Additionally, Pogonion (Pg) and Menton (Me) are crucial to analyzing chin prominence and symmetry. Dental landmarks also play a pivotal role in cephalometric analysis. The Upper Incisor and Lower Incisor are examined to assess tooth angulation and alignment, while the First Molar is used to evaluate occlusal relationships and overall bite stability. By integrating these landmarks into cephalometric evaluations, orthodontists can develop accurate diagnoses, predict growth patterns, and design effective treatment strategies. Advancements in digital imaging and AI-driven cephalometric analysis continue to refine precision, improving diagnostic efficiency and minimizing human error in orthodontic assessments.

Applying the AI algorithm to CBCT can offer analyses of images at almost the same level of quality as experienced clinicians, who represent the current gold standard [6]. This study is one of the first to successfully enable the implementation of AI in dentistry, in particular orthodontics, satisfying medical requirements by reducing the number of landmarks used in 2D tracing.

Moreover, various current orthodontic diagnostic methods that have been standardized over time rely on dozens of manually identified landmarks on the lateral cephalogram and panorama radiographs. The two-step “detection–diagnosis” method of landmark analysis is prone to human errors and the loss of useful diagnostic information. Artificial intelligence can be used in various phases of orthodontics, from diagnosis to treatment planning and follow-up monitoring, using 3D images to reach a precise analysis in a desirable time [7].

The early identification of dental and skeletal abnormalities plays a crucial role in orthodontic diagnosis, enhancing prognosis and minimizing the risk of malocclusion. The integration of 3D scans and virtual models allows for the development of sophisticated data algorithms to intelligently assess the precise positioning of teeth, enabling more accurate evaluations. To foster optimal maxillofacial development, a hybrid clinical approach should be implemented at the earliest stages, ensuring balanced and harmonious growth [8].

Additionally, using Grad-CAM in heatmap generation can show which parts of the image are most important to the model, being a tool that shows how a deep learning model makes its decisions. It creates a heatmap that helps in understanding what a deep learning model is thinking. It highlights the key areas of the image that influence the model’s performance. This is extremely useful in improving model transparency, and making the model’s decisions more reliable. Despite the promising results achieved by the proposed AI-based system for orthodontic evaluation, several limitations remain that could impact the performance of AI when applied in teeth detection. First, variabilities in radiograph quality, such as differences in contrast, resolution, and the presence of artifacts, can adversely affect landmark detection and segmentation accuracy. Additionally, anatomical variations among patients, including age-related changes, missing or overlapping teeth, or pathological conditions, introduce complexity that may challenge the generalizability of the trained models. Another limitation lies in the dependency on large, high-quality, and diverse annotated datasets; insufficient or biased data can lead to reduced model performance and overfitting. Moreover, while techniques like Grad-CAM improve interpretability, the decision boundaries of deep learning models still remain partially opaque, potentially raising concerns about reliability in critical clinical decisions. Finally, artificial intelligence (AI) holds promise in greatly simplifying and speeding up genome interpretation by integrating predictive methods with the growing knowledge of genetic dental abnormalities.

1.1. Biological Background

The basic physical and functional unit of heredity is the gene, which is found in DNA. Genetics plays a direct role in influencing dentistry. Several genetic factors can affect dental and skeletal aspects [9]. These factors can determine the alignment, shape, and size of the teeth, as well as the skeletal relations between the maxilla and mandible, regardless of dental habits. This is illustrated in Figure 1. Genetics play a significant role in shaping orthodontic outcomes, influencing everything from jaw structure to how teeth align. Inherited traits can determine factors like tooth size, eruption patterns, and the likelihood of conditions such as crowding or overbites. Certain genetic variations may predispose individuals to skeletal imbalances, making early assessment crucial for effective treatment. With advancements in genetic research, orthodontists can now better understand these hereditary influences, allowing for more personalized care tailored to each patient’s unique biological makeup. By integrating genetic insights into treatment planning, orthodontic interventions can be made more predictive and proactive, and optimized for long-term stability.

Malocclusion, also known as “bad bite”, is a common dental issue. The malocclusion is defined as a lack of alignment between upper and lower teeth. Malocclusion typically happens when teeth are crowded or spaced apart, meaning the teeth are too large for the mouth or the mouth is too large for the teeth [10]. In malocclusion, there are three classes—class 1 is named orthognathic or mesognathic (which are not the same but are related), and is a normal variation based on the standard deviation determined statistically by the European Board of Orthodontics. In a normal molar relationship, the mesiobuccal cusp of the upper first molar fits into the mesiobuccal groove of the lower first molar. For the canines, the front slope of the upper canine meets the back slope of the lower canine, and the back slope of the upper canine touches the front slope of the first lower premolar. The line of occlusion is often altered due to issues like crowding, spacing, or bite problems such as open bite, deep bite, or crossbite. A mesognathic profile refers to a straight, balanced facial appearance. Class 2 is retrognathic (overjet); in this case, the molar relationship is defined by the lower first molar positioned slightly behind the upper first molar, with the upper molar’s cusp resting between the lower first molar and the second premolar. For the canines, the front slope of the upper canine sits ahead of the lower canine, with the lower canine sitting further back by about the width of a premolar. The line of occlusion is irregular and varies depending on facial structure, crowding, or spacing. The retrognathic profile features a convex face due to a small lower jaw or a large upper jaw. Finally, class 3 is prognathic; in this case, the upper first molar’s cusp sits behind the groove of the lower first molar. The lower canines are shifted forward compared to the upper canines by about the width of a premolar, and the lower front teeth are in full crossbite. The line of occlusion is irregular, often influenced by facial shape, crowding, or spacing. A prognathic profile, with a prominent lower jaw, is linked to Class III malocclusion, as shown in Figure 1a. Moreover, Figure 1b shows an example of a 2D lateral cephalometric of malocclusion “bad bite”, and its line tracing.

Skeletal classification is an important part of orthodontic diagnosis and treatment planning, assessed by the vertical and anteroposterior positions of the jaws in lateral cephalograms. The assessment helps clinicians predict various growth patterns such as the direction of condylar growth or dentoalveolar development planning [11]. Thus, accurate diagnosis based on proper landmark identification and analysis has been considered essential to successful usage.

1.1.1. Characteristic of CBCT

CBCTs are 3D X-rays and a variation of traditional computed tomography (CT) systems. The CBCT systems used by dental professionals rotate around the patient, capturing data using a cone-shaped X-ray beam. Standard triplane views include axial, sagittal and coronal slices, as well as oblique slices through the reconstructed volume. We have reconstructed axial CBCT images using filtered back projection reconstruction. CBCT provides a high spatial resolution of bones and teeth, which allows for an accurate understanding of the relationship between adjacent structures. CBCT has helped in detecting a multiplicity of cysts, tumors, infections, developmental anomalies, and traumatic injuries compromising maxillofacial structures. The cephalogram has been used by orthodontists as a standard method for assessing the dental, soft tissue, and skeletal relationships in patients for quite some time. 2D or direct cephalometric X-ray analysis has been successfully used for over 70 years in orthodontic practices. Currently, its status amongst orthodontists is stable, although dental clinics have started to use 3D cephalometric analysis as well [12].

1.1.2. 2D and CBCT

Even though the use of 3D X-rays is becoming more a part of dental practices, this does not mean that 2D imaging has no meaningful role; it still rightfully holds its place, as the patient suffers less from the effects of radiation exposure. Cephalometric radiography has developed as a key orthodontic diagnostic tool before craniomaxillofacial surgery is considered. Typically, cephalometric factors are measured using two-dimensional cephalometric analysis in either the lateral or the frontal view. Recent studies have shown that 3D imaging provides greater reliability [13].

This 2D approach offers a 2D view of an unmistakably 3D object. 3D technology is being increasingly used and evaluated for use in optimal orthodontic treatment, but cephalometric X-rays are still the most common X-ray cephalometric analyses in use by orthodontists. The 2D method provides sufficient information for the orthodontist to make well-informed judgments regarding the most appropriate orthodontic treatment. In the not-too-distant future, 3D cephalometric analyses are likely to be introduced more often into orthodontic practices [14].

The process of conducting a 3D cephalometric scan differs from that of a 2D cephalometric scan. A 2D cephalometric scan is taken from the side of the patient’s head, while a CBCT scan involves the C-arm or gantry rotating 360 degrees around the head, capturing multiple images from different angles that are then reconstructed to form a single 3D image. Once the cephalometric analysis is ready, the orthodontist can make informed decisions about treatment options for the patient. The 3D technique enables the orthodontist to calculate how certain orthodontic treatments will affect the patient’s jaw and the surrounding bone. It also provides insight into the growth tendency of the teeth and jaw. This information, if needed, can also be used to determine a course of treatment [15].

1.1.3. Similarities Between 2D and 3D Cephalometric X-Rays

CBCT allows orthodontists to obtain craniofacial measurements in 3D with high geometric accuracy. CBCT is also being used for 3D cephalometric analyses. However, one of the drawbacks is the relatively high amount of radiation exposure involved compared to more traditional radiographs. As a result, many orthodontists limit its use to high-priority patients who require a detailed diagnosis. This constraint will need to be addressed in order to make CBCT more widely accessible [16].

One benefit of this technology for orthodontists is the improved diagnostic accuracy compared to 2D X-rays. This is especially helpful in identifying orthodontic conditions such as overlapping teeth, hidden nerve channels, impacted teeth, airway issues, and concealed roots or anomalies in the temporomandibular joint. While 3D cephalometric X-rays are a valuable technological advancement, they do share some features with 2D X-rays [12].

The 3D cephalometric X-rays entail greater exposure to radiation, but their images come with a higher resolution, so they can be more clearly viewed. When trying to locate an incisor tooth on a cephalogram for a patient with crowded anterior teeth, the 3D method can be used to easily locate these features. This method also helps reduce identification and projection errors, which can lead to mistakes in cephalometric measurements.

1.2. Literature Review

Numerous studies have demonstrated the value of anatomical references in radiographs, showcasing their ability to enhance both accuracy and consistency. One particular study [17] aimed to develop and evaluate a computerized system for identifying cephalometric landmarks in digital images of frontal faces, specifically for forensic applications, as forensic landmarks are key points on bones or teeth that help experts tell who a person is or what happened to them. Cranial landmarks commonly used in forensic analysis include the nasion, where the forehead and nose meet; the bregma, the junction point of the skull bones at the top of the head; the Gonion, marking the angle of the lower jaw (mandible); and the Menton, the lowest point on the chin. Dental landmarks involve the tips of teeth such as the incisors, canines, and molars, as well as features like dental wear patterns, gaps between teeth (diastema), and distinctive dental restorations, including fillings and crowns. This system combined computer vision, image processing techniques, and a supervised learning approach. The technology achieved an accuracy level comparable to manually placed cephalometric reference markers, and outperformed other existing frameworks for detecting facial landmarks.

Genetic factors are responsible for controlling various aspects of tooth development, such as size, morphology, number, and position, as evidenced by twin studies. The HOX genes, crucial for oral and dental development, exhibit specific anteroposterior expression patterns. The molecular genetics of tooth morphogenesis, particularly involving Hox 7 and Hox 8 genes (currently known as MSX1 and MSX2), ensure stability in dental patterning, supporting Butler’s field theory [18].

A study by Nakashima et al. [19] assessed the role of inheritance in the development of Angle’s Class II and Class III malocclusions. The results indicate a genetic basis for both malocclusions, with stronger correlations between parents and offspring observed in skeletal measurements. Notably, significant differences in certain variables between Class II and III patients were attributed to environmental factors, further supporting the influence of genetics on malocclusions.

Dental agenesis, the most common developmental anomaly in humans, is a condition characterized by genetic and phenotypic heterogeneity. Current understanding suggests that various phenotypic forms result from different genes and molecular pathways interacting during tooth development. This explains the diverse patterns of agenesis and their associations with other oral anomalies. Over 200 genes expressed during tooth development have been identified, with mutations in some causing arrested tooth development in mice [20].

Recent years have witnessed a surge in endeavors to incorporate artificial intelligence (AI) into the realm of orthodontic diagnosis and treatment planning. The bulk of ongoing research in this domain is focused on automating the analysis of cephalometric X-rays, which remains a crucial aspect of orthodontic diagnostics. Prior to the advent of AI-assisted analyses, the process of pinpointing landmarks on cephalometric X-rays as a foundation for geometric constructions and measurements was laborious and prone to errors. Recent advancements have seen several research groups successfully automate this manual process by use of various AI algorithms [21].

The bulk of studies conducted to date exploring the implementation of AI for the automated analysis of cephalometric X-ray images have assessed the accuracy of AI based on the deviation between landmarks designated by AI and those derived from a human-based gold standard. Works in the literature posit that a discrepancy of 2 mm is deemed clinically precise enough, and thus acceptable. A meta-analysis released in 2021 by Schwendicke et al. scrutinized the precision of automated cephalometric landmark detection approaches employed by different authors, and found that the majority of the studies included were able to pinpoint landmarks within this 2 mm tolerance threshold. Nonetheless, the authors cautioned about a heightened risk of bias in many of the studies encompassed in this meta-analysis [22].

The potential of automated cephalometric analysis has garnered substantial recognition, not only from clinicians for time-saving and quality management purposes, but also from various entities looking to commercialize such services. However, the foundational data used to construct the models underpinning these services are frequently opaque and lack detailed presentation. Consequently, the objective of this investigation was to assess the accuracy and quality of all providers offering automated cephalometric analyses that met the specified inclusion criteria at the time of data collection, and to make comparisons [23,24].

A recent study carried out a comparative assessment of the computational efficiency and accuracy of modern deep learning methods, specifically Single-Shot Multi-Box Detector (SSD) and YOLOv3 (You Only Look Once, version 3), in pinpointing cephalometric landmarks. To train the algorithms, the researchers used 1,028 radiographic images and identified 80 specific landmarks for labeling [25]. The testing phase involved 283 images, and the algorithms’ accuracy was evaluated through success detection rates and point-to-point error calculations, with scattergrams employed for visualization. The findings demonstrate that YOLOv3 surpassed SSD by successfully identifying 38 out of the 80 landmarks, whereas the remaining 42 showed no significant correlation. Additionally, the YOLOv3 error plot exhibited a narrower range and higher isotropic consistency. In terms of processing speed, YOLOv3 completed computations in just 0.05 s per image, which is significantly faster than SSD’s 2.89 s. Overall, YOLOv3 achieved a 5% improvement in accuracy compared to earlier studies, showcasing its potential for effective use in clinical applications [26].

This study’s contributions are twofold. Firstly, it addresses the challenge of segmenting the landmark that must be detected by 2D orthodontic tracing to enhance the accuracy compared to the conventional method of employing multi-threading machine learning. Additionally, it evaluates the results obtained from the proposed work and the results obtained using an already-launched software applied in orthodontics automatic tracing for 2D and 3D analysis, and it improves the performance of orthodontics analysis. The subsequent sections of this paper are structured as follows: the following section delineates the materials and methods employed, the next encompasses the dataset and details each phase in the study, and the last deliberates on the findings obtained.

2. Materials and Methods

The proposed model shown in Figure 2 presents the YOLO-UNet model for automatic detection segmentation, comprising a serial combination of the YOLO and UNet architectures based on the integration between handcrafted features and panoptic segmentation. It begins with data preprocessing, followed by feature extraction and segmentation steps, leading to final decision-making.

2.1. Radiographic Image Enhancement and Pre-Processing Using “Local Contrast Enhancement” Technique

Ceph-X was developed by applying some image processing techniques to enhance the X-ray images, locate landmarks, and compute automatically linear and angular cephalometric measurements, as shown in Figure 3.

Local contrast enhancement (LCE) is a technique for bringing out detail and texture in an image. It is very similar to sharpening, which accentuates only fine detail, but it operates at a larger scale. LCE sometimes goes by other names such as clarity or micro-contrast. Most of the X-ray images in this work lack detail and create artifacts, which are especially seen for the digital image that is taken with an X-ray film. Therefore, X-ray images are necessarily pre-processed to derive a clearer image, for which the local contrast enhancement technique has been applied in this paper [27].

For the local contrast enhancement technique, let the intensity, f, of the picture elements (pixels) of an N × N digital image be indexed by (i,j) 1 <= i, j <= N. Consider a sub image of size M × M centered on (i, j), and denote the mean and standard deviation of the intensity within W by <f>W and σW (f), respectively. The objective of this method is to define a point transformation dependent on W such that the distribution is localized around the mean of the intensity and covers the entire intensity range, as shown in Figure 4. The implicit assumption is that W is large enough to contain a statistically representative distribution of the local variation of grey levels, yet small enough to be unaffected by the gradual change of contrast between the center and the periphery of the X-ray image. The adaptive contrast enhancement transformation is defined by

$$f\left(i,j\right)\to g\left(i,j\right)=255{\displaystyle \frac{\left[{\psi }_w\left(f\right)-{\psi }_w\left(f_{min}\right)\right]}{\left[{\psi }_w\left(f_{max}\right)-{\psi }_w\left(f_{min}\right)\right]}}$$

(1)

where the sigmoidal function is

$${\psi }_w\left(f\right)={\left[1+exp\left[{\displaystyle \frac{{<f>}_w-f}{{\sigma }_w}}\right]\right]}^{-1}$$

(2)

$f_{max}$ and $f_{min}$ are the maximum and minimum values of intensity within the whole image, with

$${<f>}_{w(i,j)}={\displaystyle \frac{1}{M^2}}\sum _{(k,l)ϵ(i,j)}f\left(k,l\right)$$

(3)

$${\sigma }_W^2\left(f\right)=\sum _{(k,l)ϵ(i,j)}{(f\left(k,l\right)-{<f>}_w)}^2$$

(4)

S (sella):	The midpoint of the sella turcica (a saddle-shaped depression in the sphenoid bone at the base of the skull); Used as a stable reference point in cephalometric analysis because it does not change much after early childhood;
PNS (posterior nasal spine):	The rearmost point of the bony hard palate (part of the roof of the mouth); Important for evaluating the length and position of the maxilla (upper jaw);
ANS (anterior nasal spine):	The tip of the bony projection at the front of the upper jaw (part of the nasal cavity); Helps measure facial height and the forward positioning of the maxilla;
Me (Menton):	The lowest point on the contour of the chin (mandible); Essential for assessing vertical jaw relationships and facial symmetry;
Go (Gonion):	The most outward and backward point at the angle of the mandible (lower jaw); Important for evaluating the mandibular plane angle and jaw growth patterns.

2.2. ROI Handcrafted Feature Detection

The identification of features is conducted using algorithms that describe them with vector values. The LBP and HOG feature descriptor algorithms are widely used in facial recognition applications [28,29,30,31]. LBP encodes the texture of a grayscale image by comparing the intensity differences between each pixel and its neighbors, forming a 256-dimensional feature vector as shown in Figure 5a. An advanced version, the uniform pattern LBP, addresses frequently occurring patterns. The HOG technique [28,29,30,31] as shown in Figure 5b depicts edges and corners by analyzing local intensity gradients, and generating magnitude and angle matrices, which are divided into rectangular cells. Each cell’s gradients form a nine-point histogram, creating a feature vector of size 36, as shown in Figure 5.

2.3. ROI Identification, YOLO-UNet-Based

In response to the recent challenge of panoptic segmentation, which involves assigning a label to every pixel in an image along with an instance ID, a new architectural design based on YOLO has been introduced for this specific task in computer vision. This design makes use of the YOLOv8 framework, incorporating parallel heads for semantic and instance segmentation to carry out comprehensive scene parsing. The results from this architecture demonstrate successful segmentation, ensuring that the boundaries between background classes align closely with the ground truth, while the instance masks accurately represent the shapes of objects in a scene.

Semantic segmentation [32] is the process of assigning each pixel in an input image to a specific class, a task that poses significant challenges in computer vision. This segmentation is essential for identifying classes at the pixel level, proving highly beneficial in applications like autonomous driving, pedestrian detection, and medical diagnosis. Deep learning, especially Convolutional Neural Networks (CNNs), has introduced various event segmentation techniques, leading to significant improvements in segmentation accuracy. The event segmentation module predicts the centroid of each event along with an offset vector originating from each foreground pixel and pointing towards the corresponding centroid.

Panoptic segmentation [33] emerges as a method to bridge the gap in understanding scenes through both semantic and instance segmentation. The fundamental idea of panoptic segmentation revolves around the comprehensive recognition and segmentation of the different components present in input images. Distinct colors are assigned to the teeth segmentation results to distinguish the segmentation outcomes. In contrast to semantic and instance segmentation, panoptic segmentation conducts a thorough visual examination of each individual element within the entire scene. This holistic approach greatly enhances the ability to comprehend the scene. Initially, the early panoptic segmentation algorithm simply combined the semantic segmentation model with the instance segmentation model to address the limitations of both techniques.

Nevertheless, an increasing understanding of the challenges presented by panoptic segmentation is needed. The combination of handcrafted feature extraction and panoptic segmentation enhances the incorporation of various deep learning techniques such as vgg16, Resnet 50, and YOLO-UNet.

The YOLO-UNet model is a fusion of YOLO and UNet architectures for automated detection and segmentation based on instance and semantic segmentation. YOLO (You Only Look Once) is a deep learning method focused on object detection, designed for real-time processing by treating objects as a single regression problem from raw image pixels to distinct spatial bounding boxes and their respective probability classes. The human brain rapidly identifies and interprets visual information. During object detection, the segmentation technique used by the current detection framework guides the classifier to a specific object, assessing it across various scales and positions in the test image. The selection of the YOLO architecture in this research is based on its speed and accuracy in detecting teeth. YOLO is integrated with the UNet architecture. YOLOv8, also referred to as Darknet-53 [34], consists of a total of 53 convolutional layers. This version of YOLO incorporates a residual block or shortcut connection that involves adding the inputs before the convolutional layer residual block, along with outcomes from a 1 × 1 convolutional layer filter followed by batch normalization and leaky ReLU, as well as a 3 × 3 convolutional layer filter with batch normalization and a final leaky ReLU. YOLOv8, an improved version of YOLOv5, excels in speed and accuracy. The YOLO-UNet approach is employed for the detection, segmentation and localization of teeth, which steps are essential in orthodontic analysis and diagnosis. A hybrid deep learning model that precisely segments the required landmark regions with UNet to enable YOLOv8s to efficiently detect, classify and localize maxilla and mandible teeth is shown in Figure 6b. Moreover, Figure 6c shows a sample of ROI detection results derived from lateral cephalometric assessment required in a preprocessing step before line tracing and analysis, which are important in orthodontics evaluations. Figure 6a shows a key chart for tooth labeling.

2.4. Line Tracing and Analysis Using Different AI Models

The proposed method could be used to segment the needed cephalometric landmarks of the skull and face in both bones and soft surface sections of cephalometric images using the ROI segmented in Figure 7a, using the explainable deep learning model applied on panorama radiographs. Some landmarks can be generated automatically and can be used as a guideline for locating the reference landmark using the AI model and the software shown in Figure 7b,c. The model also provides tools that allow users to mark the correct ROI with ease.

A 3D dental scan, also known as Cone-Beam Computed Tomography (CBCT), is a type of X-ray used in the dental industry. It delivers a three-dimensional image of teeth, soft tissues, nerve pathways, and bone in a single scan, revolutionizing dental care with its level of detail and precision, as shown in Figure 8a,b.

3D cephalometry is a powerful tool for measuring the skull and face in three dimensions. It offers many advantages, including providing a detailed 3D view of the skull’s hard and soft tissues, enabling 3D cephalometric analysis at real-life size (1:1 scale), avoiding the overlapping of anatomical structures and providing high accuracy and reliability. Additionally, it allows for the establishment of a 3D reference system for analyzing changes in the skull and face over time [35]. Although conventional spiral CT 3D cephalometry represents a significant advancement over conventional 2D cephalometry, it still has some drawbacks, including (1) the distortion of the position of facial soft tissues due to the horizontal positioning of the patient during imaging, (2) the limited detail of occlusion due to artifacts, (3) limited access for routine dentofacial deformity patients because of higher costs, and (4) the higher level of radiation exposure compared to other traditional plain film radiographs.

CBCT addresses many of the limitations faced in the transition from traditional 2D to 3D cephalometric analysis, necessitating the comparison of previous 2D images with the new 3D images. CBCT offers simulation tools that can fill the gap between image types. CBCT acquisitions can be made to simulate panoramic, lateral, and posteroanterior cephalometric radiographs so that they can be compared with pre-existing cephalometric databases.

3. Results

3.1. Experimental Environment

A new enhanced algorithm was verified within a distinctive environment consisting of two different datasets with diverse properties. A comprehensive range of diverse images was employed to accomplish the goal of evaluating the algorithm’s important ability to be both efficient and durable in both types of images—2D cephalometric and 3D full skull.

3.2. Dataset Description

A dataset of radiography 2D cephalometric and CBCT data of a full skull was collected from dental and maxillofacial imaging centers in Alexandria, Egypt. The CBCT data are in DICOM format. There are 7650 images in total. The dataset consists of multiple classes for 2D scans and CBCT—class 1 is for normal teeth where the maxilla and mandible are in harmony, and the bite is normal but the upper teeth slightly overlap the lower teeth; class 2 (or class II) is for malocclusions, characterized by upper molars that are too far forward compared to the lower molars, and where the maxilla shows prognathism or the mandible shows retrognathism, or both are true; Class 3 is the rarest type of malocclusion. Class 3 malocclusions arise when the lower molars are positioned excessively far forward relative to the upper molars. This occurs due to the maxilla being retrognathic, the mandible being prognathic, or both.

3.3. Results Obtained from 2D Lateral Cephalogram Analysis Applying AI Techniques

The proposed architecture employs UNet. It consists of a contracting path and an expansive path, both of which involve the repeated application of two 3 × 3 convolutions, each followed by LeakyReLU activation and either 2 × 2 max pooling (in the contracting path) or up-sampling (in the expansive path) [18]. Data augmentation was not applied on the input images to ensure that the deep learning model could learn from the original images [2]. The input images averaged 2067 × 1675 pixels, with each having a unique pixel size.

In general, the process of image segmentation involves a comprehensive description of the entire image through the extraction of features or the utilization of methods that facilitate the learning of maxilla and mandible positions. During the X-ray acquisition and transmission process, images are degraded, often with several types of noise originating from multiple sources. Thus, enhanced X-ray images are required to ensure the accuracy of the location and measurement process. Subsequently, panoptic segmentation is employed to discern the identity of the object in radiography.

In the realm of deep learning, the projection of segmentation maps for complete images is accomplished by sequentially predicting each slice while moving in several directions. The fusion of different handcrafted features aids in reaching an accurate result. In the majority of instances, the output of a neural network yields either a projected value for the relevant output factor or a set of probabilities, as in Figure 9, where we see the angle between the Gnathion and the Gonion, and Occ. refers to occlusion. Moreover, the results of angle measurement are shown in detail in Figure 10. The benefits of applying AI in orthodontics analysis were analyzed for each landmark, as shown below. These figures show the analytical tracing applied on 2D lateral cephalometric images.

After applying explainable deep learning, the results obtained are shown in Figure 10. The results were obtained by applying the proposed architecture in terms of vertical skeletal relations, sagittal skeletal relations, dento-basal relations, and dental relations. The normal values of these relations were set by the European Orthodontics Board, and the actual value is the result of the difference between the normal and the actual one. A comparison will be presented in the discussion section between this architectural analysis and the analysis performed on the Romexis version 6.4.3.33.66 platform including 2D and CBCT orthodontics analysis.

4. Discussion

The comparison of orthodontic analysis parameters between the groups revealed no significant differences in measurements. Furthermore, the intra-operator reliability, evaluated through paired T-tests, demonstrated high consistency in measurements.

The integration of technology has become a crucial element in medical and dental education, clinical research, diagnosis, and treatment planning. The rising use of dental smartphone applications by students and professionals to enrich their learning and clinical practice showcases technological advancements.

This system can be smoothly incorporated into digital workflows, enhancing the efficiency of patient management. Moreover, the data gathered from the Romexis application for cephalometric analysis can be easily stored, utilized, and retrieved as necessary, saving valuable office space typically allocated for record-keeping. The study integrated variables from a commonly used cephalometric analysis to mimic real-life situations and consistently evaluate the application’s performance. Utilizing this model as an initial step before manual tracing with any analysis software in the future will enhance results with fewer errors in a shorter time frame. However, the focus on CBCT analysis is projected to grow, as it signifies the future direction in the dental field.

4.1. Proposed Model Performance Measures

This section shows the performance measure mainly in terms of the training and validation loss and accuracy of the proposed model, and different scenarios applied to test the most reliable one.

Table 1. The training and validation loss and accuracy of different applied scenarios.

	YOLOv5				YOLOv8
	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy
Model 1 LBP	0.0215	0.9785	0.0212	0.9788	0.0114	0.9886	0.011	0.989
Model 2 HOG	0.0213	0.9787	0.0209	0.9791	0.0111	0.9889	0.0108	0.9892
Model 3 Segmentation	0.0107	0.9893	0.0103	0.9897	0.0103	0.9897	0.0102	0.9898
Model 4 Proposed Model	0.0104	0.9896	0.0101	0.9899	0.0101	0.9899	0.01	0.99

demonstrates a comparison between different performance measures for each scenario versus model four, which is the proposed one.

Mode 1 and 2 show feature fusions between LBP and HOG with YOLOv5 and YOLOv8 Model 3, applying semantic and instance segmentation with panoptic segmentation combined with YOLOv5 and YOLOv8. Figure 11a identifies the training loss and validation loss, which are all slight for the four proposed scenarios, and Figure 11b highlights the validation and training accuracy values of the different models shown in Table 1, which increased during execution to reach 98.8%. These measures are indicators of a reasonable result, with good performance. The confusion matrix for this model is shown in Figure 12, containing the counts of predicted and true values that help in the calculation of precision. Figure 13a,b illustrates a comparison between training and validation loss and accuracy when executing versus epochs of the proposed model and the confusion matrix of the proposed architecture.

Moreover, Model 4 shows the proposed model, which integrates the handcrafted feature extraction with panoptic segmentation, deriving an enhanced result that improves the accuracy and robustness of this architecture, as shown in Figure 13, representing training and validation loss and accuracy and showing a confusion matrix of the predicted and validated results.

The heatmaps in

Table 2. Applied heatmaps that predict the ROI of the explainable deep learning approach.

Class	Original	Yolov5	Yolov8	Mean Deviation
Class 1
Class 2
Class 3

improve explainability by providing a clear and understandable representation of the underlying factors influencing the model’s predictions. Grad-CAM enhances model interpretability, aids in bias detection, and facilitates decision-making refinement. Moreover, it reveals the reasoning behind the deep learning model’s predictions, by highlighting important regions in the heatmap.

The table below shows heatmaps of the explainable deep-learning predictions of various categories.

Furthermore, during the testing and validation process, this model faced some challenges—mainly the presence of some carries in multiple teeth, as ROI shown in

Table 3. The results of applying heatmaps in carries prediction and the resulting confusion matrix.

Original	Predict Phase	Heat Maps	Confusion Matrix

. We applied heatmaps and deep learning techniques to predict and detect these carries.

4.2. Comparison Between This Architecture and the Launched Software’s Results

Figure 14 shows the results for the patient’s actual value, derived using AI in 2D lateral cephalometric analysis, and applying the fusion between handcrafted features and instance and semantic segmentation to enhance the panoptic segmentation. Using this step as a preprocessing step to apply deep learning techniques in segmenting the ROI can enhance the operation of orthodontics analysis in linear and angular measurements. Figure 15 shows the mean value and the actual value using a previously launched software for orthodontics analysis based on the recommendations of the European Board of Orthodontics. The results obtained from different sources were approximately the same, which means that this model achieved a higher accuracy and good performance by the fusion of feature extraction and ROI segmentation in evaluation, thus potentially assisting the orthodontists in developing a treatment plan with much fewer errors in a shorter time.

4.3. Comparison Between This Architecture and Recent Software Result

Table 4. Comparison between models used in recent research and the proposed model.

Authors	Method	Remark	Evaluation
(Oktay, 2017) [37]	CNN AlexNet	CNN and AlexNet used for tooth detection and classification	Acc = 0.943,
(Yang et al., 2018) [38]	CNN	CNN on small dataset for automated diagnosis	Pre = 0.756,
(Zhang et al., 2018) [39]	CNN	CNN-based cascade network used to identify tooth loss, filled and decay	Pr = 0.958,
(Singh et al., 2022) [40]	CNN	CNN-based UNet employed to segment mandibule	training = 0.768, validation = 0.805
(Muramatsu et al., 2020) [41]	CNN Res-Net	Res-net with CNN, a small size of image	Acc for cnn = 0.932, Acc for Resnet = 0.98
(Tuzoff et al., 2019) [42]	CNN	Faster region-based CNN approach, moderate dataset size	Pr = 0.994
(Laishram & Thongam, 2020) [43]	CNN	Faster region-based CNN applied for tooth detection and classification	Detection Acc = 0.910, Classification = 0.99
(Eun & Kim, 2017) [44]	CNN	CNN; teeth localization	Acc = 0.90
This proposed model	Fusion between handcrafted features and segmentation based on YOLO-UNet	Different experimental scenarios employed to test the performance of the proposed model	ACC = 0.99 Pre = 0.98 Avg. improvement = 0.087

provides an advanced comparison between the proposed model and other recent models mentioned before in Reference [36], which use different models of DL and morphological pre-processing for diverse image analysis. The table includes models tested on various datasets and a large number of images under different conditions.

4.4. CBCT Analysis Discussion

Cephalometric tracings were performed using 3D CBCT imaging to ensure accurate and comprehensive orthodontic assessment. The full-skull cephalometric scans, provided in DICOM format, were processed using the Romexis software version 6.4.3.33.66 by Planmeca, which supports detailed 3D analysis. Before starting the analysis, the images were calibrated using a standardized scale to ensure precise measurements. Key skeletal and dental landmarks—including Condylion Left (CoL), Condilyon Right (CoL), Nasion (N), Anterior Nasal Spine (ANS), Posterior Nasal Spine (PNS), Menton (Me), and Gonion (Go), as shown in Figure 16a—were carefully identified, as they are also used in conventional 2D cephalometry. These landmarks served as reference points for calculating linear and angular measurements in accordance with the European Board of Orthodontics guidelines. After identifying all necessary points, the software automatically computed critical measurements, which were thoroughly recorded for evaluation. This process not only facilitated the accurate documentation of the patient’s cephalometric profile, but also helped determine their skeletal classification, and supported clinical diagnosis and treatment planning, as shown in Figure 16b.

In CBCT-based orthodontic analysis, skeletal and dental classifications serve different but complementary purposes. Skeletal classification focuses on the positions and relationships of the jaws (maxilla and mandible) within the facial structure, helping to identify underlying bone discrepancies like Class I (normal), Class II (returned mandible), or Class III (protruded mandible). Dental classification, on the other hand, examines how the upper and lower teeth align and occlude, assessing bite patterns and crowding. While skeletal analysis reveals the bone foundation, dental classification shows how the teeth fit within that structure. Together, they provide a more complete understanding of the patient’s orthodontic needs.

A comprehensive statistical analysis served as the reference landmark for this framework, maintaining a 99% accuracy level. Following the initial measurements, a significant portion of the available radiographs was randomly selected for repetition with various models, including the proposed model, 2D lateral cephalometric analysis, and CBCT analysis tracing software. This was done to evaluate the performance of the suggested framework and explore the potential transition from 2D to 3D analysis, as well as update the number of ROI utilized in the orthodontic evaluation.

5. Conclusions

The application of explainable deep learning models presents a promising approach for the early detection of dental misalignment. The findings demonstrate that the model is highly reliable, effectively identifying key regions of interest (ROI) during the prediction stage.

Despite its potential, this study faced challenges related to tooth positioning, variations in size, anatomical location, overlapping structures, and interproximal gaps, all of which complicate the detection process. One of the primary research limitations involves localization accuracy, which remains a critical aspect of improving model performance. Future advancements in automatic classification and segmentation techniques may address these challenges, leading to more robust and precise diagnostic tools. Opportunities exist for enhancing automatic classification and segmentation using updated methods with better performance.

The Grad-CAM method was used to generate heatmaps to visualize the regions upon which the model focused during the decision phase. The proposed model facilitates automated detection and segmentation, reducing the number of landmarks required for orthodontic analysis while improving diagnostic precision. By minimizing human error in landmark identification, it streamlines orthodontic evaluations. The UNet architecture, known for its pixel-wise prediction capabilities, further enhances localization accuracy, demonstrating strong performance across various biomedical segmentation applications. Among the tested models, the YOLOv8-UNet architecture achieved superior results, with an impressive accuracy of 0.99 and precision of 0.98, outperforming other models by an average improvement of 0.087. Additionally, the integration of CBCT optimizes diagnostic accuracy and treatment planning in orthodontics. Moreover, the confusion matrix shows how well the model classified four categories: Maxilla, Mandible, Occlusion, and No Teeth.

The explainable deep learning techniques have the power to improve diagnosis and treatment planning in orthodontics, and could be used in conjunction with other diagnostic tools to provide a more comprehensive assessment of dental health; also, they should be compared with other available platforms on the market to evaluate differences in accuracy, performance, and efficiency.