Performance Validation of CEPH_2D, a Novel Artificial Intelligence Tool for Automatic Cephalometric and Obstructive Sleep Apnea Syndrome Analyses

Highlights

What are the main findings?

• CEPH_2D showed promising performance for automatic cephalometric landmark detection and pharyngeal airway segmentation on lateral cephalograms.
• The system achieved a rapid processing time and showed stable performance across the examined age and sex subgroups.

What are the implications of the main findings?

• CEPH_2D may represent a useful adjunctive tool to support orthodontic and obstructive sleep apnea syndrome-related radiographic assessment.
• Clinician supervision remains advisable, particularly for landmarks showing higher detection errors, and larger externally validated studies are required.

Abstract

Background/Objectives: Cephalometric analysis is essential in orthodontics and for studying conditions such as obstructive sleep apnea syndrome (OSAS). However, manually identifying anatomical landmarks and segmenting the pharyngeal airway on lateral cephalograms can be time-consuming and prone to errors. This study evaluates the CEPH_2D system, an AI-based tool designed to automate cephalometric landmark detection and pharyngeal airway segmentation from 2D lateral cephalometric radiographs. Methods: The system was evaluated on 35 anonymized lateral cephalograms obtained from patients aged 6–65 years, including mixed and permanent dentition cases. Two experienced clinicians generated and reviewed the ground truth annotations for cephalometric landmark localization and pharyngeal airway segmentation. System performance was assessed using mean radial error (MRE), successful detection rate (SDR), mean average precision (mAP), Dice similarity coefficient (DSC), precision, recall, and inference time. Results were compared with manual methods and existing automated tools. Results: The system reached a mean radial error (MRE) of 0.740 ± 0.793 mm for the key point detection task and a mean Dice Score (mDSC) of 0.935 ± 0.040 with an average processing time of 2.557 ± 0.504 s. Conclusions: CEPH_2D appears to be a promising adjunctive tool for automatic cephalometric landmark detection and pharyngeal airway segmentation on lateral cephalograms, although clinician verification remains advisable before clinical interpretation or treatment planning, particularly for landmarks showing higher detection errors.

1. Introduction

Cephalometric analysis is an important tool in orthodontics and craniofacial studies. It involves identifying anatomical landmarks on lateral cephalograms to measure distances and angles, helping to diagnose craniofacial conditions and plan treatments [1,2,3]. However, this process is often done manually, which can be time-consuming and prone to errors influenced by the examiner’s experience, focus, and fatigue [1,2,3].

In addition to orthodontic uses, lateral cephalograms are valuable for studying obstructive sleep apnea syndrome (OSAS), a condition where the airway becomes partially or fully blocked during sleep. This requires analyzing anatomical landmarks and assessing the pharyngeal airway space. Manual analysis can be challenging, particularly for complex cases, which has led to the development of automated systems to streamline the process [4,5].

However, a true three-dimensional assessment of the upper airway remains challenging. Volumetric evaluation generally requires CBCT imaging, which may provide more detailed anatomical information but is not always justified in routine assessment because of radiation exposure considerations [4,5].

Automated cephalometric landmark detection has been a significant focus of research in recent years. Early approaches, such as those employing edge detection and knowledge-based tracking, were followed by machine learning techniques, including regression-voting approaches, and later by deep learning methods [6,7]. Deep learning methods, including Convolutional Neural Networks (CNNs), have shown remarkable success in accurately and reliably identifying cephalometric landmarks [8,9,10], also benefiting from advances in training and data augmentation strategies [11]. For instance, Park et al. [12] compared YOLOv3 and SSD for landmark detection, finding that YOLOv3 outperformed SSD in terms of both accuracy and speed. Qian et al. [13] introduced a multi-head attention neural network for improved performance.

While many studies have focused on patients with permanent dentition, fewer have considered the unique challenges posed by mixed dentition. Popova et al. [14] explored performance differences between these two groups, aligning with our work’s investigation into this distinction.

Multiple commercially available AI-based solutions for automatic cephalometry are offered by various companies. These include WebCeph by AssembleCircle Corp. (Seongnam, Republic of Korea), OneCeph by NXS Corp. (Hyderabad, India), AudaxCeph by Audax d.o.o. (Ljubljana, Slovenia), DentaliQ.ortho by CellmatiQ GmbH (Hamburg, Germany), NemoCeph by Nemotec S.L. (Madrid, Spain), and CephX by Orca Dental AI Ltd. (Dover, DE, USA) [1,2,3,15].

Regarding pharyngeal airway segmentation, Sin et al. [4] proposed a deep learning-based system for CBCT images, while Meng et al. [5] developed a method for segmenting the nasopharynx, oropharynx, and hypopharynx from lateral cephalograms, similar to our approach.

Recent studies have extended AI-based cephalometric analysis to CBCT imaging, unsupervised landmark localization, and cervical vertebral assessment on lateral cephalograms [16,17,18,19,20].

Automated cephalometric superimposition and vertical dimension assessment have also recently been investigated using AI-based approaches [21,22,23].

Beyond landmark analysis, AI has been applied to panoramic radiograph interpretation, dental structure and tooth segmentation, detection of impacted or missing teeth, agenesis and supernumerary teeth, automated caries classification, and the creation of annotated dental radiographic datasets to support machine learning development [24,25,26,27,28,29,30,31,32,33,34].

In this context, CEPH_2D is a deep learning-based artificial intelligence (AI) system, integrated into the Neowise software (version 1.5, Cefla S.C., Imola, Italy), that automates two key tasks: detecting and classifying anatomical landmarks used in cephalometric and OSAS analysis and segmenting the pharyngeal airway. By automating these steps, CEPH_2D aims to save time and improve accuracy compared to manual methods.

This study aims to validate the performance of CEPH_2D, evaluating its accuracy and reliability in detecting landmarks and segmenting the pharynx. The system’s ability to handle different patient categories, such as mixed and permanent dentition, and distinguish between regions of the pharynx is also assessed. The findings may help determine the potential role of CEPH_2D as an adjunctive tool in craniofacial and sleep apnea-related radiographic assessment.

2. Materials and Methods

2.1. Dataset

A total of 35 lateral cephalograms were collected from patients aged 6 to 65 years, representing a broad spectrum of European patients, including various ethnicities and both sexes. To analyze the impact of dental development, patients were categorized into two groups: mixed dentition (6–12 years) and permanent dentition (13–65 years). The distribution of these images is summarized in

Table 1. Distribution of lateral cephalograms for system evaluation (testing dataset) by age and sex.

Population		System Evaluation (Test) Dataset-N	System Evaluation (Test) Dataset-%
Sex	Male	18	51.43%
	Female	17	48.57%
Age	6–12 (mixed dentition)	14	40.00%
	13–65 (permanent dentition)	21	60.00%

.

All lateral cephalograms were acquired using a Hyperion X9 Pro MyRay device (Cefla S.C., Imola, Italy) under routine clinical operating conditions. Although all radiographs were obtained using the same imaging platform, minor variability in acquisition conditions and image characteristics representative of routine clinical workflows was present across the dataset.

All radiographs were anonymized before analysis and included cases with various conditions, such as missing teeth, fillings, and root canal treatments, as well as metallic elements such as splints and crowns.

Table 2. Distribution of lateral cephalograms for system evaluation (testing dataset) by finding.

Condition	System Evaluation (Test) Dataset-N	System Evaluation (Test) Dataset-%
Splint	5	14.29%
Filling	6	17.14%
Crown	2	5.71%
Canal treatment	2	5.71%
Missing or extracted teeth	3	8.57%

shows the distribution of clinical conditions across all images. Patients with maxillofacial syndromes, previous reconstructive surgery, or a history of maxillofacial trauma were excluded from this study. Radiographs considered unsuitable for diagnostic evaluation because of insufficient image quality were also excluded during the annotation and review workflow.

The present study constitutes an independent external performance validation study of CEPH_2D that was conducted without access to training data or implementation details.

As CEPH_2D is a commercially developed AI system integrated into Neowise software, detailed information regarding the original training dataset, including its size, source, diversity, and annotation workflow, was not available to the investigators because it is proprietary to the manufacturer. Therefore, the present study focuses exclusively on the independent external validation of the system on a dedicated testing dataset.

2.2. Ground Truth Definition for System Validation

The validation dataset was prepared with particular care in order to provide a reliable reference for evaluating the system’s performance. Two experienced clinicians, each with at least five years of clinical practice, were involved in the preparation of the dataset and worked with complementary roles, namely annotation and review. This workflow was intended to improve the consistency of the ground truth data and reduce potential annotation-related bias.

The Computer Vision Annotation Tool (CVAT, version 2.21.0, CVAT.ai, Wilmington, DE, USA), a self-hosted web application, was used to manually classify and annotate the radiographic images. The use of a self-hosted system also allowed the annotation platform to be installed on a server physically located in the European Union, in compliance with GDPR requirements.

The annotation process was organized into two phases:

• Initial Annotation: The annotator performed manual annotations of the keypoints and pharynx segmentation on each radiograph using the annotation tool and assessed image quality to exclude radiographs unsuitable for diagnostic use.
• Review and Quality Assurance: The reviewer meticulously examined the initial annotations to ensure their accuracy and adherence to established anatomical guidelines. Using cross-referencing techniques and visual validation, the reviewer corrected any discrepancies identified in the annotations. This phase also involved quality control checks to address any issues related to image quality and resolution, segmentation consistency, or potential operator errors.

The test dataset served as ground truth for evaluating the algorithm’s performance. By incorporating an annotation and review protocol, this process aimed to improve the consistency of the reference annotations and provide a clinically relevant benchmark for performance evaluation.

Representative examples of CEPH_2D outputs are shown in Figure 1 and Figure 2. These figures illustrate the automatic detection of cephalometric landmarks and pharyngeal airway segmentation on lateral cephalograms from patients with permanent and mixed dentition, respectively.

2.3. Metrics

As mentioned before, validation involves comparing the model’s predictions against the ground truth manual annotations using a range of established metrics commonly used:

• Keypoint detection metrics assess the accuracy of the detected landmarks.
• Segmentation metrics assess the accuracy of the detected pharynx segmentation.

2.3.1. Keypoint Detection Metrics

Mean Radial Error (MRE)

The mean radial error (MRE) measures the average Euclidean distance (radial error) in millimeters between the predicted points and the ground truth points. The radial error (R) of a point detected on the input image is as follows:

$$R=\sqrt{{\left(x_p-x\right)}^2+{\left(y_p-y\right)}^2}$$

where

• x_p, y_p are the predicted coordinates;
• x, y are the ground truth coordinates.

The MRE, along with the (corrected) standard deviation (std) for each landmark, is defined as follows:

$$MRE={\displaystyle \frac{{\sum }_{I=1}^N{R}_i}{N}}$$

$$std=\sqrt{{\displaystyle \frac{{\sum }_{I=1}^N{\left(R_i-MRE\right)}^2}{N-1}}}$$

where

• N is the number of images;
• R_i is the radial error on the i-th image.

COCO-mAP for Keypoints

COCO (Common Objects in Context) measures for keypoint detection are widely used to evaluate pose and keypoint detection algorithms [35]. MS-COCO proposed Object Keypoint Similarity (OKS) as a measure of similarity between two sets of keypoints.

Mathematically, the keypoint similarity (KS_i) for keypoint i is given as

$$K{S}_i=\left({\displaystyle \frac{-d_i^2}{2s^2{k}_i^2}}\right)$$

where

• d_i is the Euclidean distance between the ground truth and predicted keypoint i;
• k is the constant for keypoint i; in our tests, we set k = 0.001;
• s is the scale of the ground truth object. s² hence becomes the object’s segmented area.

Each ground truth annotated keypoint should have a visibility flag, which can take one of three values:

• 0: unlabeled keypoint;
• 1: labeled but not visible keypoint;
• 2: labeled and visible keypoint.

The mathematical notation for OKS is given by

$$OKS={\displaystyle \frac{{\sum }_{i}K{S}_i\cdot \delta \left(v_i>0\right)}{{\sum }_i\delta \left(v_i>0\right)}}$$

where

• KS_i is the keypoint similarity for keypoint i;
• v_i is the ground truth visibility flag for keypoint i;
• $\delta \left(v_i>0\right)$ is the Dirac-delta function, which is computed as 1 if keypoint i is labeled, otherwise 0.

The primary COCO metric for keypoint detection is the mean Average Precision (mAP) for keypoints. This calculates the area under the precision-recall curve, averaged across different levels of OKS, which is a metric that evaluates the similarity between predicted keypoints and ground truth keypoints, normalized by object scale. COCO mAP for keypoints uses 10 OKS thresholds, ranging from 0.5 to 0.95 in steps of 0.05, providing a more detailed evaluation compared to the standard threshold of 0.5. A COCO mAP closer to 1.0 indicates better performance.

The COCO mAP for keypoints thus provides a comprehensive measure of performance in keypoints detection, considering not only accuracy but also the ability to detect keypoints across various similarity thresholds, object categories, and sizes.

Successful Detection Rate (SDR)

The SDR is a metric that measures the percentage of correctly identified points within a predefined error tolerance z. For each landmark, it is defined as

$$SD{R}_z={\displaystyle \frac{Number of correct identifications}{Number of total identifications}}\cdot 100$$

Here, we consider three error tolerances: z = 1 mm, 2 mm, 3 mm, and 4 mm.

2.3.2. Segmentation Metrics

Mean Intersection over Union (IoU)

Intersection over Union (IoU) measures the degree of overlap between the predicted object and the corresponding ground truth object. It is calculated as the ratio between the area shared by the prediction and the reference annotation and the total area covered by their union. The IoU is defined as follows:

$$IoU={\displaystyle \frac{Area of Overlap}{Area of Union}}$$

Higher IoU values indicate a greater overlap between the prediction and the ground truth.

The mean Intersection over Union (mIoU) calculates the average IoU across all categories. The mIoU is defined as follows:

$$mIoU={\displaystyle \frac{1}{C}}\sum _{i=1}^C{\displaystyle \frac{T{P}_i}{T{P}_i+F{P}_i+F{N}_i}}$$

where

• C is the number of categories.
• TP_i (true positive) indicates the pixels correctly assigned to class i, according to ground truth segmentation.
• FP_i (false positive) is the number of pixels incorrectly detected as belonging to category i, although they do not belong to that category in the ground truth segmentation.
• TN_i (true negative) is the number of pixels correctly detected as not belonging to class i, according to the ground truth segmentation.
• FN_i (false negative) is the number of pixels that belong to category i in the ground truth segmentation but were not detected by the system.

An IoU value of 1 indicates complete agreement between the predicted and ground truth segmentations. However, this value should be interpreted carefully when a category is absent from both the prediction and the ground truth, because this condition may also produce an IoU of 1 and therefore overestimate segmentation performance.

COCO mAP for Segmentation

COCO (Common Objects in Context) measures for segmentation are widely used to evaluate object detection and segmentation algorithms [35]. The key metrics include:

• mean Average Precision (mAP): it is the area under the precision-recall curve averaged across different Intersection over Union (IoU) thresholds and across all categories. COCO mAP uses 10 IoU thresholds (from 0.5 to 0.95 in steps of 0.05) to provide a more detailed evaluation compared to the standard IoU threshold of 0.5.

The COCO mAP provides a robust measure of how well a segmentation system performs, taking into account not just the accuracy of object detection but also the precision at different recall levels and across multiple object classes and sizes.

Mean Dice

The Dice coefficient, also known as the Dice Similarity Coefficient (DSC) or F1-score, is used to evaluate the agreement between a predicted segmentation and the corresponding ground truth segmentation. It ranges from 0 to 1, where values closer to 1 indicate higher agreement between the two segmentations.

The DSC is defined as

$$DSC\left(i\right)={\displaystyle \frac{2\cdot T{P}_i}{2\cdot T{P}_i+F{P}_i+F{N}_i}}$$

The Mean Dice Score (mDSC) is calculated as the average of the Dice scores obtained across all categories. If C represents the number of categories, the formula is:

$$mDSC={\displaystyle \frac{1}{C}}\sum _{i=1}^{C}DSC\left(i\right)$$

The mDSC provides an overall measure of segmentation performance across multiple categories. It reflects the balance between false-positive and false-negative pixels, with values closer to 1 indicating better segmentation quality.

Mean Precision

Mean precision (mPrecision) measures the proportion of pixels predicted as positive that actually belong to the corresponding category in the ground truth segmentation. It is defined as:

Mean Recall

Recall (mRecall, also known as sensitivity or true positive rate) measures how many of the actual positive pixels were correctly predicted. It focuses on the completeness of the predictions made by the model. The mRecall is defined as follows:

$$mRecall={\displaystyle \frac{1}{C}}\sum _{i=1}^C{\displaystyle \frac{T{P}_i}{T{P}_i+F{N}_i}}$$

3. Results

The model was evaluated on a dataset of 35 lateral cephalograms. The system’s average inference time was 2.557 ± 0.504 s. For the keypoint detection task, it achieved an MRE of 0.740 ± 0.793 mm. Additionally, the results show an SDR of 75.0%, 88.0%, 93.9%, and 96.5% for tolerance radii of 1, 2, 3, and 4 mm, respectively, indicating favorable performance within the present validation dataset. For pharynx segmentation, the mAP was 0.740 ± 0.040, and the mDSC was 0.935 ± 0.040, indicating favorable segmentation performance within the present validation dataset.

A detailed analysis of the keypoint detection and segmentation performance is presented in the subsequent subsections.

3.1. Keypoint Detection Results

Table 3. Evaluation results for keypoint detection from lateral cephalograms.

Metric	Mean	Std
MRE	0.74	0.79
mAP	0.79	0.09
SDR % 1 mm	74.99	26.55
SDR % 2 mm	87.99	16.93
SDR % 3 mm	93.91	10.25
SDR % 4 mm	96.48	7.09

presents the MRE, mAP, and SDR (for 1, 2, 3, and 4 mm accuracy ranges) for keypoint detection. For each metric, values were computed per landmark across all images in the respective subgroup and then averaged across landmarks. The results show that the system achieved an MRE of 0.740 ± 0.793 mm. Moreover, it reached an SDR of 75.0%, 88.0%, 93.9%, and 96.5% for tolerance radii of 1, 2, 3, and 4 mm, respectively.

To assess the potential impact of age and sex on system performance, evaluation metrics were computed separately for age and sex subgroups.

Table 4. Evaluation results for keypoint detection from lateral cephalograms grouped by sex.

Metric	Male		Female		Test	p-Value
	Mean	Std	Mean	Std
MRE	0.78	0.80	0.70	0.83	Mann–Whitney	0.418
mAP	0.78	0.08	0.81	0.10	Mann–Whitney	0.303
SDR % 1 mm	73.98	27.35	76.04	26.96	Mann–Whitney	0.838
SDR % 2 mm	87.18	17.88	88.83	17.71	Mann–Whitney	0.553
SDR % 3 mm	93.23	10.98	94.63	11.04	Mann–Whitney	0.261
SDR % 4 mm	95.81	7.94	97.19	7.03	Mann–Whitney	0.093

and

Table 5. Evaluation results for keypoint detection from lateral cephalograms grouped by age.

Metric	Age 6–12		Age 13–65		Test	p-Value
	Mean	Std	Mean	Std
MRE	0.72	0.87	0.75	0.77	Mann–Whitney	0.692
mAP	0.79	0.07	0.79	0.10	Mann–Whitney	0.859
SDR % 1 mm	75.57	28.63	74.60	26.00	Mann–Whitney	0.653
SDR % 2 mm	87.99	18.96	87.99	16.65	Mann–Whitney	0.498
SDR % 3 mm	94.10	12.31	93.79	9.98	Mann–Whitney	0.268
SDR % 4 mm	96.79	8.53	96.27	6.88	Mann–Whitney	0.079

show these results. Prior to group comparisons, normality was assessed using the Shapiro–Wilk test and homogeneity of variance using Levene’s test. Per-landmark MRE and SDR distributions were markedly non-normal in all subgroups (Shapiro–Wilk p < 0.05 for all); accordingly, Mann–Whitney U was applied for all these metrics in both the sex and age comparisons. For mAP, which is computed per image, normality was also violated in both groups; Mann–Whitney U was applied consistently throughout. The null hypothesis for each test was that there is no significant difference between the distributions of the respective subgroup performance metrics. The results indicate that there are no statistically significant performance differences between children (6–12 years) and individuals older than 13 years, or between males and females.

Additionally,

Table 6. Evaluation results for each keypoint detected from lateral cephalograms.

Points	Radial Error Mean	Radial Error Std	Metric SDR % 1 mm	Metric SDR % 2 mm	Metric SDR % 3 mm	Metric SDR % 4 mm
A	0.98	0.69	60.00	91.40	100.00	100.00
A’	0.20	0.37	94.30	100.00	100.00	100.00
Ar	1.57	1.39	40.00	74.30	88.60	91.40
B	0.91	0.68	54.30	97.10	100.00	100.00
Ba	2.52	1.90	25.70	45.70	65.70	74.30
Cv3sa	0.52	0.41	88.60	97.10	100.00	100.00
Cv3ia	0.85	3.29	94.30	94.30	94.30	94.30
Cl	0.14	0.26	97.10	100.00	100.00	100.00
Cm’	0.15	0.22	97.10	100.00	100.00	100.00
Cd	0.54	1.03	80.00	85.70	97.10	100.00
Cdl	0.49	0.97	80.00	91.40	94.30	100.00
Cdm	0.63	1.09	74.30	91.40	94.30	97.10
Cor	0.47	1.10	82.90	85.70	94.30	97.10
Ag	0.15	0.45	94.30	97.10	100.00	100.00
Ct	0.32	1.33	97.10	97.10	100.00	100.00
NC’	0.01	0.02	100.00	100.00	100.00	100.00
Fro	1.30	2.75	71.40	85.70	88.60	88.60
G’	0.12	0.33	94.30	100.00	100.00	100.00
Gn	0.71	0.36	77.10	100.00	100.00	100.00
Go	1.35	0.95	42.90	74.30	94.30	100.00
Hy	1.49	1.26	42.90	80.00	91.40	91.40
ii	0.73	0.48	68.60	100.00	100.00	100.00
is	0.70	0.42	71.40	100.00	100.00	100.00
Li’	0.05	0.10	100.00	100.00	100.00	100.00
Me’	0.14	0.55	100.00	100.00	100.00	100.00
Me	2.38	1.18	11.40	34.30	71.40	94.30
L6d	0.13	0.51	94.30	97.10	100.00	100.00
L6m	0.11	0.34	94.30	100.00	100.00	100.00
U6d	0.44	1.36	88.60	91.40	91.40	94.30
U6m	1.85	2.38	31.40	85.70	85.70	88.60
N	1.00	0.83	60.00	91.40	97.10	97.10
Rh	0.06	0.34	97.10	97.10	100.00	100.00
N’	0.22	0.47	94.30	100.00	100.00	100.00
Op	0.02	0.10	100.00	100.00	100.00	100.00
Or	1.82	1.81	34.30	68.60	85.70	91.40
SOr	1.60	1.15	37.10	62.90	85.70	97.10
U	1.52	1.27	31.40	82.90	94.30	94.30
Pog’	0.07	0.08	100.00	100.00	100.00	100.00
Pog	0.08	0.13	100.00	100.00	100.00	100.00
PM	0.10	0.28	97.10	100.00	100.00	100.00
Pn’	1.35	1.21	42.90	68.60	94.30	97.10
Po	1.94	1.46	31.40	45.70	88.60	94.30
Rcv	0.69	1.40	74.30	88.60	91.40	97.10
L4	1.02	1.70	65.70	80.00	91.40	94.30
U4	0.48	1.51	91.40	97.10	97.10	97.10
PSAS	3.36	2.63	17.10	40.00	51.40	62.90
Pt	0.37	0.75	82.90	94.30	97.10	100.00
Ptm_inf	0.12	0.34	94.30	100.00	100.00	100.00
R0	0.21	0.91	94.30	97.10	97.10	97.10
R1	0.08	0.49	97.10	97.10	100.00	100.00
R3	0.08	0.44	97.10	97.10	100.00	100.00
Rcc	2.24	2.77	54.30	54.30	62.90	71.40
LIA	0.03	0.20	97.10	100.00	100.00	100.00
UIA	0.09	0.40	94.30	97.10	100.00	100.00
S	0.49	0.38	85.70	100.00	100.00	100.00
Syp	0.11	0.48	97.10	97.10	100.00	100.00
B’	0.04	0.18	100.00	100.00	100.00	100.00
Subm’	0.15	0.55	94.30	97.10	100.00	100.00
Sn’	0.20	0.35	97.10	100.00	100.00	100.00
ANS	1.02	0.82	60.00	88.60	97.10	100.00
ANS1	0.43	0.99	82.90	88.60	94.30	100.00
PNS	2.68	2.48	37.10	48.60	65.70	77.10
Stoi’	0.09	0.26	97.10	100.00	100.00	100.00
Stos’	0.03	0.13	100.00	100.00	100.00	100.00
Sy	2.13	1.30	17.10	57.10	77.10	91.40
Sy2	1.74	1.41	28.60	68.60	77.10	94.30
Ls’	0.09	0.41	97.10	97.10	100.00	100.00
Ls-U1	0.12	0.38	94.30	100.00	100.00	100.00
V	1.25	1.18	57.10	77.10	88.60	94.30

provides the mean radial error for each detected landmark, offering a detailed breakdown of the system’s accuracy at specific points. The most challenging landmarks to detect are Ba, Me, PNS, PSAS, and Rcc, with MRE values exceeding 2 mm, suggesting that these landmarks require careful clinical review to ensure accurate measurements.

3.2. Pharynx Segmentation Results

Table 7. Evaluation results for pharynx segmentation from lateral cephalograms.

Metric	Mean	Std
mAP	0.74	0.04
mIoU	0.88	0.07
mDSC	0.94	0.04
mPrecision	0.95	0.03
mRecall	0.92	0.07

presents the results for pharynx segmentation, with a mAP of 0.740 ± 0.040 and a mDSC of 0.935 ± 0.040.

To assess potential biases due to age and sex, evaluation metrics were computed for age and sex subgroups.

Table 8. Evaluation results for pharynx segmentation from lateral cephalograms grouped by sex.

Metric	Male		Female		Test	p-Value
	Mean	Std	Mean	Std
mAP	0.75	0.03	0.73	0.04	Mann-Whitney	0.255
mIoU	0.89	0.06	0.87	0.08	t-test	0.359
mDSC	0.94	0.03	0.93	0.05	Mann-Whitney	0.541
mPrecision	0.94	0.02	0.95	0.04	Mann-Whitney	0.067
mRecall	0.94	0.05	0.91	0.08	Mann-Whitney	0.347

and

Table 9. Evaluation results for pharynx segmentation from lateral cephalograms grouped by age.

Metric	Age 6–12		Age 13–65		Test	p-Value
	Mean	Std	Mean	Std
mAP	0.73	0.04	0.75	0.03	Mann-Whitney	0.490
mIoU	0.89	0.07	0.87	0.07	Mann-Whitney	0.354
mDSC	0.94	0.04	0.93	0.04	Mann-Whitney	0.354
mPrecision	0.96	0.03	0.94	0.03	Mann-Whitney	0.066
mRecall	0.93	0.07	0.92	0.06	Mann-Whitney	0.449

show these results. Prior to group comparisons, normality was assessed using the Shapiro–Wilk test and homogeneity of variance using Levene’s test. For the sex comparison (Table 8), the mAP, mDSC, mPrecision, and mRecall distributions violated normality in at least one group; Mann–Whitney U was applied for those metrics. For mIoU, all assumptions were satisfied, and a standard independent t-test was used. For the age comparison (Table 9), normality was violated in at least one group for all metrics; Mann–Whitney U was applied throughout.

4. Discussion

The null hypothesis for each test was that there is no significant difference between the distributions of the respective subgroup performance metrics.

The results indicate that there are no statistically significant differences in the performance of cephalometric landmarks between children (6–12 years) and individuals older than 13 years, or between males and females. These findings suggest that the CEPH_2D system maintains stable performance across the examined demographic groups.

Overall, the present study showed that CEPH_2D achieved good performance in both cephalometric landmark detection and pharyngeal airway segmentation on lateral cephalograms. In relation to landmark detection, the results appear to be consistent with those reported in previous studies on AI-assisted cephalometric analysis. Although direct comparisons among studies should be interpreted with caution because of differences in datasets, annotation protocols, number and definition of landmarks, and evaluation methods, CEPH_2D demonstrated a level of accuracy that is in line with the current literature. Kunz et al. [10] evaluated a fully automated cephalometric analysis based on a customized convolutional neural network and reported that CNN-based approaches may provide reliable support for automated landmark identification. Park et al. [12] compared two deep learning object detection models, YOLOv3 and SSD, for automated cephalometric landmark detection, showing that YOLOv3 achieved better performance in terms of both accuracy and processing speed. This is relevant to the present study because CEPH_2D was also designed to combine diagnostic accuracy with a rapid processing time suitable for clinical workflow integration. Qian et al. [13] proposed CephANN, a multi-head attention neural network developed to improve the localization of cephalometric landmarks, highlighting the role of increasingly specialized neural network architectures in this field. Popova et al. [14] investigated the influence of growth-related anatomical structures and fixed orthodontic appliances on automated landmark recognition using a customized convolutional neural network. Their study is particularly relevant to the present validation because CEPH_2D was assessed in both mixed and permanent dentition, allowing the evaluation of system performance across different developmental stages. Taken together, these studies support the growing evidence that AI-based systems may provide useful support for cephalometric analysis, although clinician supervision remains necessary, especially for landmarks showing higher detection errors.

A similar consideration applies to pharyngeal airway segmentation. CEPH_2D showed high agreement with the reference annotations, suggesting that it may represent a useful support tool for the automated analysis of the nasopharynx, oropharynx, and hypopharynx on lateral cephalograms. This is particularly relevant in OSAS-related assessment, where the evaluation of upper airway morphology on lateral cephalograms may contribute to a more standardized radiographic analysis. Although two-dimensional imaging does not replace volumetric assessment when this is clinically indicated, the present findings suggest that CEPH_2D may have potential applicability as an adjunctive tool in OSAS-oriented airway evaluation.

The present study was conducted with a structured validation process, particularly in the construction of the testing dataset. The dataset was prepared with careful attention to detail, involving two experienced clinicians who fulfilled complementary roles as annotator and reviewer. This dual-review protocol helped minimize bias and improve the consistency of the ground truth data, which served as the benchmark for evaluating the system’s performance. Manual annotations, performed using CVAT, were systematically reviewed and cross-validated for adherence to established anatomical guidelines. Quality control checks addressed issues related to image resolution and segmentation consistency, helping to ensure that the dataset was comprehensive and reflective of real-world clinical scenarios. The performance validation was conducted on standard clinic-grade hardware, demonstrating its potential suitability for routine use without requiring specialized equipment. This suggests that CEPH_2D may be integrated into typical clinical workflows, supporting its potential practicality in everyday settings.

CEPH_2D demonstrated consistent accuracy across various demographic subgroups, with no statistically significant differences in performance observed based on age or sex. These findings suggest potential applicability across diverse patient profiles, including children with mixed dentition, a category that may be more challenging for automated systems. The computational efficiency of CEPH_2D may also support its integration into clinical workflows. With an average processing time of 2.557 ± 0.504 s, the system showed rapid performance compatible with routine clinical settings. Its compatibility with standard imaging hardware may further facilitate its use in diagnostic and treatment planning processes, saving time and reducing the clinician’s workload.

These results suggest that CEPH_2D may help address some limitations of manual and semi-automated methods, such as variability in accuracy and the time required for analysis. By providing automated support, CEPH_2D may contribute to the level of precision required for orthodontic and obstructive sleep apnea syndrome evaluations. Considering that AI is increasingly being implemented in dental protocols, in areas such as smartphone-based tools, tooth shade assessment, brushing monitoring, pediatric behavior support, orthognathic outcome prediction, and AI-enhanced CBCT image analysis for nano-orthodontic diagnosis and treatment [36,37,38,39,40,41,42,43,44], further research is warranted to better define and expand its clinical applications.

Some limitations should also be acknowledged. First, the present validation was performed on a relatively limited testing dataset of 35 lateral cephalograms from a single center, which limits statistical power and may reduce the generalizability of the findings to broader and more diverse clinical populations. Although the dataset included both sexes, a broad age range, mixed and permanent dentition, and different clinical conditions, external validation on larger, multicenter, and geographically diverse cohorts is required to confirm these results.

Second, certain landmarks, including Ba, Me, PNS, PSAS, and Rcc, exhibited MRE values above 2 mm, indicating that clinician verification remains advisable for these points before final clinical interpretation or treatment planning. These errors are probably related to anatomical complexity, landmark definition, image acquisition variability, model-related limitations, and the intrinsic limitations of two-dimensional projection imaging. Ba is located in an area affected by complex bony superimposition from the petrous portions of the temporal bones. Me may be influenced by individual variations in chin morphology. PNS is a small and variable bony projection at the posterior limit of the hard palate and may overlap with adjacent palatal and nasal structures, developing teeth, and palatal arch anatomy. PSAS is a soft-tissue landmark related to the soft palate and lacks a discrete osseous reference; its localization may therefore be affected by soft-tissue contrast variability and by the reproducibility of manual landmark definition. Rcc is located along the posterior border of the mandibular ramus and may be influenced by patient positioning during acquisition because misalignment can generate overlapping right and left mandibular ramus outlines on lateral cephalograms. These factors may explain the higher localization variability observed for these landmarks.

Third, formal inter-observer and intra-observer reliability metrics were not computed for the ground truth annotations. Although the annotation and review workflow was designed to improve consistency, future studies should quantify annotation reliability using dedicated agreement metrics. In addition, because this independent validation was conducted without access to the original training dataset or full implementation details, the size, source, diversity, and annotation process of the training data could not be evaluated.

Fourth, although all radiographs were acquired using the same imaging platform under routine clinical conditions, minor variability in acquisition conditions and image characteristics representative of routine clinical workflows was present across the dataset. Detailed exposure parameters, image resolution, and calibration information were not available for all radiographs; however, this reflects the retrospective and real-world nature of the validation dataset. Future studies should include more standardized acquisition metadata to further assess the influence of imaging parameters on AI performance.

Fifth, although the age stratification was based on dental development stage, the permanent dentition group included a broad age range from 13 to 65 years. This may have masked age-related anatomical or imaging differences among adolescents, young adults, and older adults. Future studies with larger samples should consider more detailed age stratification.

Finally, because the analysis was based on two-dimensional lateral cephalograms, the results should be interpreted within the intrinsic limitations of 2D imaging, especially for OSAS-oriented airway assessment.

Further studies are warranted to validate these findings in broader clinical settings and to better define the potential applications of CEPH_2D in routine practice.

5. Conclusions

Within the limitations of the present study, CEPH_2D showed promising performance in automatic cephalometric landmark detection and pharyngeal airway segmentation on lateral cephalograms. The system demonstrated rapid processing time and stable performance across the examined age and sex subgroups, suggesting potential usefulness as an adjunctive tool in orthodontic and obstructive sleep apnea syndrome-related radiographic assessment. However, clinician verification remains necessary before automated outputs are used for clinical interpretation or treatment planning, particularly for landmarks showing higher detection errors. Larger, multicenter, externally validated studies are required to confirm these findings and to compare CEPH_2D with other available AI-based systems.