Enhancing Vertical Trajectory Reconstruction in SASS-C: Advanced Segmentation, Outlier Detection, and Filtering Techniques

Abstract

This paper presents significant enhancements to the vertical reconstruction component of EUROCONTROL’s Surveillance Analysis Support System for ATC Centres (SASS-C). We introduce four key improvements: (1) a novel segmentation algorithm for more precise flight phase identification, (2) an improved invalid height detection process using LOWESS and sliding window analysis, (3) a protection mechanism against simultaneous measurements at the Kalman filter level, and (4) an optimized approach for smooth overshoot correction during segment transitions. These advancements address limitations in the current system, particularly in trajectory segmentation accuracy and robustness against measurement anomalies. Our methodology employs both synthetic and real-world data for comprehensive evaluation, ensuring performance under controlled and operational conditions. The results demonstrate substantial improvements in segmentation precision, outlier detection, and overall trajectory reconstruction quality. The invalid detection algorithm, while incurring a slight computational cost, significantly enhances trajectory accuracy. These enhancements contribute to more reliable air traffic analysis, supporting safer and more efficient airspace management. The paper concludes by discussing potential future work, including the application of machine learning techniques and the extension of these improvements to horizontal reconstruction processes.

1. Introduction

In the modern era of aviation, the increasing density and complexity of air traffic pose significant challenges to Air Traffic Control and Management (ATC/ATM) systems. The demand for enhanced safety, efficiency, and operational optimization necessitates increasingly sophisticated surveillance and data analysis capabilities [1]. Within this context, accurate trajectory reconstruction plays a pivotal role in evaluating and improving air surveillance infrastructure.

EUROCONTROL, the European Organisation for the Safety of Air Navigation, plays a crucial role. As an intergovernmental organization comprising 41 Member States, EUROCONTROL not only establishes industry standards but also develops essential tools to support service providers in meeting stringent quality requirements [2]. Its mission encompasses harmonizing and integrating air navigation services throughout Europe, aiming to create a Single European Sky that can handle the continuous growth in air traffic while maintaining high safety standards, reducing costs, and respecting the environment.

A key component of EUROCONTROL’s toolkit is the Surveillance Analysis Support System for ATC Centres (SASS-C), a comprehensive offline software suite designed for evaluating and monitoring air surveillance infrastructure performance [3]. SASS-C is utilized by over 100 users both within and beyond EUROCONTROL Member States, including civil and military air navigation service providers (ANSPs), R&D organizations, and the ATM industry.

SASS-C provides two primary functions: Surveillance Infrastructure Performance prediction (PREDICTion) and Surveillance Infrastructure Performance Verification (VERIFication). At the core of SASS-C is the Opportunity Traffic Reconstructor (OTR), an offline trajectory reconstruction system that fuses data from multiple sensors to create an accurate picture of air traffic. This system deals with a complex array of data sources, including both cooperative and non-cooperative surveillance methods, each with its own characteristics and challenges.

Current Limitations and Proposed Improvements

Despite the sophistication of the SASS-C system, the current vertical segmentation component, developed in the early 2000s [1,2,3,4,5], presents certain limitations that affect its accuracy and usefulness. These limitations have become more apparent as both users’ and standards’ requirements have become more demanding.

• Lack of detailed maneuver information: Users have consistently reported that the temporal segmentation produced by the current system does not accurately reflect real flight modes. The inability to provide detailed information about different types of vertical maneuvers severely limits its utility for advanced flight analysis, as well as may potentially impact safety assessments and operational efficiency analyses.
• Susceptibility to data quality issues: Users have noted that the reconstruction results are not as accurate as they should in an offline tool, and there is a tendency of this due to bad measurements in raw data. This vulnerability leads to unreliable trajectory reconstructions, compromising the integrity of subsequent analyses and potentially leading to incorrect decisions in air traffic management.
• Evolving air traffic patterns: The air traffic landscape has changed significantly since the early 2000s, with increased traffic density, new flight patterns, and more diverse aircraft types. The current system’s limitations in adapting to these changes have become increasingly problematic.
• Advancing technology: Advancements in data processing capabilities and algorithmic approaches since the system’s initial development offer opportunities for significant improvements that were not previously possible.

These critical issues, highlighted through extensive user feedback and systematic analysis of the system’s performance, underscore the urgent need for a comprehensive study to address these problems. This paper presents the culmination of that study, offering a series of targeted improvements to the vertical component of the SASS-C system. Our contributions include:

• A novel segmentation algorithm for more precise flight phase identification and an additional mode of flight detection.
• An improved invalid height detection process for enhanced reconstruction accuracy.
• A protection mechanism against simultaneous measurements at the Kalman filter level.
• An optimized approach for combining forward and backward filter passes during segment transitions, without using the measure and with improved smoothing.

By addressing these aspects, we aim to provide a more robust and informative vertical trajectory reconstruction system that better serves both the filtering system and operators analyzing the data. These improvements are expected to yield benefits in two primary ways: providing the system with greater knowledge to refine its filtering results and offering operators more precise information for comparative analysis.

The remainder of this paper is organized as follows:

In Section 2, related works on the topics this article studies are analyzed. Section 3 provides a detailed overview of EUROCONTROL’s OTR system, with a particular focus on the vertical segmentation component. Section 4 details the proposed improvements and their implementation, providing in-depth explanations of the novel algorithms and techniques developed to address the identified limitations. Section 5 outlines the evaluation methodology, for the results analysis carried out, offering a comprehensive analysis of the system’s performance under various conditions. Finally, Section 6 concludes the paper with a summary of findings, discussing the implications of this work for air traffic management and surveillance systems, and outlining future research directions.

2. Related Works

The field of avionics and trajectory analysis has seen a shift in focus over recent years. The accessibility of this domain and its data make it complex, as does the low number of open-source tools (with few exceptions, such as [6,7]). While data mining, prediction [8,9], and planning tasks are currently trending, tracking, fusion systems, and flight trajectory reconstruction seem to have passed their peak of research hype. However, after decades of research in tracking and surveillance infrastructure assessment [10], there is still work to be conducted [8] as new technologies emerge. Moreover, other fields of study, such as maritime data reconstruction with AIS, offer potentially useful insights due to their high similarities with aviation trajectory reconstruction.

Recent trends in trajectory analysis show an increasing interest in deep learning approaches [11]. Convolutional networks for trajectories reconstruction [12] and LTSMs [13] in forward and backward directions are being explored [14], and Large Language Models (LLMs) are being explored for time series analysis [15] and reconstruction [16], potentially paving the way for future advancements in this field. Using transformers for filtering [17], or real-time reconstruction, particularly for handling outliers, has been investigated using simulated data [18]. These emerging techniques aim to address challenges such as outlier removal, spatiotemporal filling, and the segmentation of landing and take-off phases [12].

The improvements presented in this paper touch on several key areas of air traffic management research. We will review related works in four main categories: invalid measurement detection, simultaneous measurement handling, trajectory segmentation, and smooth overshoot correction. Each subsection will highlight current approaches and identify where our work advances the state-of-the-art.

2.1. Invalid/Outlier Measurement Detection

The detection of invalid values and outliers is crucial in trajectory reconstruction, particularly when dealing with raw data. Studies have shown that received data can be highly erroneous depending on how it is transmitted [19,20], making it essential to expect and handle erroneous data effectively.

In the vertical plane specifically, working with data is particularly challenging due to the high quantity of noisy measurements [21]. There is a high probability of receiving noise in the measures transmitted from the aircraft [22]. Some researchers have attempted to address this issue by implementing a priori outlier detection in ADS-B using a sliding window approach [23]. However, this method has proven insufficient in certain cases, necessitating the development of more robust solutions.

Multiple solutions are already available in the open-source community [24], proven for various types of trajectories. In the maritime domain, specifically, AIS data error detection and handling have been extensively studied [25], providing valuable insights that could be applied to aviation data. These techniques focus on analyzing raw data to identify and handle invalid measurements before the trajectory reconstruction process.

2.2. Same-Time Measurements Handling

Handling simultaneous measurements from multiple sensors is crucial in our trajectory reconstruction work. While not identical to out of sequence measurements (OOSMs), our problem shares similarities and can benefit from related solutions. OOSMs occur when measurements arrive at a tracker with time delays, often due to sensor processing time, communication delays, or differing update rates [26].

The literature on OOSMs proposes three main handling techniques: neglect, reprocessing, and retrodiction [27,28]. These methods, while not directly applicable to our case, offer insights into managing complex measurement scenarios.

In our specific situation, we observe that some sensors periodically repeat values while others provide updated measurements simultaneously. This creates a unique challenge where we have multiple measurements at the same timestamp, but with potentially different levels of relevance.

2.3. Trajectory Segmentation

Trajectory segmentation, particularly in the context of flight phase identification (FPI), has been a focal point of numerous recent studies. Unlike other vehicle types, aircraft operate within a limited and initially guided operational space, making their trajectory segmentation a unique challenge.

Recent approaches to trajectory segmentation have explored various techniques, including fuzzy logic and clustering [29], deep learning approaches for ADS-B data [18] and hidden Markov models (HMM) for flight phase identification [30]. The latter is particularly noteworthy, as it bears similarities to the interacting multiple model (IMM) used in horizontal component analysis.

It is important to note that trajectory segmentation serves purposes beyond mere phase identification [12,31,32,33,34]. Segmentation can be context-free or context-aware and can also be utilized for trajectory compression by identifying the most relevant points in a trajectory.

In our previous work, we conducted a comprehensive review of trajectory segmentation algorithms [35] and applied these concepts in the maritime context [32]. These experiences offer valuable insights that can be applied to the aviation domain. Techniques from other contexts such as maritime tracking, and even drone tracking, can provide useful perspectives for solving aviation-specific challenges [35].

2.4. Smooth Overshoot

In trajectory reconstruction, particularly for vertical rate estimation in complex environments, the combination of forward and backward Kalman filters presents unique challenges. Our approach involves running separate filters forward and backward in time, then combining their outputs for a more accurate trajectory estimate. However, this combination can lead to significant overshoots during velocity transitions, especially in scenarios with rapidly changing altitudes and noisy measurements. These overshoots are particularly problematic in our specific application, where precise vertical positioning is crucial for safety and operational efficiency.

While the literature on Kalman filtering and smoothing is extensive, the specific problem of managing overshoots in combined forward-backward filters for trajectory reconstruction with varying velocities in mountainous terrains is less explored. Traditional smoothing techniques, such as those discussed by Bar-Shalom et al. [36], often assume more stable environments or focus on different aspects of filter combination. Similarly, classic works like [37] on optimal linear smoothers provide valuable insights, but do not directly address the unique challenges posed by our specific scenario of rapid altitude changes and sensor limitations in mountainous environments. Our research aims to address this gap by developing a tailored solution that maintains the benefits of both forward and backward filtering while mitigating the overshoot problem in these challenging conditions.

In conclusion, while traditional methods of trajectory reconstruction and segmentation continue to be relevant, the field is seeing a shift towards more advanced techniques, particularly those leveraging machine learning and deep learning approaches. These new methods show promise in addressing longstanding challenges such as handling noisy data, identifying complex flight phases, and dealing with measurement inconsistencies. However, there remains a need for robust, real-time solutions that can handle the complexities of multi-sensor fusion and adapt to the unique characteristics of aircraft trajectories.

3. System Overview

The Surveillance Analysis Support System for ATC Centre (SASS-C) is a comprehensive software suite designed by EUROCONTROL for the evaluation and monitoring of air surveillance infrastructure performance. As illustrated in Figure 1, SASS-C comprises two primary components: Surveillance Infrastructure Performance Prediction (PREDICT), shown in green, and Surveillance Infrastructure Performance Verification (VERIF), shown in blue.

VERIF is responsible for utilizing information from the surveillance infrastructure to recreate the functioning of each surveillance and tracking system used by service providers. Its goal is to obtain the most accurate tracking of each aircraft at specific times, but unlike real-time systems, VERIF can exploit all trajectory measurements simultaneously, resulting in a more accurate reference track. VERIF also extracts metrics such as sensor biases, systematic error estimation, and latency.

Using those reconstructed trajectories, CoMParator (CMP) analyses reconstructed trajectories with each raw sensor datum and generates multiple performance reports to assess as standards rule each piece of the monitoring infrastructure.

All this system is developed with a central database to store all the system information and data collected and generated. There is also a GUI to execute all actions and configure all available parameters, as well as a GIS data visualization tool.

3.1. Opportunity Traffic Reconstructor (OTR)

At the core of SASS-C’s VERIF component lies the Opportunity Traffic Reconstructor (OTR), responsible for the offline reconstruction of aircraft trajectories. Figure 2 illustrates the detailed structure of the OTR, and its associated performance report components executed by the CoMParator (CMP).

The OTR consists of two main stages:

• Association: This process aims to unite all spatio-temporal measurements from multiple sensors, either collaborative (e.g., ADS-B) or not (e.g., PSR), into a trajectory performed by the same aircraft over time. This computationally intensive procedure uses Kalman filtering at various stages and precision levels (i.e., gross association, fine association). During execution, noise and bias correction for each sensor is measured and cleaned to better associate noisy data. As output, it obtains multi-sensor tracks, and each one of them will enter the reconstruction stage individually.
• Reconstruction: This stage comprises horizontal and vertical components, as well as discrete codes reconstruction. Both horizontal and vertical reconstruction are handled separately at this stage, as it is common in the literature [21]. This separation allows for specialized algorithms tailored to the unique characteristics of horizontal and vertical movement. It uses tracking filters along with a backward-operating smoother, alongside a final combination of both filter passes (forward and backward) to produce a precise estimation over time, removing biases from the filter and noise from the measurements. This is a well-known and proven solution to obtain the best possible trajectory without noise [38].

The horizontal component uses IMM (interacting multiple model) filtering with specific functionalities adapted to this problem, while the vertical filter is an adaptive Kalman filter (detailed later). The code reconstruction fills in and recovers information sent by sensors in Mode A, Mode 1, Mode 2, SPI, Emergency Status, and Aircraft Identification. In the end, a post-processing stage is applied to handles specific cases and needs, such as the reconnection and filling in on the same trajectory if it was split into two; flight classification, to analyze the trajectory dynamics to prepare for the CoMParator’s work; or an association and enrichment with external tracks to enhance the reconstruction results, if available.

3.2. Vertical Component

The vertical component of the OTR system employs a sophisticated processing flow to accurately reconstruct aircraft altitude profiles by using barometric and geometric measurements. It is composed of four steps, as visualized in Figure 3:

• Invalid height detection: The process begins with an initial analysis of height measurements to identify and mark those invalid data points coming from Mode C. This step is crucial for maintaining the integrity of subsequent processing stages, especially the filtering. Its functioning is explained in detail on this same section. Non-valid height measurements are marked as noisy measurements to prevent using them in the vertical filtering process (next step).
• Forward and backward filtering: After that, the forward and backward vertical filtering steps are executed. They are completely independent of one another. Two accelerated adaptive Kalman tracking filter (AKF) passes are performed over the valid height measurements of the multitrack. Each vertical measurement of the multitrack (composed of the height, and if provided and aligned with the filter, also the vertical rate) is received as input. As output, it produces the estimated and smoothed height, vertical rate, and vertical acceleration.
• Segmentation in modes offFlight: Using all available information of the track, the next process focuses on detecting the different maneuvers that were performed in the vertical plane. It first analyses the kinematic to detect the transitions between those maneuvers and then classifies the gaps in between to assign a vertical mode of flight mode (VMoF). The kinematic parameters of the segment are calculated as additional information.
• Combination: The last step is to combine back both FW and BW vertical data, creating the final and merged vertical data after the whole vertical reconstruction process, composed of height, vertical rate, and associated variances. This combination is based on a logic dependent on the variances and availability of FW and BW. This combination is based on a logic dependent on the variances and availability of FW and BW. Also, it takes advantage of the generated segments to reduce the smoothing effects produced by the filters in the transitions, thus obtaining a realistic transition.

Although all of them are of such relevance, the vertical filtering is the central piece. It employs an adaptive Kalman filter (AKF) [39] to estimate the aircraft’s vertical state. Represented with $\widehat{\mathrm{x}}\left[\mathrm{k}\right]$, the estimated state vector at time $k$, is composed of the estimated height $\widehat{\mathrm{h}}$, vertical rate ${\mathrm{v}}_{\mathrm{r}}$, and vertical acceleration ${\mathrm{v}}_{\mathrm{a}}$. It goes along $\mathrm{P}\left[\mathrm{k}\right]$, the covariance matrix associated with the state vector at time $k$.

$$[\mathrm{k}]=\left(\begin{array}{ccc}{\mathsf{\sigma }}_{\mathrm{h}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{v}\mathrm{h}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{a}\mathrm{h}}^2\left[\mathrm{k}\right]\\ {\mathsf{\sigma }}_{\mathrm{v}\mathrm{h}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{v}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{v}\mathrm{a}}^2\left[\mathrm{k}\right]\\ {\mathsf{\sigma }}_{\mathrm{a}\mathrm{h}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{v}\mathrm{a}}^2\left[\mathrm{k}\right]& {\mathsf{\sigma }}_{\mathrm{a}}^2\left[\mathrm{k}\right]\end{array}\right)$$

(1)

The input of the filter $\mathrm{Z}\left[\mathrm{k}\right]$ can include either height alone or height and vertical rate:

$$\mathrm{Z}\left[\mathrm{k}\right]=\left(\mathrm{h}\right)\parallel \left(\begin{array}{c}\mathrm{h}\\ {\mathrm{v}}_{\mathrm{r}}\end{array}\right)$$

(2)

The initialization of the filter is performed by the use of the same AKF set in reverse direction to the main filter. It performs 60 s run to have enough time to reach a stable state at the main filter initialization point. This way, it will start with a stable filter from the beginning of the trajectory, both in state and variances.

In the case that there is not enough time to initialize and stabilize, the initialization of the main filter would be performed in the traditional way, by using the two first registers with available height, so the initial values of the vertical rate and variances are provided.

In the prediction phase, the filter estimates the next measurement based on the time gap. It generates the plant–noise covariance matrix Q and the transition matrix F. The F matrix depends only on the time gap between measurements:

$$\mathrm{F}=\left(\begin{array}{ccc}1& ∆\mathrm{t}& {\displaystyle \frac{{\left(∆\mathrm{t}\right)}^2}{2}}\\ 0& 1& ∆\mathrm{t}\\ 0& 0& 1\end{array}\right)$$

(3)

The Q matrix depends on the acceleration variance ${\sigma }_a^2$ and an adaptation factor, which is calculated by the following graph between the residual obtained by the filter prediction and the measurement. With this curve, shown in Figure 4, the filter maintains logical variance values when it adequately follows the measurements, but when it does not, it increases its uncertainty to follow the measurement more closely. This allows it to behave smoothly in level flight, but when transitions happen, to be more reactive.

$$\begin{array}{l}\mathrm{Q}=\mathrm{F}\mathrm{A}\mathrm{C}\mathrm{T}\mathrm{O}\mathrm{R}·{\displaystyle \frac{2·{\mathsf{\sigma }}_{\mathrm{a}}^2}{{\left(∆\mathrm{t}\right)}^2}}·{\left(∆\mathrm{t}\right)}^2·\left(\begin{array}{ccc}{\displaystyle \frac{{\left(∆\mathrm{t}\right)}^4}{36}}& {\displaystyle \frac{{\left(∆\mathrm{t}\right)}^3}{12}}& {\displaystyle \frac{{\left(∆\mathrm{t}\right)}^4}{6}}\\ 0& {\displaystyle \frac{{\left(∆\mathrm{t}\right)}^4}{4}}& {\displaystyle \frac{{\left(∆\mathrm{t}\right)}^3}{2}}\\ 0& 0& 1\end{array}\right)\\ =\mathrm{F}\mathrm{A}\mathrm{C}\mathrm{T}\mathrm{O}\mathrm{R}·{\mathsf{\sigma }}_{\mathrm{a}}^2·\left(\begin{array}{ccc}{\displaystyle \frac{{(∆\mathrm{t})}^4}{18}}& {\displaystyle \frac{{(∆\mathrm{t})}^3}{6}}& {\displaystyle \frac{{(∆\mathrm{t})}^4}{3}}\\ 0& {\displaystyle \frac{{(∆\mathrm{t})}^4}{2}}& {(∆\mathrm{t})}^3\\ 0& 0& {\displaystyle \frac{1}{2}}\end{array}\right)\end{array}$$

(4)

The filter then predicts the state vector and covariances:

$${\widehat{\mathrm{X}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]={\mathrm{F}}_{\mathrm{j}}\left(\mathsf{\Delta }\mathrm{t}\right)·{\widehat{\mathrm{x}}}_{0\mathrm{j}}\left[\mathrm{k}-1\right]$$

(5)

$${\mathrm{P}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]={\mathrm{F}}_{\mathrm{j}}\left(\mathsf{\Delta }\mathrm{t}\right)·{\mathrm{P}}_{0\mathrm{j}}\left[\mathrm{k}-1\right]·{{\mathrm{F}}_{\mathrm{j}}\left(\mathsf{\Delta }\mathrm{t}\right)}^{\mathrm{t}}+{\mathrm{Q}}_{\mathrm{j}}\left(\mathsf{\Delta }\mathrm{t}\right)$$

(6)

In the update step, the measurement is used to correct the prediction previously made, updating both the state vector prediction and its associated noise.

If the measurement was detected as invalid, the update phase is not performed, giving as output the calculated prediction.

Due the adaptability of the filter to several inputs $\mathrm{z}\left[\mathrm{k}\right]$ (only height or both height and vertical rate), the projection matrix H is used to correctly match the measurement with the state vector.

$$\mathrm{H}=\left(\begin{array}{ccc}1& 0& 0\end{array}\right)\parallel \left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\end{array}\right)$$

(7)

The filter performs the following equations:

$$\begin{array}{l}{\mathrm{S}}_{\mathrm{j}}\left[\mathrm{k}\right]=\mathrm{R}\left[\mathrm{k}\right]+\mathrm{H}·{\mathrm{P}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]·{\mathrm{H}}^{\mathrm{t}}\\ {\mathrm{W}}_{\mathrm{j}}\left[\mathrm{k}\right]={\mathrm{P}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]·{\mathrm{H}}^{\mathrm{t}}·{\left(\mathrm{R}\left[\mathrm{k}\right]+\mathrm{H}·{\mathrm{P}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]·{\mathrm{H}}^{\mathrm{t}}\right)}^{-1}\\ {\widehat{\mathrm{x}}}_{\mathrm{j}}\left[\mathrm{k}\right]={\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]+{\mathrm{W}}_{\mathrm{j}}\left[\mathrm{k}\right]·\left(\mathrm{z}\right[\mathrm{k}]-\mathrm{H}·{\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}[\mathrm{k}\left]\right)\\ {\mathrm{P}}_{\mathrm{j}}\left[\mathrm{k}\right]={\mathrm{P}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]·(\mathrm{I}-{\mathrm{H}}^{\mathrm{t}}·{\mathrm{W}}_{\mathrm{j}}[\mathrm{k}\left]\right)\end{array}$$

(8)

3.2.1. Vertical Rate Smoothing

To obtain a smoother speed for reconstruction, in parallel to the AKF, an alpha filter [40] is applied to the AKF-filtered vertical rate. This procedure performs the alpha parameter calculation $\mathsf{\alpha }\left[\mathrm{k}\right]$, which adjusts how much the prediction is affected by the measurement (innovation). The following equations explain the behavior:

$$\begin{array}{l}\widehat{\mathrm{X}}\left[\mathrm{k}\right]=\widehat{\mathrm{x}}\left[\mathrm{k}-1\right]+\mathsf{\alpha }\left[\mathrm{k}\right]·\left(\mathrm{z}\right[\mathrm{k}]-\widehat{\mathrm{x}}[\mathrm{k}-1\left]\right)\\ \mathsf{\alpha }\left[\mathrm{k}\right]={\mathsf{\alpha }}_{\mathrm{g}\mathrm{a}\mathrm{i}\mathrm{n}}·(1-{\mathrm{n}}_{\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}})\\ {\mathrm{n}}_{\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}}={\mathrm{n}}_{\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}}+{\mathrm{n}}_{\mathrm{g}\mathrm{a}\mathrm{i}\mathrm{n}}·\left(1-{\displaystyle \frac{\mathsf{\Delta }\mathrm{t}}{\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{T}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{D}\mathrm{i}\mathrm{f}\mathrm{f}}}\right)·({\displaystyle \frac{\mathrm{n}\left[\mathrm{k}\right]}{\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{N}\mathrm{o}\mathrm{i}\mathrm{s}\mathrm{e}}}-{\mathrm{n}}_{\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}})\end{array}$$

(9)

Note that $\widehat{\mathrm{x}}\left[\mathrm{k}\right]$ is the final velocity prediction generated by this alpha filter. For its calculation, it uses the observation $\mathrm{z}\left[\mathrm{k}\right]$ and the prediction in the previous target report of this alpha filter, $\widehat{\mathrm{x}}[\mathrm{k}-1]$. $\mathrm{n}\left[\mathrm{k}\right]$ is the normalized variance predicted by the AKF filter.

${\mathsf{\alpha }}_{\mathrm{g}\mathrm{a}\mathrm{i}\mathrm{n}}$ is the innovation control, ${\mathrm{n}}_{\mathrm{g}\mathrm{a}\mathrm{i}\mathrm{n}}$ regulates the gain of the AKF filtered noise $\mathrm{n}\left[\mathrm{k}\right]$. $\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{T}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{D}\mathrm{i}\mathrm{f}\mathrm{f}$ sets the maximum allowable time gap, and $\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{N}\mathrm{o}\mathrm{i}\mathrm{s}\mathrm{e}$ sets the maximum noise allowed as an innovation. All those are set or initialized by parameters.

3.2.2. Outlier Detection

The filter includes outlier protection to enhance accuracy. As with adaptation, it calculates a residual of the prediction with respect to the sensor measurements and compares it with set thresholds ($\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}\mathrm{O}\mathrm{u}\mathrm{t}\mathrm{l}\mathrm{i}\mathrm{e}\mathrm{r}$ for normalized residual and $\mathrm{d}\mathrm{i}\mathrm{f}\mathrm{f}\mathrm{O}\mathrm{u}\mathrm{t}\mathrm{l}\mathrm{i}\mathrm{e}\mathrm{r}\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}$ for height difference). If the residuals exceed these thresholds, the register is flagged as an outlier. Two consecutive outliers trigger the filter reinitialization. That is why vertical invalid measurements detection is crucial.

$$\left|{\widehat{\mathrm{X}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]-\mathrm{z}[\mathrm{k}]\right|>\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}\mathrm{O}\mathrm{u}\mathrm{t}\mathrm{l}\mathrm{i}\mathrm{e}\mathrm{r}$$

(10)

$$〈\left(\mathrm{z}\left[\mathrm{k}\right]-\mathrm{H}·{\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]\right),\left({{\mathrm{S}}_{\mathrm{j}}[\mathrm{k}]}^{-1}\right(\mathrm{z}[\mathrm{k}]-\mathrm{H}·{\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}[\mathrm{k}]\left)\right)〉>\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}\mathrm{O}\mathrm{u}\mathrm{t}\mathrm{l}\mathrm{i}\mathrm{e}\mathrm{r}$$

(11)

3.2.3. Combination

Firstly, the data from the forward filter is combined with the data from the backward filter into a single value.

The output from the vertical component has to be only one trajectory, but we have calculated two of them. This component is going to produce the final output by combining the FW and BW values based on the inverse of their variances to generate each value.

This can be seen in the theoretical example of Figure 5, where both FW and BW filters produce overshoots differing to the ground truth, where the error increases. As each filter is more accurate on each side of a transitions, the combination mitigates those effect by giving more weight to the ones with smaller variances.

The combination does not completely solve the overshoots, but it reduces the height.

First, it computes the variance of combination result, $P_{Comb}$, as the inverse of the sum of both inverted variances:

$$P_{Comb}={\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\left(\frac{1}{P_{FW}}+\frac{1}{P_{BW}}\right)$}\right.}$$

(12)

This variance is used to calculate the combined value ($X_{Comb}$ which can be height or vertical rate). This is performed by summing the FW and BW samples divided by the respective variances, and multiplying the sum by the combined variance:

$$X_{Comb}=P_{Comb}\cdot ({\displaystyle \frac{X_{FW}}{P_{FW}}}+{\displaystyle \frac{X_{BW}}{P_{BW}}})$$

(13)

4. Issues and Improvements

Despite the sophistication of the SASS-C system, the current vertical reconstruction component presents certain limitations that affect its accuracy and usefulness. This section details these limitations and introduces our proposed improvements to address them. Specifically, the modifications are described in Figure 6: in blue, they remain the same; in green, they have been completely redesigned; in purple, the component is completely new; and in orange, a small change is added in the existing component.

Before going into the specific improvements, it is crucial to acknowledge the scope and limitations of this study. Firstly, our focus is primarily on the vertical component of trajectory reconstruction, although it is expected that the same algorithms will be able applicable on the horizontal component too. The following proposed improvements are designed to work within the existing SASS-C framework, an offline tool with over 40 years of industry presence. This context allows for solutions that can observe future data points and perform initial analyses of the entire dataset, potentially at a higher computational cost than online solutions. However, it may limit the direct applicability of our improvements to other air traffic management systems, especially those working in real-time.

4.1. Invalid Measurement Detection

The existing invalid height detection process operates by examining measurements in a forward mode, using a sliding window of triplets with height values. Figure 7 provides a theoretical example of this process, illustrating how the sliding window moves through the trajectory data and how different scenarios are handled. Due to its own behavior, invalidating only the middle value of the triplet, it may skip values, such as measurement 7 in the example.

This approach, while functional in some cases, has clear limitations to accurately identifying all invalid measurements and sometimes incorrectly flags valid ones as invalid. It has limited context, due to comparing only three consecutive data points, missing broader patterns or trends in the trajectory. It has rigid theorical thresholds, which are not optimal for all flight phases or aircraft types. In addition, its inability to evaluate the first and last measurements of a multitrack can lead to the retention of erroneous data, potentially skewing trajectory reconstruction, especially for initialization, a very delicate task.

These limitations can result in the inclusion of invalid data in the reconstruction process or the exclusion of valid measurements, both of which can significantly impact the accuracy of the final trajectory reconstruction.

To address these issues, we propose a new implementation that completely redesigns the approach for a more powerful analysis. Our improved method aims to solve the weaknesses of the previous implementation and expand the analysis to multiple measurements, leveraging the wider scope of data to more accurately identify invalid measurements.

The main idea of this improvement design is to expand the analysis to multiple measurements, instead of relying only on a simple logic of three measurements. That way, we are able to consider the wider scope of data, thus having a clearer base of measurements where the invalid ones will be more noticeable.

The proposal, illustrated in Figure 8 and described in Algorithm 1, is to use a window of several seconds that considers N registers and applies the locally weighted scatterplot smoothing (LOWESS) algorithm [41]. It is a non-parametric regression method that fits a smooth curve to data points. For each point x, it performs a weighted least squares regression on nearby points, with weights decreasing as distance from x increases. The basic equation for LOWESS is:

$$f(x)={\displaystyle \frac{\sum w\left(x\right)\ast y}{\sum w\left(x\right)}}$$

(14)

where $f\left(x\right)$ is the estimated value at $x$, $y$ are the nearby y values, and $w\left(x\right)$ are the weights, typically calculated using a tri-cubic function: $w\left(x\right)=(1-|\mathrm{d}\left|3\right)3$, where d is the scaled distance.

Algorithm 1. Invalid measurement detection

Using this smooth curve as reference, we calculate the difference between each measurement and its estimated position on the smoothed height curve. If it exceeds the designed thresholds (standard deviation error and height error itself), it is potentially an invalid measurement.

To ensure robustness, a sliding window is applied across the entire trajectory, performing LOWESS on subsets of measurements. This process allows for multiple tests on each register, providing a numerically supported decision on whether a register is consistently detected as invalid.

However, although LOWESS is already protected against outliers, we cannot assume after only one check that it is really an outlier, as the invalid measurements also affect the other analyses. It has to be something more robust than only one check. Therefore, the proposal is to perform a sliding window all over the trajectory, which shortens the whole trajectory only in a subset of measurements to perform LOWESS. As this window slides, leaving measurements out and bringing new in, a new LOWESS algorithm is applied, thus probably (and intentionally) repeating the test on each register multiple times.

By doing this, we can have a numerically supported decision on whether a register is detected all over the trajectory as invalid or not. Specifically, we can obtain a combination of the errors on each LOWESS test, plus a ratio of potential valid/invalid registers.

The process is repeated until no invalids are detected, as bad measurements negatively impact the LOWESS estimations, potentially masking other outliers. By repeating this whole loop as many times as we need, we will ensure to clean all invalid measurements.

4.2. Same-Time Measurements Handling

An unexpected issue is the presence of several measurements at the same time for the same trajectory. This is likely due to the Surveillance Data Distribution System (SDDS) architecture, with several sensors detecting the same measurement and processing it at the same moment. However, it can happen to be an input, and VERIF must handle them adequately. Right now, these measurements can reach the filtering phase, causing the Kalman filter to fail on their prediction. Additionally, this could lead to, depending on the order of those registers, generating outliers that will modify the filter output.

An analysis has been performed on three different datasets to understand the impact of this effect. We have observed the following occurrences:

• HelloWorld2 is the main dataset used for internal developing, with all types of sensors and trajectories. We can observe that 1.63% of all the registers (reception from any sensor) share the same time. Note that not all registers have all the measurements; they can be registers with different data. Focusing only on what is important to this issue, there are 1.60% of occurrences with validated barometric height at the same time, which reach the filter. Most of these are the exact same value, sometimes with a limited jump due to quantization jump [42], or invalid values that should have been detected previously. However, 0.014% of those are significantly different and not detected as invalid measurements, with a difference of over 50 ft from each other.
• The second analyzed dataset works in a very mountainous area, with many radars, PSR, and ModeS. We can observe that a much higher 56.78% of registers are duplicate detections. A total 25.86% of cases have multiple barometric height measurements at the same time, while 0.11% of cases have significantly different barometric heights (>50 ft) at the same time.
• Finally, on a dataset of pure ADS, in contrast to the previous ones, although having 541,895 registers, not even one case of repeated time has been detected, proving that it happens depending on the user surveillance system configuration.

Same-Time Measurement Protection

The proposed solution is quite simple, adding a protection mechanism against at the Kalman filter level, ensuring only one measurement is used at a given time while maintaining filter integrity. This solution is effective in meeting the objective and does not add too much complexity and computational time to the system. Figure 9 illustrates how it can choose two measurements at the same time. Instead of going to one, and then to the other, it will evaluate both of them and only apply the one more aligned with its current state.

In order to achieve this efficiently, a prefiltering marks all measurements on the same time, so the filter protection is aware of the special case in current and future measurements. The behavior of the protection follows the approach of outlier detection, by using the residual of the prediction with respect to the sensor measurements and choosing the measurement most aligned with the filter state. The others will be marked as invalid measurements. This logic ensures that only one measurement is used at a given time, not that the chosen one is the most adequate to reality. For example, if one measurement is replicating an old value, but other measurement is saying the flight is maneuvering. However, if that is the case, further measurements and the other filter pass will eventually correct it.

4.3. Vertical Segmentation

The existing vertical segmentation process consists of three main steps:

• Edge detection: Identifying the limits between different vertical modes of flight.
• Segment generation: Creating segments based on the detected edges and assigning initial VMoF categories: Level-Flight (LF) or Climb/Descend (CD).
• Segment refinement: Ensure that the generated segments meet certain conditions (i.e., minimum length, vertical quality), modifying them if not.

The current segmentation often fails to accurately represent the true vertical behavior of the aircraft. The very limited generated information and significant post-processing resources spent trying to improve it highlights the need for this redesign. To address these issues, the proposal primarily redesigns edge detection but also makes changes to the other two steps. The objectives of this redesign are to be more accurate in the detected segments, with emphasis on Level-Flight, to detect multiple segments of the same type if they have sufficient variation, and to separate the Climb/Descend type by four different types depending on their speed and direction, including transitions between them.

4.3.1. Edge Detection

The foundation of the current segmentation approach lies in detecting the moments (edges) where the trajectory transitions from one vertical mode of flight (VMoF) to the other, which currently is identifying departures from a stable vertical rate around zero feet per minute (FPM) to non-zero values. For this task, the process utilizes both forward (FW) and backward (BW) filtered vertical rates. The FW values can identify the start, and the BW values the end of Climb/Descend (CD) VMoF segments (labelled as start and end edges, respectively).

The behavior of the algorithm is explained in the following figures. Figure 10 illustrates the detection of a start edge. Both FW and BW filtered vertical rates must be over the threshold (so it is not a simple spike of one filter) and not have a previous start edge close enough. Figure 11 demonstrates the vertical edges delay-correction process, as the filters produce a delay in vertical rate, especially, in this case by using an additional alpha filter. That delay must be corrected to identify accurately on time the edge. It goes back a fixed amount of time, and then continues until the raw height varies.

Although, conceptually, the detection is correct, in reality, the exact adjustment of the edge, because it depends on the raw measurement, ends up being incorrect, as it is not constant in a general way (quantization, noise.).

The proposed new edge detection (see Algorithm 2) maintains the estimated vertical rate as input to perform the detection. However, instead of checking the point when it differs from zero, the new approach is to detect the instants where the vertical rate before and after each measurement has a bigger differential, as represented in Figure 12.

Algorithm 2. Edge detection

A key aspect of our improvement to the vertical segmentation component involves redefining the fundamental concepts of vertical mode of flight (VMoF) and segment. In the new approach, we consider a new VMoF or segment to occur when the trajectory exhibits a significant change relative to the state of the previous segment. This redefinition represents a paradigm shift in how we interpret vertical flight behavior.

This is performed per measurement in both FW and BW modes, generating six new time series: before, after, and difference, as illustrated in Figure 13. A high value on the difference means that the vertical rate at both sides (before and after) varies a lot; thus, a severe change in vertical rate has occurred, which we consider as a change in VMoF.

Once the curves to be analyzed are contained, the algorithm seeks to find the local maxima, for which another sliding window is performed to check that both curves are at local maximum. At that moment, an edge will be determined at the intermediate point of both measurements. As the new proposal detect more segment types, it does not differentiate between start and end edges. In addition, the alpha filter has been disabled, as it was smoothening the sharp transitions that make this solution work.

4.3.2. Segments Generation

Once the start and end edges are determined, they are used to generate the segments themselves. This process involves joining pairs of start and end edges, with special cases for the first measurement to the first edge, and the last edge to the last measurement.

Figure 14 shows an example of this segment generation logic using both start and end lists. The segment generation continues until one or both edge lists are depleted, creating segments to cover the remaining of the trajectory.

The proposed solution gives the same results but using only one list of edges.

After segment generation, each segment is assigned a corresponding VMoF type. This classification is based on the consideration of all measurements within the segment. The original process involves calculating the proportion of those measurements have the vertical rate above certain thresholds. If more than a 75% of those do not reach the threshold, it will be classified as Level-Flight, if more than 30% are above the threshold, it will be Climb/Descend, and the 5% in the middle will be classified as Unknown.

As the proposal must detail the type further, it needs to calculate an estimate of the average velocity of the segment with which to perform this classification. In order not to be biased by the filter velocity values, a best linear unbiased estimator (BLUE) [43] is used, which is provided with the height values (without invalids or outliers).

$${\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]=\widehat{\mathrm{h}}[0]+\widehat{{\mathrm{v}}_{\mathrm{r}}}[0]·∆\mathrm{t}, ∆\mathrm{t}=\mathrm{t}\left[\mathrm{k}\right]-\mathrm{t}[0]$$

(15)

It is defined in the following equation, where z[k] is the height and ${\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]$ is the estimated height from the ideal trajectory. By comparing the estimated vertical rate average against a certain threshold, the type is retrieved.

4.3.3. Segments Refinement

Once the segments have been generated and classified, refinement is available. This procedure has also been adjusted to the new needs of the current segmentation. We had three types of refinement.

The first one, the fuse adjacent short segments, exemplified in Figure 15, has been improved by making a more intelligent join. Before, it joined short segments (under 10 s) to the next one without considering the types to merge. Now, it looks at both sides, ensuring that it joins the most similar segments using the vertical rates.

On the other hand, the merging of consecutive segments with the same type has been deactivated, as one of the features of the new segmentation is being able those changes.

Finally, the change segment type if low quality has been expanded. This final step calculates a vertical quality metric for each segment, estimating initial height and vertical rate, using the same BLUE estimator as before. It only reclassifies LF segments to Unknown if its vertical rate is too high. Now it can be classified as its corresponding climb or descend type. In order to be reclassified as Unknown, it must exceed an additional deviation threshold, higher than the height resolution levels (25 or 100 ft). This is added as such variations in height can generate enough noise in vertical rate to stop being noisy.

The vertical quality is calculated as the average residual error between the actual height measurements and an ideal linear segment. It is defined as follows:

$$\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}={\displaystyle \frac{\sum \frac{{\left(\mathrm{z}\left[\mathrm{k}\right]-{\widehat{\mathrm{x}}}_{\mathrm{j}\mathrm{p}}\left[\mathrm{k}\right]\right)}^2}{{\mathsf{\sigma }}_{\mathrm{h}}^2[\mathrm{k}]}}{\# \mathrm{r}\mathrm{e}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{s}\mathrm{e}\mathrm{g}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t}}}$$

(16)

4.4. Smooth Overshoot

The combination between forward and backward filters is a tricky step. Both executions have small bias and slow adjustments during maneuvers. By performing the combination, the error is minimized, but the low reactivity of both is maintained, resulting in a slow filter at transition entry and exit, as seen in Figure 5.

Therefore, the original process already incorporates a smoother for the surroundings of the transitions, which works. It applies to both sides of each transition between segments with a certain margin defined by liGateVerticalOvershoot. At each measurement in this region, it computes a smoothed height by combining the raw height measurement (${\mathrm{H}}_{\mathrm{Z}}$) and the filtered height not affected by overshoot (FW or BW height depending on the side of the transition, $H_{FW/BW}$). The values are combined using the weight calculated by the following equation), that depends on the proximity from the measurement to be smoothed to the VMoF transition (gives more weight to ${\mathrm{H}}_{\mathrm{Z}}$ when it is closer to the register).

$${\mathrm{H}}_{\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b}}={\mathrm{H}}_{\mathrm{F}\mathrm{W}/\mathrm{B}\mathrm{W}}·\left(1-\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\right)+{\mathrm{H}}_{\mathrm{Z}}·\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}$$

(17)

$$\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}={\left(1-{\displaystyle \frac{∆\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}}{\mathrm{l}\mathrm{i}\mathrm{G}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{t}}}\right)}^2$$

(18)

The original solution has several problems. As it is using the raw measurements and avoiding the filter output, it is worsening the output by reintroducing noise and outliers’ values that were previously avoided by the use of the invalid detection and the filtering. Although this could be solved easily, the output does not mitigate the overshoots as much due to the used weights.

Our proposal maintains the idea of the previous smooth overshoot but changes the combination of the values. Aiming to reduce the overshoots, generate a continuous result, and stop using the raw measurement, now, four different smoothening steps are performed, using the filtered values but also the combination itself. As illustrated in Figure 16, the correction will prioritize the FW filter values before the VMoF change and the BW filter values after. It starts combining the combination to the FW filter, slowly removing the importance of the BW, which is the most affected on this location. Then, a combination between forward and backward is applied, which will have some overshoot, but it will be way smaller and without shard edges, as before, by not using the measurement. In the end, the same happens when transitioning to BW filter and then to combination.

To produce the smoothed transition between the different outputs (${\mathrm{H}}_{\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b}}$, ${\mathrm{H}}_{\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b}}+{\mathrm{H}}_{\mathrm{B}\mathrm{W}}$, ${\mathrm{H}}_{\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b}}+{\mathrm{H}}_{\mathrm{F}\mathrm{W}}$), a sinusoidal effect can be applied inside the weight allowing for a smooth transition.

$${\mathrm{H}}_{\mathrm{S}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}={\mathrm{H}}_{\mathrm{F}\mathrm{W}/\mathrm{B}\mathrm{W}}·\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}+{\mathrm{H}}_{\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b}}·(1-\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t})$$

(19)

$$\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}=\mathrm{s}\mathrm{i}\mathrm{n}({\displaystyle \frac{∆\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}}{\mathrm{l}\mathrm{i}\mathrm{G}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{t}/2}})$$

(20)

5. Experimental Evaluation

The implementation of new components in the vertical reconstruction process requires a rigorous evaluation to demonstrate their improved performance. In complex cascading systems like this one, any small change can have consequences in the following steps; thus, it is crucial to ensure that each individual process and the overall system show measurable improvements without introducing new errors. To achieve this, we have developed a comprehensive methodology for numerically testing the system, combining both synthetic and real-world data to provide a holistic view of the system’s performance under various conditions.

Our evaluation approach, illustrated in Figure 17, is based on an iterative process that integrates real and synthetic trajectory datasets, but also takes into account Eurocontrol’s evolving requirements and latest new academic algorithms, as well as our users’ feedback. Improvements are then implemented in the OTR system, and the results are evaluated using quantitative metrics and qualitative visualization tools. This evaluation is not only from OTR but from the whole VERIF pipeline. If unsatisfactory, the process iterates until desired performance is achieved, ensuring our system balances operational needs with innovative research advancements.

Our evaluation approach is based on two key pillars: synthetic data analysis and real-world data validation. Synthetic trajectories, based on real aircraft kinematic models [44,45,46,47], play a pivotal role by providing a perfect associated ground truth. This allows us to optimize the system to maximize accuracy and performance, generate a wide range of flight scenarios and maneuvers, introduce controlled levels of noise and errors, and create edge cases that may be rare in real-world data.

While synthetic data offers a controlled environment, real-world data is essential for validating the system’s performance under actual operational conditions. However, obtaining comprehensive, labelled real-world datasets in the air traffic domain presents significant challenges due to the industry’s guarded nature [48,49,50], specifically on radar data. To address this, we have developed a methodology using real data provided by users, which we process to create a usable ground truth through a combination of automated reconstruction and expert manual refinement.

By evaluating both synthetic and real-world data in our testing methodology, we aim to provide a comprehensive and robust evaluation of our improved vertical reconstruction components. This approach ensures that our system not only performs well under controlled conditions but also delivers tangible benefits in real-world air traffic management scenarios. The following sections will show detail on part of the specific tests conducted and their results for each of the improved components: invalid height detection, segmentation, same-time measurement protection, and smooth overshoot correction.

Lastly, it is important to note certain limitations of our experimental approach. Our evaluation primarily focuses on comparing the proposed improvements to the existing SASS-C implementation. Although as stated in the Related Works section, there are other techniques which could be applied instead of our proposals, or other advancements in machine learning, or deep learning; a comparison with all these methods is beyond the scope of this study. The SASS-C system operates under specific computational constraints, as it needs to run on a wide range of user machines without specialized hardware requirements, such as dedicated GPUs. These practical considerations influence our approach and the types of solutions we can implement. Nevertheless, we believe our evaluation provides valuable insights into the performance and potential of our proposed improvements within the context of SASS-C’s operational requirements.

5.1. Experimental Design and Datasets

For clarity, before going further into the details of the improvements and the results of each one, this section clarifies which experiments have been used to demonstrate the functionality of each component, the approach followed for their evaluation, and information regarding each dataset utilized: the type of data, quantity, and types of trajectories included.

As can be seen on

Table 1. Summary on following experiments.

Improvement Component	Name	Numeric Analysis	Visual Analysis
Invalid Measurement Detection	Mountainous dataset	Yes. Manual ground truth for confusion matrix.	3 examples
Same Time Measurements Handling	Mountainous dataset	No.	3 examples
Vertical Segmentation	HelloWorld2 dataset	Yes. Global results of segments dynamics.	3 examples
Smooth Overshoot	Synthetic dataset	Yes. RMSE comparison to synthetic ground truth.	2 examples

, the experiments differentiate between two types of analysis: visual and numerical. Visual analysis allows for comparison without the need for metrics, enabling an experienced user of this field to compare the results on individual trajectories just by looking at the curves. On the other hand, numerical analysis permits the comparison of multiple trajectories, providing insights into overall behavior and determining global patterns on the performance. Note that each experiment is executed on the same exact system using the latest version of the code, the only difference being the use of the specific improvement under evaluation, or the previous solution, if exists.

In order to demonstrate the performance and improvement of these improvements, of all the datasets used for the validation of the system, we kept three of them, capable of covering all the necessary results and analysis, both visual and numerical. The datasets used in this study results are described in detail below in

Table 2. Summary of datasets used in this study.

Dataset Name	Data Type	Amount of Data	Data Distribution
Synthetic Dataset	Synthetic	148,660 raw measurements in 100 trajectories	Pure height values + quantization noise + sensor noise
Mountainous Dataset	Real	1,237,489 raw measurements in 1102 trajectories	65.9% SSR 25% Mode S 8.3% PSR entries
HelloWorld2 Dataset	Real	688,901 raw measurements in 603 trajectories	31.5% MLAT Mode S 30.2% SSR 20.6% ADS 15.1% Mode S 1.9% MLAT SSR 0.3% PSR

:

The first dataset is a simple synthetic one, composed by only elevation movements. Its aim is just to validate the basics of the system. It has been created by a simplistic in-house simulator which merely concatenates smoothly movements (e.g., uniform rectilinear, accelerating) on the three axes based on the values of the previous state, at a given time sampling. Then, it is able to add noise by simulating real height sensors and their communication channels quantification. The dataset is composed of 100 trajectories with 0.25 s of sample time, and the trajectories are composed of a set of simple movements concatenation (e.g., level-flight, ascent, and return to level-flight) using different velocities (from 500 ft/min to 3000 ft/min).

On the other hand, there are real datasets. They are recordings made by the actual ANSP systems in their monitoring area, each with different sensors and specific conditions.

The ‘Mountainous Dataset’ is used in this study since it presents a mixed distribution of air traffic control reports, comprising 65.9% SSR (secondary surveillance radar) entries, 25% Mode S entries, and 8.3% PSR (primary surveillance radar) entries. This dataset is characterized by a high prevalence of extreme outliers and same-time measurements, which makes it particularly suitable for testing and validation of these algorithms. It consists of 1,237,489 raw measurements.

Lastly, there is the HelloWorld2 dataset, which provides a more homogeneous representation of traffic patterns and sensor data. This dataset comprises the main port types with the following distribution: 31.5% MLAT (multilateration) Mode S, 30.2% SSR (secondary surveillance radar), 20.6% ADS (automatic dependent surveillance), 15.1% Mode S, 1.9% MLAT SSR, and 0.3% PSR (primary surveillance radar). This diverse composition accurately reflects a modern, multi-sensor air traffic control environment. In addition, the HelloWorld2 dataset covers trajectories from a wide range of operational scenarios, including the take-offs and landings of different sized aircraft, as well as military movements. This broad representation of air traffic patterns enhances the robustness and applicability of our segmentation analysis in a variety of operational contexts.

5.2. Invalid Measurement Detection

Starting from the mountainous dataset, a subset of 28 trajectories with several invalid measurements was created. All those trajectories and measurements were manually evaluated, indicating which of their measurements are invalid or not. Thanks to this ground truth, a brute force approach was used to find the optimal configuration of nine adjustable parameters that the new algorithm has. A total 1154 different configurations with different sets of parameters were executed on this subset of 28 trajectories, and the one with best results was chosen as default configuration.

5.2.1. Visual Analysis

Analyzing trajectories visually, triangles represent measurements marked as invalid, and green stars indicate what we have manually marked as potentially invalid measurements. In the example shown in Figure 18, the new proposal detects all the marked invalids, whereas the reference method only detects three of them. The first two missed detections are likely due to the sensitivity of the threshold, while the last one was impossible to detect as they are two consecutive invalids. It is worth noting that there are small outliers present, but they are minor and do not reach the threshold used.

A quite challenging scenario is shown in Figure 19, where we have extremely big outliers in the three first measurements. As can be seen, the proposal outperforms the reference, and, although not capable of detecting that the first one is invalid, it is enough for the filtering to provide a good, combined result, in contrast to the reference solution. The minimum number of measurements and the window helped to detect these. In addition, it is an example of two consecutive invalid measurements, only detected by the proposal. Also, a very subtle jump is detected by the proposal, which was not even considered by the manual intervention.

In Figure 20, another example is shown, in this case with the full trajectory. This case is interesting as it shows how, even with three consecutive outliers, the proposes solution is able to perform a good job. Before, it was able to detect one of those as it fit the reference logic. However, other invalid measurements are wrongly marked in the start and end of the trajectory due to a lack of measurements. Although it looks like there are multiple of them, in reality, only a few are present in the first/last part of this specific example. This, joined with the mirror behavior, creates a sharp middle point that is detected as an invalid measurement by the algorithm.

5.2.2. Numerical Analysis

Table 3. Comparison between original and proposed invalid height detection.

Scenario	TP	TN	FP	FN	Accuracy	Precision	Recall	F1-Score
Original	403	35,416	60	745	0.97802	0.87041	0.351045	0.50031
Proposal	530	35,571	53	590	0.982501	0.909091	0.473214	0.622431

demonstrates the numerical superiority of the proposed method across 22 trajectories. Using the standard confusion matrix evaluation procedure, the new approach not only achieves better detection results overall but also minimizes false positives in this notably noisy dataset. The proposed outlier detection method shows significant improvements over the original across key performance metrics. Accuracy, precision, and recall all increased, with recall showing a substantial jump from 35.1% to 47.3%. The F1-score improved from 0.50031 to 0.62243, indicating a better balance between precision and recall. Importantly, the ground truth was manually established, and some visually apparent outliers may fall within the defined acceptable margin. Originally designed for invalid detection, this method performs exceptionally well, capturing even minor outliers beyond the initial objective. This sensitivity, while exceeding the original goal, contributes to the method’s high performance in identifying potential anomalies or invalids in the dataset, particularly impressive given the challenging nature of the data.

The improvement in invalid detection is significant, even for this challenging dataset, as evidenced by the confusion matrix comparison between the reference and our proposal. While this achievement comes at the cost of increased computational time, it remains within reasonable bounds for an offline tool. The solution’s dynamic nature ensures efficiency: it expends more effort only on datasets with numerous invalids, while maintaining near-baseline performance when few or no invalids are present. This adaptive approach balances improved accuracy with computational efficiency.

5.3. Same-Time Measurements Handling

To demonstrate the effectiveness of our solution for same-time measurements, we adopted a visual approach only rather than relying on numerical metrics. This decision was made due to the difficulty in establishing a ground truth in real-world data, which makes quantitative evaluation, in this particular case, very challenging. Thus, for this analysis, we utilized the previously mentioned mountainous real-data dataset, as it contains a significantly higher number of same-time measurement cases compared to other datasets.

Visual Analysis

Figure 21 is a challenging example of a trajectory with simultaneous measurements. It is a trajectory where the algorithm has discarded the measurements below for both the FW and BW passes. This is because they continue to trend, and as a result, have smoother traces, which has a clear impact on the final combined trace, in both height and vertical rate. In addition, by adding the problem, we have the certainty that at each time instant, only one output is generated, and this is decided by the algorithm; we do not depend on which one is processed first.

The problem of the vertical rate in FW and BW being so disparate is common, since although it is not observed, the aircraft is starting a transition, from one side it starts it and from the other it ends it, which is why it is so disparate.

Unlike the previous case, where FW and BW always choose which measures to discard, it is also possible that, as shown in Figure 22, depending on their current state, FW’s or BW’s decisions may be different. However, this is not problematic; it is intentional, since by combining the values, they will end up in a smooth central position.

However, this other example in Figure 23 shows how the problem is not perfectly solved, as there may be measurements at practically the same time (time differences of less than one thousandth, but not exactly the same stored time) with different also quite different height values. Although this improvement is not able to deal with them, the system is robust, so the filter outlier detection marks them and prevents big jumps from occurring in case of a noisy measurement.

5.4. Vertical Segmentation

To analyze segmentation, we employed an approach similar to that used for invalid data detection, systematically testing various configuration parameters including edge-detection thresholds and window sizes. However, unlike the invalid data study, no ground truth was established for segmentation due to the inherently subjective nature of determining segment edges, even when performed manually.

5.4.1. Visual Analysis

First, in Figure 24, a case is shown with synthetic data, where the aircraft climbs and then descends at three very specific speeds. Here we can see one of the main improvements, the splitting of the climb/descent label into separate ones. Thanks to this change, we have gone from having a single segment indicating ‘maneuvering’, to having 6 clearly defined and precise segments, which even indicate if it is a slow movement (under EUROCONTROL standards). In addition, visualization has been added to the segmentation metrics, both before and after of each filter separately, as well as their combination, where peaks are generated in both the before and after, and at a point close to both, always at the exact moment where the limit of the segment is defined.

In this example (Figure 25), we can see a case of climb, where there is a large overshoot, both in speed and height. Due to this effect, another improvement in terms of segmentation is observed, and it is how the edge is moved from being at the intersection between FW and BW diff, to be located more to the left, just at the point where the combined vertical rate reaches 0 ft/min, with the aim of improving the stability of the Level-Flight segment.

Finally, in order to properly demonstrate the operation, a real, long descend trajectory is used. To illustrate the improvement with a visual approach, we have selected the case of Figure 26, which shows a typical descend trajectory, where many different speeds are reached. The generated segments clearly show how the segments are more than adequate for each maneuver. All of them are categorized as slow as they do not exceed the threshold of 6000 ft/min. Looking at the vertical rate, there are moments of very fast maneuvering that force the algorithm to generate more segments than a human would. However, they are not incorrect. Compared to the previous solution, a richer and more detailed result is returned, even detecting a short climb segment at the end of the trajectory. In addition, a segment that is clearly a level flight has been incorrectly re-classified as Unknown, showing the improvement of the new proposal.

5.4.2. Numerical Analysis

Numerically, the proposal also stands out. Looking at the overall results of the refinement in the HelloWorld2 dataset, in the original solution, 131 short segments were detected and merged. In the current solution, this number drops to 17, which are also merged more coherently. This is evidence that the solution is more suitable for this problem. As for the reclassification of segments from LF to Unknown, the number of these segments increases from 319 to 67. The solution clearly gives higher quality LF segments.

In addition, analyzing the metrics generated by CMP, the improvements are also clear. In the total number of segments, shown in

Table 4. Vertical mode of flight count comparison between original and proposal executions in HelloWorld2 all trajectories.

VMoF Category	Original	Proposal	Change
Fast climb rate	3	23	+20
Fast descend rate	5	27	+22
Level-flight	1387	1222	−165
Slow climb rate	684	1739	+1055
Slow descend rate	695	1429	+734
Undetermined	947	577	−370

, the number of ‘Level-Flight’ segments has been reduced, but the duration has been clearly extended (as shown in

Table 5. Vertical mode of flight duration (in seconds) comparison between original and proposal executions in HelloWorld2 all trajectories.

VMoF Category	Original	Proposal	Change
Fast climb rate	121	588	+467
Fast descend rate	372	771	+399
Level-flight	205,555	250,345	+44,790
Slow climb rate	72,914	81,022	+8108
Slow descend rate	68,823	82,003	+13,180
Undetermined	147,189	77,006	−70,183

), showing the generated segments are longer and more accurate. The number of ‘Undetermined’ segments also drops from 947 to 577, indicating better classification accuracy. There is a substantial increase in the detection climb and descend segments, suggesting improved sensitivity to gradual altitude changes. The same can be seen for the total duration according to segment type.

Looking at the standard deviation of the vertical rate values within each segment (

Table 6. Vertical mode of flight vertical rate standard deviation (in FTM) comparison between original and proposal executions in HelloWorld2 all trajectories VMoF.

VMoF Category	Original	Proposal	Change
Fast climb rate	1756.176	4174.461	+2418.285
Fast descend rate	2700.085	3804.016	+1103.931
Level-flight	244.275	68.645	−175.630
Slow climb rate	843.498	929.939	+86.441
Slow descend rate	692.482	862.139	+169.657
Undetermined	81.199	259.206	+178.007

), for Level-Flight segments, it has decreased significantly from 244.275 to 68.645 FTM, showing what was explained before: that the segments are way more accurate. Note that this is crucial for the business of VERIF assessing the performance of sensors. However, there is increased variability in the fast climb and descend rates, which might indicate the more sensitive detection as well.

This specific solution significantly improves segmentation quality, meeting user requirements more effectively. The small increase in computational cost is justified in our offline tool, where accuracy takes precedence over speed. This trade-off aligns with our goal of delivering precise trajectory reconstructions, even at the expense of slightly longer processing times.

5.5. Smooth Overshoot

To evaluate this latest improvement, as the expected behavior of this improvement is to have smoother curves, it is interesting to use synthetic trajectories. This way, it is possible to compare the results to the ground truth for both height and vertical rate (with and without noise), not only easing comparison visually, but also numerically, by the use of precise metrics to determine which method performs better.

5.5.1. Visual Analysis

First of all, it is worth noting that the proposed improvement solves the existing problem in its support for raw measurements, which can be invalid, or same-time, or outliers. Thanks to this new approach with filter values, the bug that caused the results to follow invalid measurements is no longer present.

In terms of the algorithm improvement itself, the end of a slow climb maneuver is shown in Figure 27 to illustrate this improvement. It can be seen how, when faced with a trajectory with limited noise like this one, the proposed method yields a result that is clearly smoother and more realistic. It consistently transitions between both filter passes, while the original method exhibits many more sharp edges, even extending beyond the area between both filters. If there were much noisier measurements, the result could be significantly adversely affected by them.

5.5.2. Numerical Analysis

Notable improvements are found across all the dataset by using the new implementation when compared to the previous one. The average root mean square error (RMSE) per whole trajectory, comparing the ground truth (GT) against the combined output, decreases from 12.12964 to 12.06803.

Thus, both visually and numerically, it is clear that this latest proposal, along with all previous ones, has an overall positive impact on vertical reconstruction.

6. Conclusions

This study has introduced significant improvements to the vertical reconstruction component of the VERIF system of EUROCONTROL’s SASS-C, addressing existing limitations in several parts of its vertical component, specifically, in invalid measurement detection, simultaneous measurement protection, vertical segmentation, and smooth overshoot correction. The results demonstrate substantial advancements in trajectory reconstruction and segmentation accuracy and reliability, contributing to the broader field of air traffic management.

The enhanced invalid measurement detection algorithm, based on LOWESS and a sliding window analysis, has proven highly effective in identifying and removing extreme outliers, significantly improving trajectory accuracy. While this process incurs a slight computational cost, the benefits in trajectory precision far outweigh this minor drawback.

Vertical segmentation has been dramatically improved, offering greater precision in edge detection and providing more detailed segment types. This enhanced segmentation allows for a more nuanced understanding of aircraft behavior and flight phases, crucial for safety analysis and efficient airspace management. The ability to accurately identify transitions between different vertical flight modes significantly contributes to the overall understanding of flight dynamics.

The introduction of simultaneous measurement protection has successfully addressed challenges arising from multiple sensor inputs, enhancing the robustness of the Kalman filtering process. This is particularly important in complex surveillance environments where multiple sensors may provide conflicting or redundant information.

The new smooth overshoot correction method has significantly reduced artifacts at transition points between flight phases, resulting in more realistic and continuous trajectory reconstructions. This improvement is crucial for the accurate analysis of aircraft maneuvers and surveillance system performance evaluation.

Collectively, these improvements contribute to a more accurate and reliable vertical reconstruction component, essential for effective air traffic management and safety analysis. The comprehensive evaluation conducted, using both synthetic and real-world data, provides robust validation of these enhancements’ effectiveness across various operational conditions.

While this work proposes solutions that address SASS-C’s current needs within its existing constraints, future research could explore more advanced approaches. Evaluating machine learning and neural network techniques for trajectory reconstruction and segmentation could yield interesting results and, potentially, further improve system performance. In the near term, work on the vertical component could focus on leveraging the more precise reconstruction to enhance the filtering process, as well as exploring the integration of barometric and geometric data sources for improved accuracy. Additionally, the horizontal component needs improvement too, and exploiting the knowledge gained from the vertical improvements seems the right way to continue. These future directions aim to continually advance SASS-C’s capabilities, ensuring it remains at the forefront of air traffic management technology.

In conclusion, the advancements presented in this paper represent a significant step forward in the accuracy and reliability of vertical trajectory reconstruction and segmentation. These improvements not only enhance the capabilities of the SASS-C system but also contribute to the broader field of air traffic management, potentially leading to safer and more efficient airspace utilization.