A Data-Driven Approach to Identify Flight Test Data Suitable to Design Angle of Attack Synthetic Sensor for Flight Control Systems ^†

Abstract

Digital avionic solutions enable advanced flight control systems to be available also on smaller aircraft. One of the safety-critical segments is the air data system. Innovative architectures allow the use of synthetic sensors that can introduce significant technological and safety advances. The application to aerodynamic angles seems the most promising towards certified applications. In this area, the best procedures concerning the design of synthetic sensors are still an open question within the field. An example is given by the MIDAS project funded in the frame of Clean Sky 2. This paper proposes two data-driven methods that allow to improve performance over the entire flight envelope with particular attention to steady state flight conditions. The training set obtained is considerably undersized with consequent reduction of computational costs. These methods are validated with a real case and they will be used as part of the MIDAS life cycle. The first method, called Data-Driven Identification and Generation of Quasi-Steady States (DIGS), is based on the (i) identification of the lift curve of the aircraft; (ii) augmentation of the training set with artificial flight data points. DIGS’s main aim is to reduce the issue of unbalanced training set. The second method, called Similar Flight Test Data Pruning (SFDP), deals with data reduction based on the isolation of quasi-unique points. Results give an evidence of the validity of the methods for the MIDAS project that can be easily adopted for generic synthetic sensor design for flight control system applications.

1. Introduction

The more electrical aircraft is one of the most challenging objectives of the aviation industry and its main stakeholders. A lot of researches are conducted in order to bring significant innovation on board with the main aim to increase performance and safety, to reduce fuel consumption and emissions [1,2]. Both industrial players and governments from all over the world are very active on latter topics, a lot of effort has been spent and many funds are and will be available in next years to achieve well defined goals. For example, in Europe, the European Parliament has allocated a huge budget to help the transition towards a greener and safer aviation [3]. A helpful resource to get those goals, is the advent of digital revolution. The more relevant role of avionics in flight control and management have pushed aircraft technology towards new limits. Fly by wire (FBW), distributed architectures, are only two of the main results achieved by the aerospace community that are certified to fly in our skies. Within this scenario, the H2020—Clean Sky 2 (under SYS ITD area) project MIDAS [4] is an example. Novel architectures, such as the FBW, are successfully applied to large (Airbus A380) or military aircraft (Leonardo M-346) and will be the new standard for civil aviation and small aircraft transport (SAT) communities in next years [5,6].

The novel digital avionic solutions enable advanced flight control systems to be available also on smaller aircraft (e.g., commuter category). One of the safety-critical segment of the A/C control system is the air data system (ADS) that provides information from the external environment. In fact, from a general point of view, a simplex (not redundant) ADS provides some safety-critical data (e.g., speed, altitude) to automatic control systems needed for correct control and navigation purposes. To this aim, a complete air data set is derived from the following measurements:

• Total pressure
• Static pressure
• Air temperature
• Angle of attack, $\alpha$
• Angle of sideslip, $\beta$

The current ADS state-of-the-art mainly enables two kinds of solutions: one based on single function probe (SFP) (Figure 1a), the other one based on multi function probe (MFP) (Figure 1b). As can be deduced from Figure 1b, MFPs integrate the necessary transducers so that they can be directly connected to a FCC instead of using dedicated devices (e.g., air data computer ADC) as in the case of the SFP-based solution. Moreover, MFPs also provides a measurement of a local flow angle in addition to pressure measurement. A third innovative category is (partially) based on synthetic sensors and one possible example is shown in Figure 1c inspired to the MIDAS project.

The MIDAS project aims to provide an innovative ADS partially based on synthetic sensors exploiting the Smart-Air Data, Attitude and Heading Reference System (Smart-ADAHRS) patent. In particular, the proposed ADS solution is based on direct measure of dynamic pressure, static pressure and air temperature using dedicated sensors (e.g., SFP in the MIDAS project). The aerodynamic angles (angle of attack and angle of sideslip) are estimated as an indirect measure by means of data fusion between measured air data available and inertial data as described in [7]. This particular choice was taken considering that the high accuracy level required to pressure and air temperature measurements is hardly achievable with synthetic sensors [8]. On the contrary, previous research shows that common aerodynamic angle requirements, derived from flight control performances, can be satisfied with synthetic sensors [9].

As a first impact, the MIDAS solution will cut weight, costs and emissions with respect to the other possible ADS solutions. Moreover, the ADS architectures presented in Figure 1 shall be replicated (in duplex or triplex solutions) in order to achieve the safety objectives defined by the airworthiness authorities for large airplanes [10], general aviation [11] and, for instance, for next generation Urban Air Mobility (UAM) aircraft [12]. From applicable safety objectives, it is clear that the automatic control systems could benefit of synthetic sensors due to high reliability levels to be guaranteed. In recent years, synthetic sensors, or analytical redundancy, have become more familiar concepts, because the avionic background is today mature to welcome such innovations [13,14].

As a second impact, the redundant ADS partially based on synthetic sensors (e.g., estimating AoA and AoS) introduces dissimilarities for some air data. Dissimilarity is crucial to overcome some issues related to common failure modes or incorrect failure diagnosis of modern ADS [15,16]. A consequent advantage is related to simplification of the whole ADS architecture. Redundancy aspect is quite easy to be managed on large passenger aircraft where there are no particular limitations for air data sensors protruding from the fuselage and cable/tube displacement inside the fuselage. Whereas, the same operation can become very challenging when the fuselage dimensions reduce (business aircraft, SAT or UAM).

In the field of the analytical redundancy techniques, several solutions can be adopted to virtually synthesize air data sensors [17]. For example, the current work deals with a patented technology, named Smart-ADAHRS, based on neural networks [18,19].

Focusing on the angle of attack, the present work describes a data-driven approach suitable for a smarter design of synthetic sensors based on machine learning when working with real flight data. Given the big amount of data and uncertainties related to real measurement taken on board, it is possible have larger errors than required. For example, when data set used for training is unbalanced as the case of real flight data where the steady state flight conditions are quite negligible (as flight data records) with respect to the dynamic ones. This particular aspect will be investigated in this work in order to propose a methodology to avoid lack of performance during the steady-state flight conditions in favor of high accuracy during dynamic maneuvers. In fact, the main objective of the proposed approach is to increase the synthetic sensor performance during steady-state flight conditions while guaranteeing acceptable performance during the dynamic ones.

Firstly, quasi-steady conditions are identified automatically from the whole flight data records and they are used for data regression. The regression function is then used to generate virtual flight test data points in order to artificially enrich the training data set with steady state flight conditions. Secondly, the training data set is pruned by those flight data records not considered unique. By means of this two actions, an augmented and pruned training set is obtained with the objective to drastically reduce the ratio between dynamic and stationary flight test points. This dataset allows to achieve the aforementioned target, i.e., increase the performance during steady-state flight conditions.

Previous research is presented in Section 2, in order to provide a suitable background to the present work. An overview of the proposed methodology is presented in Section 3. The quasi-steady conditions identification approach is presented from an analytical point of view in Section 4. The prune method is presented in Section 5. Finally, the proposed approach is validated comparing results with previous works in Section 6.

2. Background

In this work, synthetic sensors for aerodynamic angles are based on neural network techniques, as presented in previous works [20,21]. The core of the Smart-ADAHRS patent relies in exploitation of the already available data measured from attitude and heading reference system (AHRS) and ADS to provide an estimation of the aerodynamic angles, angle of attack ($\alpha$) and angle of sideslip ($\beta$), as described in Figure 2b.

According to the Universal Approximation Theorem, a multilayer perceptron (MLP) is able to uniformly approximate any function inside the ${[0;1]}^n$ hypercube. The key-to-success to design an accurate MLP, obviously, is the training data set used for the learning process. The training stage, based on flight data, therefore, is crucial for the correct learning process of the synthetic sensors.

For this work, flight data collection is the result of a collaboration between different entities. The Smart-ADAHRS technological demonstrator was developed by Politecnico di Torino and it is able to record all the input signals needed by the MLP. Target values (AoA and AoS) are measured by the Flight Test Instrumentation (FTI) developed by the Politecnico di Milano [22], which manages the flight test campaign of the ultra light machine (ULM). The test aircraft is the G-70 from Ing. Nando Groppo S.r.l. [23].

In previous works, it emerged that the training data set needs some pre-processing manipulations in order to avoid common learning issues [24]. The training procedure results in one of the several local minima and to increase the probability of selecting the most valid can be mitigated with several techniques: e.g., splitting flight data, re-training with different initial values, a quasi-uniform hypercube distribution. The latter is obtained by means of a dedicated flight test campaign, with maneuvers able to cover most of the flight envelope and exciting the natural modes of the A/C. Moreover, a uniform distribution of flight data inside the hypercube is beneficial to avoid that the MLP minimizes the error mainly in local area rather than on the entire hypercube.

It was noted that the flight data collected on the ULM are more numerous during dynamic rather than steady flight conditions. The ratio between quasi-static test points and dynamic ones is about $1÷\mathrm{100,000}$. This lead to a common error that authors documented in [25]. Verification of synthetic sensors (or analytical estimators—observers, Kalman filters, etc.) is often carried out using dynamic source of data, such as those from flight tests. As aforementioned, this work is focused on the longitudinal dynamics and one of the objectives is to improve the estimation of the synthetic sensor (SS) during steady state conditions. A set of steady conditions can be manually generated given the A/C weight, pitch angle and dynamic pressure in order to fed the SS and verify its performance. Figure 3 is an example of steady state results obtained from SS designed in previous works [24]. Once the issue with static training was pointed out, authors designed a data-driven approach to improve the SS performance during steady state flight conditions.

The steady state flight conditions are reproduced using A/C manufacturer data (e.g., $C_{L_{\alpha }}$ and ${\alpha }_0$) and assuming that the aircraft is flying on uniform horizontal flight path. Starting from known A/C weight and considering a realistic interval of AoA, the dynamic pressure and pitch angle can be calculated, while all other quantities are set null. Figure 3 represents the absolute values of reference AoA and the one estimated with the SS [24]. Although the maximum absolute errors are limited, the mean error is quite large and it should be reduced. To this aim, a data-driven approach is proposed in this work to overcome this issue mainly due to an unbalanced training pattern between dynamic and steady state maneuvers. Before description of the proposed approach, a brief digression is presented in order to provide the necessary details of the neural network used in this work.

Neural Network Architecture

The Smart-ADAHRS project deals with a very straightforward model patented in 2012, suitable for real-time and cost effective innovative avionic systems. It is possible to write an aerodynamic angle as sum of two addends, defining the output of the synthetic sensor as in [26]:

$$\begin{array}{cc}\hfill {\alpha }_{SS}& =\widehat{\alpha }+\mathsf{\Delta }\alpha \hfill \end{array}$$

(1)

where $\widehat{\alpha }$ is the first estimation obtained with flight mechanics equations [27] whereas $\mathsf{\Delta }\alpha$ is the difference between a first estimation, $\widehat{\alpha }$, and the true value, $\alpha$. According to the patent [7], the initial estimation of the angle of attack $\widehat{\alpha }$ is augmented with the evaluation of $\mathsf{\Delta }\alpha$ obtained by one MLP. The $\widehat{\alpha }$ can be evaluated as follows:

$$\begin{array}{cc}\hfill \widehat{\alpha }& =\theta -\gamma \hfill \end{array}$$

(2)

where $\theta$ stands for the pitch angle, $\gamma$ for the flight path angle. The neural network structure has one single hidden layer with 13 neurons and one linear output layer.

During the training procedure, the weights of the neural network are estimated solving the non-convex problem of the error function optimization. Different heuristic rules exist and the most common is the Levenberg–Marquardt (LM) algorithm. However, this application deals with a big amount of data, making the employment of the Levenberg–Marquardt (LM) rule not feasible. The training rule applied in this article is the Resilient Propagation (RPROP). The complete input vector needed by the SS includes data from the GPS, the ADS and the AHRS which provides body accelerations, angular rates and attitude, as can be seen in Figure 4.

For the application considered in this work (ULM category) the control surfaces are neglected. The following feed-forward predictors are hence implemented:

$$\begin{array}{cc}\hfill \mathsf{\Delta }\alpha & =f_{\alpha }\left(q_c,{\dot{q}}_c,n_x,n_z,\theta ,q,\widehat{\alpha }\right)\hfill \end{array}$$

(3)

where $q_c$ is the impact pressure, ${\dot{q}}_c$ is the time derivative of $q_c$, $n_x$, $n_y$, $n_z$ are the accelerations measured by the accelerometers respectively on $X_{Body}$, $Y_{Body}$ and $Z_{Body}$ axes and q is the pitch rate. Previous research activities on simulated turbulent environment in [20] showed the possibility of considering previous time steps of the input vector in a Time Delay Network. This practice however is not here implemented because, at this phase, it does not bring any significant advantages.

The error is calculated as the difference between the estimated angle (${\alpha }_{SS}$) and the real angle ($\alpha$). Angle of attack’s target performance were established at the beginning of the MIDAS project [28,29]. The most relevant for this work are:

• steady state max error $<0.5{}^{°}$
• dynamic $2\sigma$ error $<1.5{}^{°}$

3. Proposed Approach

After the completion of the flight test campaign, a big amount of data is available for training and test. This section describes two practical methodologies in order to identify the most suitable points for training. The final goal is to guarantee a more uniformly distributed domain where the synthetic sensor is designed by means of:

1.

avoiding coarse areas (usually related to steady state flight conditions);

2.

reducing over populated areas (usually related to dynamic flight conditions).

The first method, Data-Driven Identification and Generation of Quasi-Steady States (DIGS) is based on the following three pillars: (1) identification; (2) regression and (3) generation. The identification method, presented in [25], is based on recognition of quasi-steady flight conditions from the whole flight record, therefore, exploiting also those maneuvers not explicitly flown for the flight trials (e.g., transfer between altitudes and re-alignment on desired heading). The regression step deals with definition of a regression function considering identified quasi-steady flight conditions. Finally, the DIGS regression function is used to generate artificial flight test points to be used to increase the training population where necessary. The DIGS method is limited to longitudinal steady flight conditions in this work.

The second method, Similar Flight Test Data Pruning (SFDP), is based on a pruning schemes that aims to avoid very similar flight records (usually dynamic flight conditions) with the aim to define a balanced definition domain where the synthetic sensor is defined.

Finally, adopting this approach, it can be avoided to require more flight test hours to populate the training data set with steady flight conditions in favor of virtual ones.

4. Data-Driven Identification and Generation of Quasi-Steady States (DIGS)

This section describes the method applied to find the stationary and quasi-stationary points in a generic flight data collection in order to augment the training set. A key aspect of this analysis is the definition of quasi-stationary and quasi-symmetric flight condition. Briefly, the main procedure is based on the assignation of a value called score considering only a selection of flight parameters important to this goal (see

Table 1. List of signals and corresponding statistics.

Signal	First Statistic	Value	Second Statistic (Standard Deviation) Value
Angular rate	Sample mean	0.01 rad s−1	0.05 rad s−1
Altitude	Max deviation	1 m	0.5 m
Vertical speed	Sample mean	1 m s−1	0.5 m s−1
Acceleration	Sample mean	0.05 g	0.025 g
Impact pressure	Max deviation	100 Pa	50 Pa
Pitch angle	Max deviation	2°	1°
Roll angle	Sample mean	1°	0.5°
Yaw angle	Max deviation	1°	0.5°

). The whole flight time history is split in several non-overlapping time windows. In each time window, two values are calculated for each signal. The second parameter is always the standard deviation. The first one is defined according to the nature of the flight signal itself, as detailed below. For example, the mean value is considered for rates, whereas the maximum deviation is chosen for the altitude. All choices are reported in Table 1 with corresponding thresholds to be compared with. From this comparison, a (single) signal score is assigned. Averaging the signal scores, an overall score is assigned to the time window. If the overall score is higher than a defined threshold, then the time window is tagged as steady-state one, otherwise is discarded. In Figure 5 and Algorithm 1 are reported some details.

As mentioned above, the signals are divided in non-overlapping time windows. The length of the time windows is constant and it has been taken as 5 s. The length of the time-window is obtained as a trade-off between finding an actual stationary point and the efficiency of the algorithm. In fact, longer time window will bring to a more reliable evaluation of the flight condition whereas shorter time window will bring to extra points in the final result. Afterwards, some statistics of the signal during each time window are evaluated. As a result of this statistical analysis, the original sampling frequency of the signal can have an important influence on the final result. In fact, the number of elements in each subdivision must be sufficiently high such that the sample estimators are statistically valid. In this work, original sampling period is about 0.05 s, corresponding to 100 elements per interval, which is sufficiently high. The original sampling period comes from the sampling frequency of the FTI.

Algorithm 1: Quasi-Stationary and Quasi-Symmetric flight conditions detection algorithm

For each interval sample mean, sample standard deviation and the deviation between the minimum and maximum values covered by the signal inside the sample are calculated. A score is hence assigned to each interval depending on the descriptive values calculated beforehand. The decision on which statistics consider for the score assignment grounds on the type of signal evaluated. In some cases, the score depends on how much the sample mean is close to a given value. For instance, the angular rates must be zero in stationary conditions. In other cases, when the interest falls in observing a constant signal, the sample standard deviation or the maximum deviation drives the score assignment. For instance, the closer the maximum deviation of the impact pressure is to zero, the higher is the assigned score.

The rules governing the score assignment to a given sample has not a unique solution. In this paper, a piecewise linear function is implemented but, according to the authors, several other possibilities exist. To enhance the score assignment, the standard deviation of the sample is always accounted in this work. This helps to increase the score of the stable sample with respect to sample affected by high variability. See Section 4.1 for the complete description of the score assignment process.

When every signal has been sampled and the signal score has been assigned, the final sample score is evaluated as the mean of the scores among the signals. A weighted mean could be implemented here. However, it implies the definition of which signal is considered the most important during this evaluation and without this consideration a weighted mean cannot be implemented.

Once the scores have been obtained, the selection of the stationary and quasi- stationary points can be carried out. This procedure involves the definition of a minimum score threshold necessary to pick or not a sample, given the scores assigned to each sample. The previous steps are applied equally to each flight test. For this reason, the scores are comparable among different flights. However, it has been observed that a normalization procedure greatly simplifies the decision process. In fact, subtle differences exist between quasi-stationary and quasi-symmetric conditions. To better explain this, it must be recalled that this method can obtain a set of points on the flight envelope slightly relaxing the trim condition constraints. In some cases, very low time derivatives and deviations can be observed. However, the attitude could be too much asymmetric to be neglected. At the same time, a slightly more symmetric flight condition but corresponding to higher deviation on the measures than the previous case can be considered valid. The problem of defining when a flight condition is quasi-symmetric and quasi-stationary is obviously ill-posed. The authors have shown in [25] that it can be converted on the problem of defining a threshold on the score, that unfortunately fails to be a metric. Moreover, if the score is normalized, the equivalence between asymmetry and stationarity in the score space seems to be well faced.

4.1. The Signal Score Assignment Process

This subsection shows more details on how a sample of a signal has been related to a scalar value. This step is crucial for the effective functioning of the algorithm. Two different solutions have been applied.

Mathematically speaking, given a signal $x_i=x_i\left(t\right)$ it is possible to extract a sample $x_i\left[n\right]=E\left[x_i\left(t\right)\right]$ with $\left(n-1\right)t_s\le t<n{t}_s$. Various statistics $\theta$ of the sample $x_i\left[n\right]$ can be measured. Eventually, the signal score s is assigned to the n-th sample based on the corresponding $\theta \left(n\right)$ as in the following relation:

$$s=s\left(\theta \left(n\right)\right)=s\left(n\right)$$

(4)

The first approach on the functional form for s is piecewise linear. The triangle function can be generally defined as in Equation (5). An example can be seen in Figure 6.

$$\mathsf{\Lambda }\left(\theta \right)=\left\{\begin{array}{cc}1-\frac{\left|\theta \right|}{{\theta }_{th}}\hfill & \mathrm{for}\left|\theta \right|<{\theta }_{th}\hfill \\ 0\hfill & \mathrm{for}\left|\theta \right|\ge {\theta }_{th}\hfill \end{array}\right.$$

(5)

Setting $s=\mathsf{\Lambda }\left(\theta \left(x\left[n\right]\right)\right)$ a first evaluation of the sample score can be obtained. For instance, if the $q_c$ signal is considered, the maximum acceptable $\mathsf{\Delta }{q}_c$ during 5 $\mathrm{s}$ can be set to 100 $\mathrm{Pa}$. In this case, if the maximum deviation is higher than $\mathsf{\Delta }{q}_c$ then a null score will be assigned. At the same time, if the maximum deviation is between 0 $\mathrm{Pa}$ and 100 $\mathrm{Pa}$, then a proportional score will be assigned to that given sample.

However, it has been found that extending the previous analysis to more than one single statistic brings to higher coefficient of determination with respect to using only one statistic. To this aim, Equation (5) can be modified considering similarities to a triangulation problem.

Given ${\theta }_{1,1},{\theta }_{1,2},{\theta }_{2,1},{\theta }_{2,2}\in \mathbb{R}$ with ${\theta }_{1,1}<{\theta }_{1,2}$ and ${\theta }_{2,1}<{\theta }_{2,2}$ it is possible to identify four regions as reported in Figure 7 and Figure 8. For convenience, each region is identified by the same index of one of the two border vertices.

Let $P\in {\mathbb{R}}^2$ a point belonging to the $O{\theta }_1{\theta }_2$ space, it is possible to write

$$\begin{array}{c}\hfill P\in {\mathcal{R}}_1\iff \left\{\begin{array}{c}{\theta }_2>{\theta }_{2,s_1}\left({\theta }_1\right)\hfill \\ {\theta }_1<{\theta }_{1,m}\hfill \\ {\theta }_2<{\theta }_{2,m}\hfill \end{array}\right.,P\in {\mathcal{R}}_2\iff \left\{\begin{array}{c}{\theta }_1\ge {\theta }_{1,m}\hfill \\ {\theta }_2<{\theta }_{2,m}\hfill \\ {\theta }_2>{\theta }_{2,s_2}\left({\theta }_1\right)\hfill \end{array}\right.,\end{array}$$

(6)

$$\begin{array}{c}\hfill P\in {\mathcal{R}}_3\iff \left\{\begin{array}{c}{\theta }_1\ge {\theta }_{1,m}\hfill \\ {\theta }_2\ge {\theta }_{2,m}\hfill \\ {\theta }_2<{\theta }_{2,s_3}\left({\theta }_1\right)\hfill \end{array}\right.,P\in {\mathcal{R}}_4\iff \left\{\begin{array}{c}{\theta }_1<{\theta }_{1,m}\hfill \\ {\theta }_2\ge {\theta }_{2,m}\hfill \\ {\theta }_2<{\theta }_{2,s_4}\left({\theta }_1\right)\hfill \end{array}\right.\end{array}$$

(7)

where $s_{1,2,3,4}$ stand for the 4 straight lines defining the border.

Once the r-th region in which P belongs to has been found, it is possible to write Equation (8) which describes each planar face of the pyramid as follows:

$$\nabla s_r·\left(\mathit{\theta }-{\mathit{\theta }}_r\right)=s_r\left(\mathit{\theta }\right)-s_r\left({\mathit{\theta }}_r\right)$$

(8)

where $\mathit{\theta }$ represents the vector of coordinates of the point P. Moreover,

$$\begin{array}{c}\hfill \nabla s_r·\left({\mathit{\theta }}_m-{\mathit{\theta }}_r\right)=s_r\left({\mathit{\theta }}_m\right)-s_r\left({\mathit{\theta }}_r\right)=1\end{array}$$

(9)

$$\begin{array}{c}\hfill \nabla s_r·\left({\mathit{\theta }}_{r+1}-{\mathit{\theta }}_r\right)=s_r\left({\mathit{\theta }}_{r+1}\right)-s_r\left({\mathit{\theta }}_r\right)=0\end{array}$$

(10)

For simplicity, Equation (9) contains a little abuse of notation. In fact, $r+1$ becomes 1 for $r=4$.

The following linear system can be written:

$$\left[\begin{array}{cc}{\theta }_{1,m}-{\theta }_{1,r}& {\theta }_{2,m}-{\theta }_{2,r}\\ {\theta }_{1,r+1}-{\theta }_{1,r}& {\theta }_{2,r+1}-{\theta }_{2,r}\end{array}\right]\left\{\begin{array}{c}{\displaystyle \frac{\partial s_r}{\partial {\theta }_1}}\\ {\displaystyle \frac{\partial s_r}{\partial {\theta }_2}}\end{array}\right\}=\left\{\begin{array}{c}1\\ 0\end{array}\right\}$$

(11)

which brings to Equation (8),

$$\begin{array}{cc}\hfill \frac{\partial s_r}{\partial {\theta }_1}& =\frac{{\theta }_{2,r+1}-{\theta }_{2,r}}{\det \mathbf{G}}\end{array}$$

(12)

$$\begin{array}{cc}\hfill \frac{\partial s_r}{\partial {\theta }_2}& =-\frac{{\theta }_{1,r+1}-{\theta }_{1,r}}{\det \mathbf{G}}\end{array}$$

(13)

representing the two partial derivatives of the score function s with respect to ${\theta }_1$ and ${\theta }_2$, where $\mathbf{G}$ is the matrix at the LHS of Equation (11).

Eventually, the score assigned to any sample $x_i\left[n\right]$ is evaluated as follows:

$$s\left(x_i\left[n\right]\right)=\left\{\begin{array}{cc}\nabla s_r\left(\mathit{\theta }\left(x_i\left[n\right]\right)-{\mathit{\theta }}_r\right)\hfill & \mathrm{if}P\in {\mathcal{R}}_r\mathrm{with}r\in \left\{1,2,3,4\right\}\hfill \\ 0\hfill & \mathrm{if}P\notin {\mathcal{R}}_r\mathrm{with}r\in \left\{1,2,3,4\right\}\hfill \end{array}\right.$$

(14)

With this formulation, two statistics can be considered. In this paper, the standard deviation of the sample is always applied as second statistic. This help to increase the score for stable samples. Table 1 shows the values used in this paper.

4.2. Methodology Verification

This section shows the results obtained by the DIGS method on a flight test campaign conducted in the north Italy during June 2017. A total amount of 18 flight tests have been collected on a ULM named G70 Figure 2a and manufactured by Ing. Nando Groppo. The G70 is a propeller driven aircraft with traditional wing-tail configuration, 2 seats, non-retractable landing gear. A fully-fledged FTI suite developed by Politecnico di Milano [30,31] is installed on-board. This FTI system, called Mnemosine, is capable of supporting certification procedures [22]. Its design has been tailored for ULM and it consists of a low cost, low intrusive and flexible solution for flight testing.

A second equipment, the Smart-ADAHRS demonstrator, is installed on board. This system is equipped with an independent ADAHRS platform integrated with GNSS. The only parts shared by the Mnemosine and Smart-ADAHRS are the pressure ports, probes and the first segment of pneumatic links. The aerodynamic angles of the aircraft have been measured by two vanes mounted on a Pitot-boom structure. An example of the flight data recorded during one flight test can be seen in Figure 9.

To evaluate the capability of the DIGS method, the $C_L\left(\alpha \right)$ curve is identified from flight test records not performed for this scope. As a result that no engine measurement is available on-board, a fuel consumption linearly decreasing with time has been assumed. In this way, it is possible to evaluate the lift coefficient as $C_L=\frac{mg}{q_{c}S}$, where m is the aircraft mass, g is the gravitational acceleration, $q_c$ is the impact pressure and S is the wing surface area. It is interesting to note that the obtained samples organize on three straight lines in Figure 10. In fact, some of them corresponds to flaps down in Take-Off (about 14${}^{°}$) and Landing (about 36${}^{°}$) conditions. As a result that the flap angle deflection, for leading edge plain flap, is equivalent to offset the null lift direction with an angle proportional to the ${\delta }_f$, it is possible to identify two analytic values of $\mathsf{\Delta }\alpha$ to compare our results. In fact

$$\mathsf{\Delta }\alpha =\tau {\delta }_f$$

(15)

In this work, to assess the accuracy of the method, the effect given by the flap on ${\alpha }_0$ has been evaluated. Considering a 2-dimensional ${\tau }_{2D}=0.5$ [32] and a surface ratio $S_f/S=0.66$, the 3-dimensional value for $\frac{\partial \alpha }{\partial {\delta }_f}$ becomes ${\tau }_{3D}=0.33$. Unfortunately, this value is valid up to 20${}^{°}$ of flap deflection. The ${\kappa }^{\prime }$ parameter [33] has been used to extend the evaluation beyond this limit, leading to the following more general formulation:

$${\tau }_{3D}={\tau }_{2D}*{\kappa }^{\prime }*\frac{S_f}{S}$$

(16)

Therefore, the following values are used in this paper ${\tau }_{3D,14{}^{°}}=0.33$, ${\tau }_{3D,36{}^{°}}=0.27$.

Looking at the Figure 10 it is possible to identify all the sample points obtained with a score $s\left(n\right)\ge 0.667$ and ${\delta }_f=0{}^{°}$.

The samples resulted to be organized on a straight line, with slope $C_{L_{\alpha }}$ and zero ${\alpha }_0$ close to the independent analysis previously conducted by [34]. To avoid non linear effects, only the samples with $\alpha <12{}^{°}$ have been taken in consideration. The quality of the linear regression has been measured with the coefficient of determination.

Actually, the effects of $C_{L_{{\delta }_e}}$ and asymmetric flight condition should be accounted for in a post-processing correction. Shortly, according to [35], the slope obtained by the regression should be properly called $C_{L_{\alpha }}^{*}$. However, the difference between $C_{L_{\alpha }}^{*}$ and $C_{L_{\alpha }}$ has been here dropped for sake of clarity.

Figure 10 shows the samples corresponding to flaps in TO condition and it collects the samples in LND condition. It is important to notice that no flight control surface data has been directly applied into the score assignment. In fact, the flap deflection angle ${\delta }_f$ has been used only in post-processing to distinguish clean, take-off and landing configurations in order to clearly identify the three linear regression.

Table 2. Results on the estimation of $C_{L,\alpha }$ and ${\alpha }_0$.

Source	${\mathit{C}}_{\mathit{L},\mathit{\alpha }}$	${\mathit{\alpha }}_0$	${\mathit{R}}^2$
Battaini [34]	$4.881$	$-5.9015{}^{°}$	-
Regression, clean	$4.8779$	$-5.7673{}^{°}$	$0.95812$
Regression, TO	$4.817$	$-9.2128{}^{°}$	$0.70726$
Regression, LND	$4.1979$	$-15.3087{}^{°}$	$0.88861$

collects the results obtained. The comparison of the regression obtained with the DIGS identification method with respect to the independent analysis conducted in [34] confirms the validity of the method. A difference about $-0.06$% has been obtained on $C_{L_{\alpha }}$ with respect to the manual derivation, neglecting the effect of the elevator. For what concern the ${\alpha }_0$, the difference is about $-2.27$%.

Although the coefficient of determination of the linear regression is lower than $0.9$ for the TO and LND conditions, the results are in accordance with the manual estimation.

4.3. DIGS Training Pattern Augmentation

The DIGS identification method described in Section 4 is used to define a regression function to be used to artificially generate quasi-steady state training points. The aim is to populate the training data set with steady-state flight test points if not already included. Although this action can be helpful in order to obtain a better data distribution, the hypercube (where the neural network is defined) is still unbalanced and another action is required as described in the following section.

5. Similar Flight Test Data Pruning (SFDP)

The SFDP main objective is to leave only the quasi-unique flight data records in the training data set by means of pruning similar ones.

The concept of similar, or quasi-unique, applied to flight test data records, is defined according to a threshold vector. The threshold vector, ${\mathbf{\delta }}_{prune}$, contains a well defined limit for each one of the training signals (e.g., ${\delta }_{q_c}=30\mathrm{Pa}$ is the threshold applied on the dynamic pressure). Then each time record is compared with all others. If each element (i.e., flight data recorded) of the vector comparison is within its corresponding threshold, than the time record is tagged as duplicate and pruned away, otherwise it is kept as unique as described in the Algorithm 2.

Algorithm 2: Pruning approach of the training data set

The proposed approach for pruning leads to a lighter training data set avoiding over fitting issues (e.g., data concentration).

In this work, the threshold vector is based on previous sensitivity analysis [36] and upon considerations on sensors’ uncertainties. The threshold vector associated to input vector (see Section 2) for the present work is made up of the following components:

$$\begin{array}{cc}\hfill {\mathbf{\delta }}_{prune}=& \left[{\delta }_{q_c},{\delta }_{{\dot{q}}_c},{\delta }_q,{\delta }_{n_x},{\delta }_{n_z},{\delta }_{\theta },{\delta }_{{\alpha }_T}\right]=\hfill \\ \hfill =& \left[15\mathrm{Pa},45\mathrm{Pa}/\mathrm{s},1{}^{°}/\mathrm{s},1 \mathrm{m}/{\mathrm{s}}^2,1 \mathrm{m}/{\mathrm{s}}^2,1{}^{°},1{}^{°}\right].\hfill \end{array}$$

(17)

The threshold applied to the dynamic pressure, considering the altitudes flown by the present A/C, can be read as an absolute threshold $\le 1.5\mathrm{kt}$ for speed higher than $40\mathrm{kt}$. Whereas, the one applied to the time derivative of the dynamic pressure can be translated in an absolute threshold $\le 4.5\mathrm{kt}/\mathrm{s}$ for speed higher than $40\mathrm{kt}$. All other thresholds are clear from Equation (17).

The Figure 11 is an example of the results of the pruning process on training data. A beneficial data reduction of higher than $1÷100$ is observed during the present work.

Applying the threshold vector of Equation (17) to distinguish quasi-unique element, the training hypercube is modified as in Figure 12. For each single box, the central mark indicates the median and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers (dashed vertical lines) cover up to the most extreme data points not considered outliers whereas the outliers are plotted using the + symbol.

Quasi-unique points are only the $3\%$ of the original training data set (from 511,103 to 13,248), with extended areas of 25th and the 75th percentile. Another significant improvement is the outlier percentage: from $48.9\%$ to $18.8\%$. An important weight should be given to outliers because they allow to cover the most of the flight envelope. This means that the original training set had the most of the points in the same region relying only a few importance to those points classified as outliers. The pruning process also overcomes this issue.

6. Results and Validation

Considering the SFDP method, the ratio between steady-state and dynamic conditions is increased to about $1÷500$. Moreover, 200 artificial steady state flight points from the DIGS method are added to the training in order to adequately increase the steady state population. It was noted that only 200 artificial steady state points are enough to keep a good balance between SS steady and dynamic accuracies. Anyway, this aspect could be further investigated.

With this set-up, new trainings are performed and the best one (selected according to criteria of [24]) is used to define the SS for angle of attack estimation.

The Figure 13 collects the error PDF for AoA estimation obtained both for the SS training and test. Obviously, the flight tests used for the training are pruned as described in Section 5, whereas the flight test records used for SS test are not pruned and they are the same used in [24].

From Figure 13a can be noticed that the PDFs for pruned and augmented training show higher $2\sigma$ errors. This behavior was expected as a consequence of the SFDP method because the error distribution has a strict relationship with the data reduction of the pruned training pattern. Anyway, this issue is negligible with respect to the improvement of the PDF on test maneuvers. From Figure 13b it can be noted that the SS, trained with DIGS and SFDP, shows a PDF with lower standard deviation than the original ones for all test flights considered here. Main features are also summarized in

Table 3. PDF characteristics of the corresponding flight test. The $2\sigma$ value is calculated as the error of the $95.45\%$ of the data points.

	SS Original [24]			SS Based on DIGS and SFDP
Flight Test No.	mean	2σ	∣max∣	mean	2σ	∣max∣
1	$-0.13{}^{°}$	$1.06{}^{°}$	$4.05{}^{°}$	$-0.13{}^{°}$	$1.48{}^{°}$	$3.31{}^{°}$
2	$0.24{}^{°}$	$1.17{}^{°}$	$5.78{}^{°}$	$0.21{}^{°}$	$1.39{}^{°}$	$4.44{}^{°}$
3	$-0.18{}^{°}$	$1.17{}^{°}$	$2.53{}^{°}$	$-0.15{}^{°}$	$1.33{}^{°}$	$1.83{}^{°}$
4	$-0.24{}^{°}$	$1.62{}^{°}$	$3.59{}^{°}$	$-0.29{}^{°}$	$1.82{}^{°}$	$3.35{}^{°}$
5	$-0.39{}^{°}$	$1.99{}^{°}$	$5.43{}^{°}$	$-0.23{}^{°}$	$1.51{}^{°}$	$3.63{}^{°}$
6	$-0.04{}^{°}$	$1.28{}^{°}$	$4.00{}^{°}$	$0.10{}^{°}$	$1.10{}^{°}$	$3.33{}^{°}$
7	$-0.50{}^{°}$	$2.43{}^{°}$	$5.19{}^{°}$	$-0.30{}^{°}$	$1.48{}^{°}$	$4.02{}^{°}$

that shows that the new SS (i.e., trained using both DIGS and SFDP methods) works better than the original one for all test maneuvers even though a light decrease of accuracy can be noted on the training data as said before. As far as the dynamic regime is considered, the proposed approach (DIGS + SFDP) can satisfy MIDAS dynamic accuracy requirements (see Section 2) both for training and test maneuvers. Therefore, joining both DIGS and SFDP would lead to a synthetic sensor with enhanced generalization capabilities.

The final verification is performed on steady state conditions in order to check the MIDAS steady state performance of Section 2. Performance of the SS trained with DIGS and SFDP are plotted in Figure 14. Results obtained show a mean error of $0.41$${}^{°}$ and a maximum error of $0.69$${}^{°}$ at $\alpha =6.1{}^{°}$. Even though the steady state performance is not achieved (max error $<0.5{}^{°}$), these results are promising because the SS trained joining DIGS and SFDP methods can significantly improve the performance of the original SS that showed higher mean ($1.03$${}^{°}$) and max ($1.37$${}^{°}$) errors.

Results reported in this section highlight that the proposed approach (based on DIGS and SFDP methods) is valid and it is required when working with real flight test data in order increase the generalization capabilities.

As a side result, the proposed approach leads to an important CPU and time saving for the training stage. In fact, results presented in this section demonstrate that the same performance obtained with a very large amount of data (about $\mathrm{500,000}$ points) as used in the previous work [24] can be achieved and improved with a reduced training pattern (about $\mathrm{13,000}$ points). This aspect is crucial for neural network training and can be considered valid even for simulated flight data. On a common laptop equipped with intel $i7$-$8550U$ and GPU NVIDIA $MX150$ with 2 $\mathrm{GB}$ RAM reserved, the time saving for training 20 NNs is about of $97\%$, in terms of absolute time from $\sim 6000 \mathrm{s}$ to $\sim 200 \mathrm{s}$).

7. Conclusions

With the larger use of digital solution in avionics, analytical redundancy and synthetic sensors are more familiar concepts in flight control systems. The FBW solutions, successfully applied on larger passenger aircraft, will be even used by Small Air Transport community following the path towards a greener aviation. The present work is part of the MIDAS (funded in Clean Sky 2) project that has as final goal to design a simplex and complete ADS architecture partially based on synthetic sensors, exploiting a patented technology named Smart-ADAHRS. The synthetic sensors, based on neural networks, are dedicated to aerodynamic angle estimation. In order to achieve accurate and reliable synthetic sensors, the training stage is a crucial part of the design process. The Smart-ADAHRS technological demonstrator, mounted on a ULM test A/C and used in real time during flight tests, is presented and lesson learned will be transferred to the MIDAS project.

A common issue arising from use of real flight data for the SS training is presented: the ratio of dynamic points to steady-state ones is of the order of magnitude of $\mathrm{10,000}÷1$. This situation leads to over fitting on dynamic flight maneuver with a significant lack of accuracy during the steady-state flight conditions. The flight trials, therefore, highlighted the need of flight manipulation before the learning process in order to achieve a more balanced distribution of the flight data among the n-space dimension of the SS definition. A two step approach is presented in this work to overcome this issue.

The DIGS method starts from the automatic identification of the quasi-stationary and quasi-symmetric situation in the entire flight test database. The method is based on a down sampling technique followed by a piecewise linear multi-variable score assignation. The validity of the method has been assessed with the estimation of $C_{L_{\alpha }}$ and ${\alpha }_0$. The result comparison with respect to values estimated independently by means of classical flight test procedure and semi-empirical methods confirms the validity of the DIGS method. This proposed method worked also in flaps down conditions and the comparison on the corresponding $\mathsf{\Delta }{\alpha }_0$ has been conducted with analytical analysis, showing good accuracy. Although the method showed good performance, the new quasi steady state points are still limited if compared with the entire flight record. By means of using a suitable regression, the $C_L\left(\alpha \right)$ fitting can be used to provide the training pattern with artificial steady-state flight conditions in order to augment the training data set.

On the other hand, a pruning method for (quasi) duplicated training data points is necessary. The SFDP method is based on the observation that real flight data are mainly affected by measurement uncertainties, structural disturbances and sensor noise. The concept of unique flight test record was introduced and it enabled elimination of many quasi-duplicated training points. The SFDP method provides a pruned training pattern that is significantly reduced with respect to the starting one (about $2.5\%$ of the original set).

The proposed approach, based on data augmentation, using the DIGS method, and pruning, using the SFDP method, leads to a new training data set that has the same dynamic information of the starting one with less data records in addition to new artificial data for the steady-state flight conditions. Using the proposed approach, the training pattern is populated by about 500 dynamic points for each steady-state point. In this work, this set-up was considered adequate to guarantee a correct balance between dynamic and steady state accuracies. In order to find an optimum set-up, this particular aspect could be further investigated in future works.

Validation results show that the SS trained with the pruned and augmented data set has a significant performance improvement during the steady-state maneuvers and still acceptable accuracy during dynamic regimes.

To conclude, a synthetic angle of attack estimator based on neural network is introduced for flight control purposes. This work highlights that flight test data for training application are affected by several uncertainty factors that influences the ability of the neural network to generalize introducing some over-fitting issues. The proposed approach based on data pruning and artificial augmentation demonstrates beneficial effects: (1) better distribution within the hypercube where the neural network is defined; (2) consequent improved generalization capability; (3) save CPU time for training.