Maximization of Eigenfrequency Gaps in a Composite Cylindrical Shell Using Genetic Algorithms and Neural Networks

: This paper presents a novel method for the maximization of eigenfrequency gaps around external excitation frequencies by stacking sequence optimization in laminated structures. The proposed procedure enables the creation of an array of suggested lamination angles to avoid resonance for each excitation frequency within the considered range. The proposed optimization algorithm, which involves genetic algorithms, artiﬁcial neural networks, and iterative retraining of the networks using data obtained from tentative optimization loops, is accurate, robust, and signiﬁcantly faster than typical genetic algorithm optimization in which the objective function values are calculated using the ﬁnite element method. The combined genetic algorithm–neural network procedure was successfully applied to problems related to the avoidance of vibration resonance, which is a major concern for every structure subjected to periodic external excitations. The presented examples illustrate a combined approach to avoiding resonance through the maximization of a frequency gap around external excitation frequencies complemented by the maximization of the fundamental natural frequency. The necessary changes in natural frequencies are caused only by appropriate changes in the lamination angles. The investigated structures are thin-walled, laminated one- or three-segment shells with different boundary conditions.


Introduction
For many years, composites have been used for a variety of applications, such as mechanical, aerospace, and civil engineering structures [1][2][3]. However, the dynamic behavior of composite structures has not been widely investigated: composite structures are slender and light compared with steel and concrete structures, and their dynamic behavior may be entirely different. Understanding this behavior is crucial to the effective application of composite materials in structural engineering [1,[4][5][6]. Thus, it is very important to study the dynamic behavior of composite structures in response to changes in material-specific factors (e.g., lamination fiber angles). The structural design process, demand, and the reliability of numerical analysis usually require the optimization of the shape, size, and/or placement of fibers within the material. However, this approach increases the difficulty and complexity of the design process [7] since the design of optimized composite laminate is combined with the search for the best fiber orientation in each layer of composite material. The optimization procedure of laminated composite structures is usually more complex than the one associated with isotropic material structures because of the large number of variables involved and the intrinsic anisotropic behavior of individual layers [8,9].
In optimization problems associated with the design of a composite structure, two scenarios are possible [7,10]: (i) a constant-stiffness design in which the lamination parameters are constant over the whole domain; (ii) a variable-stiffness design in which the lamination parameters may vary in a continuous manner over the domain. The constant-stiffness design scenario is simpler than the variable-stiffness design since there are generally fewer design variables involved [7]. In this study, a constant-stiffness design is applied for a one-segment shell structure, in which the same stacking layup is used over the whole structure, and a variable-stiffness design is applied for a three-segment shell, in which a different stacking layup is used in segments.
In many engineering problems involving dynamic phenomena, the optimization of selected parameters is necessary. Such optimization is usually performed through the minimization or maximization of a carefully selected objective function that describes the distance of particular structure parameters from their desired values (e.g., overall mass, strength, or fundamental natural frequency). The objective function is usually repeatedly solved during the optimization procedure. Moreover, the application of first-order optimization algorithms also involves derivatives of the objective function. In composite laminate design, the precise calculation or even approximation of objective function derivatives is often computationally costly or, in some cases, even impossible. One of the advantages of direct search algorithms (or zero-order optimization algorithms, e.g., Genetic Algorithms, GAs) is that they require only objective function values; no information about the gradient of the objective function is necessary. The single calculation of the objective function is neither difficult nor time-consuming. However, it is extremely computationally demanding when it has to be repeated thousands of times (e.g., when a zero-order search algorithm is applied). Artificial Neural Networks (ANNs) are employed to reduce this demand [11,12]. The task of objective function calculation is herein improved and accelerated by using ANN as a tool to replace the Finite Element Method (FEM) for the prediction of dynamic parameters. The authors of [13] considered a reliability problem for laminated four-ply composite plates in free vibration. The design variables were the material properties, stacking sequence, and thickness of the plies, and the goal of the procedure was to obtain a fundamental natural frequency that was higher than the assumed excitation frequency. The application of neural networks shortened the computation time in the considered problem by more than 40%. Neural networks, trained and tested in advance with FE data almost instantly provide their precise prediction from a big set of candidate solutions, and a GA optimization procedure involving ANNs is thus highly efficient and robust.
In the proposed technique, GA was included to solve the optimization task because functions that describe the dynamic parameters to be optimized have several variables and usually a great number of minima. GA ignores potential local minima and leads to the best solution. This paper presents the maximization of the fundamental natural frequency by determining the optimal fiber orientation in a cylindrical shell's layers of composite material. The lamination fiber angles are treated as constrained variables that are either continuous or discrete. In the context of improving anti-resonance performance, the maximization of the natural frequency of a structure is a primary task in engineering dynamics because it provides the optimal solution to the resonance problem for all external excitation frequencies between zero and the particular optimum fundamental eigenfrequency. Optimization with respect to higher-order eigenfrequencies produces a considerable gap between the specified excitation frequency and the adjacent lower and upper eigenfrequencies. In the offshore oil and gas exploitation industry, risers are indispensable components that are vulnerable to vortex-induced vibration. Vortex-induced vibration strictly depends on the natural frequencies of the risers, so there is a need for optimizing the design of fiber reinforced polymer composite risers by considering different laminate stacking sequences and different lamina thicknesses [14,15]. This approach grants the possibility of avoiding resonance if external excitation frequencies are confined to a large interval with finite lower and upper limits [16].
The optimization of the layer stacking sequence of composite structures (plates, panels, cylinders, beams) for the maximization of their fundamental frequency has been analyzed by many authors.
Using the weighted summation method, the authors in [11] presented a multi-objective optimization strategy for the optimal stacking sequence of laminated cylindrical panels with respect to the first natural frequency and critical buckling load. In order to improve the speed of the optimization process, ANNs were used to reproduce the behavior of the structure in both free vibration and buckling conditions. In [17], the authors reported the optimization of laminated cylindrical panels based on the fundamental natural frequency.
In [18], a study of layer optimization was carried out for the maximum fundamental frequency of laminated composite plates under a combination of three classical edge conditions. The optimal stacking sequences of laminated composite plates were found by means of a GA and ANN procedure. The procedure successfully predicted the natural frequencies of the composite plates, and the predicted frequencies and optimal layered sequences corresponded with those of the Ritz-based layerwise optimization method.
The usual goal in the design of vibrating structures is to avoid resonance of the structure in a given interval of external excitation frequencies. This was discussed in [16], in which the design objective was the maximization of specific eigenfrequencies and distances (gaps) between two consecutive eigenfrequencies of the structures. The results demonstrated that the creation of structures with multiple optimum eigenfrequencies is the rule rather than the exception in topology optimization of vibrating structures. The study emphasized that this feature needs special attention.
The structural optimization problem of laminated composite materials with a reliability constraint was addressed in [19] by using GA and two types of ANNs. The optimization process was performed using GA, and a Multilayer Perceptron or Radial Basis ANNs were applied to overcome high computational costs. This methodology can be used without loss of accuracy and with large computational time-savings (even when dealing with nonlinear behavior).
As a new concept, a Layerwise Optimization Approach (LOA) to optimizing vibration behavior for the maximum natural frequency of laminated composite plates was proposed in [20]. As design variables, the fiber orientation angles in all layers were selected, and the search for the optimum solution in N-dimensional space was replaced by N repetitions of the optimization, with each repetition in one-dimensional space. This idea is based on the physical consideration of bending plates, during which the outer layer has a greater influence on the structure stiffness than the inner layer and is more important in the determination of the natural frequency. The Layerwise Optimization (LO) scheme was also used in [21] to determine the optimum fiber orientation angles for the maximum fundamental frequency of cylindrically curved laminated panels under general edge conditions.
In [22], differential evolution optimization was used to find stacking sequences for the maximization of the natural frequency of symmetric and asymmetric laminates. Optimized stacking sequences for eight-ply asymmetric laminates were presented.
The authors in [23] dealt with the maximization of the fundamental frequency of laminated plates and cylinders by finding the optimal stacking sequence for symmetric and balanced laminates with neglected bending-twisting and torsion-curvature couplings. Vosoughi et al. [24] introduce a combined method to obtain the maximum fundamental frequency of thick laminated composite plates by finding the optimum fiber orientation. To find the optimum fiber orientation of a thick plate, a mixed implementable evolutionary algorithm was used, and the particle swarm optimization (PSO) method was added to improve the specified percent of the GA population. Excellent reviews of techniques applied to the optimization of stacking sequence are given in [7,25].
The use of ANNs in optimization problems in civil engineering have been documented in several other publications. In the vast majority of them, ANNs were applied to reduce the time-consuming evaluations of objective functions that occur in iterative global search algorithms, such as GAs or other evolution strategies [26][27][28]. The authors in [27] examined the application of ANNs to reliability-based structural optimization of large-scale structural systems.
In [26], a critical assessment of three metaheuristic optimization algorithms-differential evolution, harmony search, and particle swarm optimization-was used to find the optimum design of real-world structures. Furthermore, a neural network-based prediction scheme of the structural response, which is required to assess the quality of each candidate design during the optimization procedure, was introduced. In [28], ANN approximation of the limit-state function was proposed in order to find the most efficient methodology for performing reliability analysis in conjunction with the performance-based optimum design under seismic loading. The non-uniqueness of the optimum design of cylindrical shells for vibration requirements was presented in [29], and its implications were discussed in depth. Authors have shown that the design optimization of cylindrical shells for modal characteristics is often non-unique. In the above-mentioned paper, some selected issues, such as the new mode sequence, mode crossing, repeated natural frequencies, and stationary modes, were demonstrated and discussed. The numerical results were compared with the analytically obtained ones.
This paper presents a novel method for the maximization of eigenfrequency gaps around external excitation frequencies through stacking sequence optimization in laminated structures. The proposed procedure makes it possible to create an array of suggested lamination angles to avoid resonance for each excitation force frequency within the considered range. The main task (the maximization of frequency gaps) is preceded by an auxiliary task, namely, the maximization of the first natural frequency of considered structures. The aim of the initial step is to test the proposed procedure by comparing the obtained results with the results of the classical optimization approach, which involves FEM, and the results of an optimization example from the literature that used a different evolutionary optimization algorithm. The results of the comparison verified the proposed procedure. The results obtained using the proposed method are more accurate and significantly less time-consuming-which should be emphasized-because so-called "deep networks" were applied, enabling the use of very large sets, with the number of patterns reaching at least 10,000.

Optimization of the Selected Dynamic Parameters of Laminate Structures
The dynamic behavior of structures whose numerical model is known can be described by the following equation: where M is a mass matrix, C is a damping matrix, K is a stiffness matrix, x is a vector of nodal displacements, and P is a vector of external forces at nodes, whileẍ andẋ indicate the second and the first derivative of x after time t, respectively. For linear models, such as those considered in this paper, matrices M, C, and K are constant over time t, while the vector of external forces (excitations) P and, obviously, the vector of nodal displacements x are variable. If excitation forces do not occur, i.e., P ≡ (0), and the damping is neglected, i.e., C ≡ (0), then Equation (1) is simplified to This equation leads to the generalized eigenproblem [30], where matrix Φ consists of eigenvectors φ i (in columns), and Ω is a diagonal matrix with natural frequencies f i adequate for the vectors φ i .
Knowing the values of natural frequencies f i is crucial to avoiding the resonance phenomenon, which occurs when, for any i, f i ≈ p (p is the frequency of an excitation load). Resonance is an extremely dangerous phenomenon for engineering structures since it leads to structure oscillations with increasing amplitude (theoretically, amplitude can increase to infinity where undamped systems are concerned). In turn, these oscillations cause serious structural failures.
One of the most popular methods used to avoid resonance is to change the structure's geometry, stiffness, and/or mass (see Equation (3)) in order to offset the natural frequencies from the excitation force frequency p. For engineering structures made of laminates, there is a unique approach that cannot be applied to isotropic materials. The stiffness K can be altered without any change to the structure's geometry or mass, and the stiffness can be changed by changing the lamination fiber angles.
As described by Miller and Ziemiański in [31], changes in the lamination angles have a serious impact on natural frequencies. Although some natural frequencies increase and others decrease, in both cases, the changes are relatively big [31].
The avoidance of resonance for a known frequency of the excitation force should, therefore, be possible only by changing the stacking sequence. However, although it would be relatively easy to choose a lamination angle for one or two layers, it is not a trivial matter for three, four, or more layers. This paper presents a novel method for selecting lamination angles to prevent the appearance of the resonance phenomenon using the optimization approach.
The optimization task described above can be written as where Λ is a vector of lamination angles, Λ = {λ 1 , λ 2 , . . . , λ n }; L n is an n-dimensional space of possible values of Λ (herein defined by λ min and λ max , the lower and upper limits of λ i ); and g(Λ) is an objective function to be minimized over the possible values of lamination angles λ i . In this paper, two cases are discussed in detail, i.e., the maximization of the fundamental natural frequency f 1 and the maximization of the distance (in frequency space) between an arbitrarily selected excitation frequency p and the neighboring natural frequencies. In both cases, the presented approach enables the avoidance of a dangerous overlap between the frequency of the external force and any of the natural frequencies, thus avoiding the resonance phenomenon. The objective function is expressed by one of the following equations: for the maximization of f 1 , where f 1 (Λ) shows a relation between the fundamental natural frequency f 1 and the set of lamination angles Λ, and for the maximization of the distance between an arbitrarily selected excitation frequency p and the neighboring natural frequencies. The number of considered natural frequencies, namely, ten, was chosen arbitrarily to cover the range from 0 to 100 Hz in frequency space. In both optimization tasks, the limits λ min and λ max are equal to −90 and +90 degrees, respectively. The optimization discussed in this paper can be finally described by the following formulas: for the maximization of f 1 and for the maximization of the distance between an arbitrarily selected excitation frequency p and the neighboring natural frequencies.
The optimization problems given by Equations (7) and (8) are herein solved using GA, a derivative-free search method. It works on a population of possible solutions and uses both deterministic computations and random number generators. The GA advantage that is crucial from the point of view of the problem to be solved is the ability to search the entire solution space when trying to find a global minimum. This, however, makes it necessary to calculate the value of the objective function repeatedly, which is computationally very challenging when FEM is applied to objective function calculations. In the proposed optimization procedure, the objective function is solved by using ANN instead of FEM calculations, so the GA procedure works extremely quickly.

Genetic Algorithms
Genetic algorithms [32,33] are representative of the group of evolutionary algorithms, whose operating principle is based on the observed evolution of living organisms. The algorithm works on a set of possible solutions (the set is called the population, and the possible solutions are called chromosomes) that are coded as, for example, binary strings (e.g., the number 12 is coded as chromosome 110 in binary format) or real values (12.568 is recalled and processed as 12.568, and there is no transformation into binary strings). For each chromosome, the value of the function being minimized (called the objective function) is calculated. The chromosomes with the smallest corresponding objective function values are then selected as parents of the chromosomes in the next iteration step (called a generation), while the selection strategy may, for example, depend directly on a value of the objective function calculated for each chromosome (the smaller the value, the greater the probability of being selected, i.e., roulette selection) or on a position in a ranking of best solutions (the higher the location in a ranking of all solutions, the greater the probability of being selected, i.e., ranking selection). The selected chromosomes (parents) are then randomly combined to form pairs and exchange some parts of the information they carry. In binary chromosome coding, a single-point crossover of two chromosome parents generates chromosome children: for instance, crossover of 111 and 000 produces chromosome children 100 and 011 (crossover before the second gene) or 110 and 001 (crossover before the third gene  3.521} (crossover before the second gene), where β is a random number. The objective function is then calculated for each chromosome from the second generation, and the best solutions are selected according to selection rules, e.g., roulette selection rules, and are combined to form pairs to produce third-generation chromosomes through crossover. In addition to crossover, some other operators may be introduced to produce children chromosomes, the most popular being mutation, which changes a gene at a random location inside a chromosome to the opposite value; e.g., in chromosome 111, the third gene may be changed through mutation, so the child chromosome becomes 110 (or, for real-value coding, the randomly selected gene g 0 changes into g = g 0 + σN(0, 1), where σ is a random number, and N(0, 1) is a normal distribution with a mean of 0 and variance of 1). While the crossover operator is present in every generation, the probability of mutation should be very low (as it is for living organisms).
The iteration procedure finishes when a stop criterion is fulfilled (e.g., the assumed number of generations is reached or the change in the best objective function value calculated for each chromosome is less than an arbitrarily selected value ε).
Genetic algorithms have numerous advantages: (1) they are capable of solving minimization problems described by objective functions that have a considerable number of local minima, (2) they do not need any objective function derivatives to be calculated, and (3) they are easy to implement.
All GA optimization processes presented in the paper were performed in the Matlab (R2018b, The MathWorks Inc., Natick, MA, USA) environment [34].

Artificial Neural Networks
Neural networks were applied in this study to evaluate the natural frequencies of the considered structure. The most popular neural networks [35] are Multi-Layer Perceptrons (MLPs) or Feed-Forward Neural Networks (FFNNs) and Radial Basis Function (RBF) networks. A basic structure of the feed-forward neural network consists of fundamental processing components, called units (neurons), arranged in layers. Data flow through a neural network via connections between units: data start from the input layer and then proceed through hidden layers, and finally reach the output layer to return the processed data. A neural network with N layers consists of one input layer, N − 2 hidden layers, and one output layer. Its architecture can be described by where I is the number of inputs, H i is the number of neurons in the i th hidden layer, and O is the number of neurons in the output layer. Each neuron performs a linear transformation (by means of coefficients called weights and biases, all gathered in vector w) followed by a nonlinear (usually sigmoidal) transformation prior to transmitting the signal to the subsequent layer. The dimension of vector w can be specified as ANN architecture is partially determined by the problem to be solved. The number of units in the hidden layers, as well as the number of hidden layers themselves, is usually determined by a trial-and-error procedure. The calibration of the ANN (training or learning procedure) is based on P patterns concerning the vector of the selected FE model parameters Λ (herein consisting of lamination angles) and the corresponding eigenfrequency vector f, which is obtained by means of FEM. The computation of the optimal vector of a network's free parameters w * can be formulated as follows: where and y i = ANN(Λ i , w) is the approximation of vector f i given by the ANN (in what follows, this formula is simplified to y i = ANN(Λ i ) in order to focus not on the ANN itself but on the input-output relation). For classical MLP, the optimization problem described by Equation (10) is usually solved by using a back-propagation algorithm designed especially for ANN training.
In the method described in this paper, instead of a standard MLP (shallow networks), so-called deep networks are applied. Training neural networks with more than three hidden layers is called deep learning [36]. Deep learning allows for the application of computational models composed of multiple processing layers. This, in turn, enables the networks to learn data representation with multiple levels of abstraction. Deep networks, in contrast to a standard one-hidden-layer MLP, have a great number of hidden neurons, usually in a few hidden layers. The number of free network parameters gathered in vector w is, therefore, significantly higher than that in the case of MLP application.
Deep learning techniques can be introduced to deal with very large data sets with thousands or even hundreds of thousands of patterns, which is the case in many optimization problems with a large number of variables.
All ANN training and testing processes were performed in the Mathematica (V12.0, Wolfram Research Inc., Champaign, IL, USA) environment [37], and the completed ANNs were then transferred to Matlab.

Investigated Structures
Two models were investigated, and both are perfect cylindrical shells without imperfections or openings that would change their dynamic behavior [38]. The first one, MODEL1, is a cantilever tube, and its model is a thin-walled cylindrical shell with a circular cross-section. The diameter of the middle-surface of the cross-section is R = 0.6103 m, the thickness of the shell is t = 0.016 m, and the length of the cantilever is l = 6.0 m. The shell web consists of a varying number of layers of composite material of equal thickness but with different fiber orientations (Figure 1b shows three layers of composite material). The second investigated model, MODEL2, has the same overall length, l = 6.0 m, but it consists of three two-meter-long segments (see Figure 1c). Again, each of the segments is a thin-walled cylindrical shell with a circular cross-section. The first and the third segments have constant diameters (R 1 = 0.6103 m and R 3 = 0.4103 m, respectively), and the diameter of the middle section changes linearly from R 1 to R 3 . Moreover, the axes of any segment are not parallel to any other segment axes. The shell web of each segment consists of four symmetric layers of composite material. The second model is not a cantilever; there are supports on both ends, with some parts of the outer rings free, some parts supported (no displacements are possible), and some parts fixed (neither displacement nor rotation is possible).

FE Models
The dynamic parameters of the structure, that is, the natural frequencies and modal shapes, were obtained by using the commercial FE code ADINA [40]. The FE model was built using multi-layered shell 4-node MITC4 elements (first-order shear theory), the number of which equals 80 along the circumferential direction and 120 in the axial direction (each shell element is thus about 5 cm long and 4.8 cm wide). The overall number of FE elements, nodes, and degrees of freedom equals 6900, 9680, and 58,080, respectively. The described FE model parameters are based on the authors' previous experience [31]. For the second structure, namely, the three-segment MODEL2 shell, in addition to the above-described dense model, a sparse model was prepared, with a number of finite elements that is four times smaller (40 elements along the circumferential direction and 60 in the axial direction).
The numbers of elements in MODEL1 and MODEL2 were selected carefully after detailed studies of FE convergence. For MODEL1, the adopted FE mesh gives very precise results, and an increase in the number of elements does not change the results of FE analysis. The so-called sparse FE model adopted in MODEL2 FE calculations enables FE calculations that are several-fold faster while the computed natural frequencies remain precise. The maximum difference between the values obtained from the dense and sparse models does not exceed 2-3%. The mode shapes, even when using the sparse model, are computed precisely.

One-Step Optimization
As mentioned above, in order to make the optimization procedure less time-consuming, the values of natural frequencies (gathered in vector f = { f 1 , f 2 , . . . , f 10 }) for the given lamination angles (gathered in vector Λ) reported herein were calculated using neural networks f = ANN(Λ) (instead of the usual FEM calculations) and the GA was applied as a tool to find the lamination angles that yield the maximum value of the fundamental natural frequency f 1 .
The scheme of the one-step optimization procedure proposed herein is shown in Figure 2 and is explained in the following: 1. Initial solution population: Random generation of 100 vectors Λ ∈ L n , where λ min = −90 and λ max = +90. In the jargon of GAs, each vector Λ is called a chromosome, and a set of Λ vectors is called a population. The dimension n of L n space is determined by the number of lamination angles in Λ.

Objective function evaluation:
For each chromosome (Λ vector of λ i lamination angles), the value of the objective function g (Λ) being minimized (see Equations (7) and (8)) is calculated by a trained deep network: g ANN (Λ) = ANN(Λ). 3. Stop criteria: All stop criteria are verified, e.g., the number of iterations is compared with the arbitrarily selected maximum, or the value of the decrease in the objective function in the subsequent iteration steps is compared with the required minimum. 4. GA: selection, crossover, mutation: If the stop criteria are not fulfilled, the usual GA procedure involving selection, crossover, and mutation will produce a new population (a set of new Λ vectors), and the next iteration step begins. 5. Optimal solution: When the stop criteria are fulfilled, the chromosome (Λ * vector) giving the smallest value of the objective function Λ * = arg min {g(Λ)} is treated as an optimal solution; the value of the objective function g (Λ * ) for the solutions obtained during multiple repetitions of the GA+ANN procedure with random initiation (denoted as g ANN (Λ * ) or g ANN for short) are verified by FEM (g FEM (Λ * ) or g FEM for short). FEM is undoubtedly a more accurate method than ANN; therefore, as an optimal solution, the best g ANN resulting from multiple GA+ANN runs is not being chosen. Instead, the maximum value from among all obtained g FEM is chosen (the corresponding ANN prediction for the chosen g FEM is called g FEM (ANN) ).

Initial (random) solutions population
Objective function evaluation (ANN)  The procedure described above lacks one important step shown in Figure 2: the generation of patterns for ANN training and the training itself (in the figure, the step is labeled FEM: examples for ANN training). FE simulations obtain the patterns (or examples) that consist of already known values of the parameters to be obtained (i.e., the natural frequencies gathered in vector f), together with varying parameters (lamination angles gathered in vector Λ). This is the most time-consuming part of the whole procedure. In order to increase time efficiency in comparison with the procedure without ANN application, the number of FE calls during the generation of patterns for ANN training should be as low as possible; otherwise, this approach is pointless. The training of deep networks is relatively fast. In all analyzed cases, training took approximately 30 min on a 4-core system with Intel R Core R i5-4570 CPU@3.20 GHz (Intel Corporation, Santa Clara, CA, USA) and 8 GB RAM and can be considered independent of the number of variables in the range considered herein.

Iterative Optimization (Curriculum Learning)
The one-step optimization procedure may be difficult to apply in a multi-dimensional space L n . Let us take the three-segment MODEL2 shell structure as an example (see Figure 1c). When each segment shell is composed of four symmetric layers of composite material (symmetric from the point of view of lamination angles), the overall number of different lamination angles reaches n = 6. The generation of patterns with a dense λ i mesh may be, in this case, so time-consuming that even sophisticated approaches to the numerical experiment design are of no use.
The novel idea of the optimization procedure to overcome the high-dimensionality problem described above is presented in Figure 3 and consists of the following steps:  The proposed procedure differs from the previously tested one (see Figure 2) in two major aspects: the pre-generation of a relatively small pattern set and the possibility of retraining the applied ANN with new patterns surrounding the optimum solution. The loop shown in Figure 3-application of GA+ANN optimization, generation of a fine grid of patterns around the optimum, and additional ANN training-is herein called Curriculum Learning (CL). The number following CL (e.g., CL2) shows the number of CL loops made (two in the case of CL2).

Verification Example
The optimization procedure described above, with GA as a tool to solve the optimization problem and ANN as a tool that substitutes time-consuming FE calculations, was verified using an example described in [41]. All of the following assumptions (regarding the FE model, material parameters, lamination angles, and so on) are based on this article.
The structure (see Figure 4a) is a cantilever tube. Its model is a thin-walled cylindrical shell with a circular cross-section, and the diameter of the middle-surface of the cross-section is R = 0.  The fundamental natural frequency reported herein was obtained by using the commercial FE code ADINA [40]. The FE model (called MODEL3) was built using multi-layered shell 4-node MITC4 elements (first-order shear theory), the number of which equals 72 along the circumferential direction and 20 in the axial direction. The overall number of FE elements, nodes, and degrees of freedom equals 1440, 1514, and 9084, respectively.
In order to verify the compatibility of MODEL3 with the FE model used by Koide and Luersen in [41], the fundamental natural frequency was calculated for three different stacking sequences described in [41] and compared with the values in that paper. The results, shown in Table 1, prove that the proposed FE model and the FE model in [41] can be considered compatible. The optimization of the lamination angles in order to obtain the maximum value of the fundamental natural frequency f 1 was performed according to the scheme shown in Figure 2. Four pattern sets were generated. The first one, P 1 , consists of 5184 patterns, each composed of 16 angles of neighboring laminate pairs (ANN inputs) and one corresponding fundamental frequency (ANN output). The fundamental natural frequency corresponding to the particular set of lamination angles was computed using FEM. The patterns gathered in P 1 were generated according to two scenarios: The second, the third, and the fourth pattern sets-P 2 , P 3 , and P 4 -consist of 16 randomly generated angles of neighboring laminate pairs (ANN inputs) and one corresponding fundamental frequency (ANN output). The number of patterns in P 2 , P 3 , and P 4 equal 1500, 3500, and 5000, respectively.
The pattern sets were applied in different combinations to train the deep network (see Table 2), and the trained network substituted the FE simulations in the GA optimization of the lamination angles. The results reported herein from the GA+ANN optimization procedure, together with the results presented in [41], are shown in Table 2. The results presented in Table 2 prove that the proposed GA+ANN optimization procedure is capable of solving the optimization problem. Moreover, even when applying a significantly smaller number of time-consuming FE calls, the results obtained by applying the GA+ANN procedure are still better than those presented in [41]. The proposed procedure is able to solve the optimization problem involving half the number of FE calls, which directly leads to considerable time-savings. Analysis of the results presented in Table 1 leads to the conclusion that there are multiple local maxima with f 1 equal to or higher than the value found in [41]. The value of f 1 = 3581.4 Hz seems to be a global extreme.
Special attention should be given to the results in the rows labeled "GA+ANN (CL1)", which were obtained after one curriculum learning loop (see Figure 3). The CL approach was tested with all considered pattern set combinations and, in all of them, gave more precise ANN predictions of f 1 fundamental frequency. The cases shown in the rows "GA+ANN (CL1)" are those in which the CL approach also improved the optimization result. The most important observation is that the maximum of the fundamental frequency, which is higher than the one presented in [41], was obtained with half the number of FE calls (5100 instead of 11,250).

Case Description
This rather simple problem was chosen to check the method's accuracy and time consumption. Moreover, the use of three free model parameters makes the analysis of the behavior of the model's natural frequencies possible. The investigated model, a six-meter-long laminate tube with three layers of composite material, is shown in Figure 1.

Classical FEM-Based Optimization Procedure
The average time necessary for FEM computations to find the first ten natural frequencies for a six-meter-long, three-layer non-symmetric laminate tube fixed at one end reaches approximately 13.9 s on a 4-core system with Intel R Core R i5-4570 CPU@3.20 GHz and 8 GB RAM. This, of course, depends very much on the adopted FE mesh and the applied numerical procedure. However, it can be accepted as a basis for further consideration. The problem of maximization of the fundamental natural frequency f 1 needs the objective function, that is, g(Λ) = − f 1 (Λ), to be solved about 180 times on average. The parameters of the applied interior-point optimization algorithm (see [34,42]) are as follows: no constraints, no gradient nor Hessian supplied, maximum number of iterations equal to 100, 50 repetitions with a unique random starting point, and each calculation of the objective function needs an FE call. The exact number of objective function evaluations varies from 4 to 592, and the time necessary to perform f 1 maximization reaches an average of 25 min. However, for a particular case, it varies from 33 to 85 min (the time covers the entire optimization procedure, including all necessary FE simulations).
Each random initiation of the procedure leads to a different solution (and probably to a different local minimum). For 50 multi-start random initiations, the best-obtained result is f 1 = 37.786 Hz for lamination angles Λ * = [50.9/ − 4.5/77.7]. The overall time necessary to repeat the procedure 50 times and the number of FE calls are 21 h and 9061, respectively. The result of calculations involving classical, derivative-based optimization methods and FEM to obtain the objective function value is a reference point for the proposed GA+ANN optimization procedure. The genetic algorithm needs the objective function to be computed thousands of times, and for the maximization of f 1 , the time necessary to obtain the results using FEM to calculate the objective function value would be 28 days. Obviously, this is an unacceptable time investment.
In order to make the GA procedure applicable with limited time constraints and typical computer hardware, the evaluation of natural frequencies was not done through FEM. Instead, neural networks were applied since, in contrast to FE simulations, a trained neural network is capable of giving its prediction almost instantly.

Pattern Generation and ANN Training
The procedure described above was tested on the MODEL1 structure (see Figure 1), with three layers of composite material and dense FE mesh applied during FEM computations. Patterns for ANN training were calculated using the FE ADINA code [40]. Each one is composed of three lamination angles Λ = {λ 1 , λ 2 , λ 3 }, one for each layer of the tube shell, and the corresponding natural frequencies of the tube f = { f 1 , f 2 , . . . , f 10 }: where the natural frequencies in pattern number j were obtained through FEM computations for lamination angles Λ j : The number of natural frequencies computed amounted to ten in order to make the second optimization task (see Equation (8)) solvable. The patterns were generated on a regular grid in L 3 space; that is, natural frequencies of the laminate tube were computed for all parameter vectors Λ = {λ 1 , λ 2 , λ 3 }, with λ i varying according to one of the scenarios gathered in Table 3. Only the last pattern set P 5 was generated with random values of lamination angles. The generation of patterns, together with ANN training, formed the longest part of the simulation (in Figure 2, it is labeled FEM: examples for ANN training), which predicts the actual optimization. The penultimate column in Table 3 shows the duration of the generation of a particular P i pattern set. The average time necessary to compute one pattern (the last column in Table 3) differs for particular sets since, in some cases, the method applied to solve the generalized eigenproblem (Equation (3)), namely, the subspace iteration method (see [30]), has to calculate a number of natural frequencies that is greater than the desired one (i.e., ten).
Pattern set generation is the most time-consuming part of the proposed GA+ANN procedure. The number of patterns should be large enough to accurately describe the function f j = FEM(Λ j ) and simultaneously small enough to ensure that the time necessary for their generation is acceptable. Obviously, the number of FE calls during pattern set generation should be significantly smaller than the number of FE calls during the classical minimization procedure (here that number is 9061) and the assumed number of FE calls during GA+FEM optimization (50 repetitions of GA optimization with 100 iterations of 100-element generation: 500,000 FE calls in total). In the considered case, the largest acceptable number of patterns was assumed to equal about 3000-4000. Some preliminary calculations showed that, instead of one pattern set generated using a regular grid (e.g., λ min = −90, λ max = 90, ∆λ = 12), it is advisable to generate a few small pattern sets, each generated using a different grid and completed by a randomly generated pattern set. The considered pattern sets, described in Table 3, follow this preliminary conclusion. One of the goals of the following is to find a solution of a given optimization problem using the minimum number of patterns. In the following experiment, five different learning sets were considered. In each case, the ANN was trained and tested on a different combination of P i sets (see Table 4, the first column). In the last row of the table, the network ANN 245 is described. All available patterns were used to train this network, so no testing was performed. The overall time consumption shown in the last column in Table 4 includes the time spent both on the generation of the learning and testing patterns as well as on network training. The errors shown in Table 4 are commonly used error measures in ANN training, i.e., standard deviation σ, Mean Squared Error (MSE), and f 1 prediction Average Relative Error (ARE 1 ). Assuming that y is the ANN prediction of natural frequencies, y = {y 1 , y 2 , . . . , y 10 } = ANN (Λ) .
MSE and ARE 1 are given by the following formulas: where P is the number of testing patterns (see Equation (10)). The results shown in Table 4 prove that the applied deep networks are properly trained and can predict the values of the natural frequencies of a laminate tube with very high accuracy. However, the accuracy is significantly lower in two cases, namely, ANN 5 (trained using random patterns only) and ANN 13 (trained using a very limited number of patterns).

Maximization of the Fundamental Natural Frequency f 1
The maximization (optimization) of f 1 was performed by applying the procedure shown in Figure 2, with the following assumptions: • the lamination angle increment is expressed either in real numbers (in the tables with the results coded as '0'), integer numbers ('1'), or integers divisible by 2, 5, or 15 ('2', '5', or '15', respectively); • the usefulness of all networks listed in Table 4 is verified; • the GA optimization procedure for each lamination angle increment ('0', '1', '2', '5', or '15') and for each network (from ANN 13 to ANN 245 ) is repeated 50 times with different random initial populations.
Altogether, the optimization procedure was repeated 5 × 6 × 50 = 1500 times. The results obtained are illustrated in Figure 5. Figure 5 presents the so-called box-and-whisker diagram with the following metrics used to describe the results of 250 runs (five different lamination angle increments and 50 repetitions in each case) of f 1 maximization for a particular ANN applied: minimum and maximum values of the obtained results (whiskers); the interquartile range, representing the results between the 75th and 25th percentiles (the colored boxes); and the mean (the white lined inside the colored boxes). The box-and-whisker diagram was created using the f max 1 = g FEM results. Of six verified networks, only one, ANN 13 , gave a mean result of f max 1 = g FEM that is lower than 36.97 Hz (the maximum value of f 1 available in the data sets). Four of the six networks gave a minimum of f max 1 = g FEM of almost 36.97 Hz, with the mean and the maximum clearly higher than 36.97 Hz. The best results are from applying optimization procedure using the ANN 1235 network: the value f 1 = 37.77 Hz is almost 1 Hz higher than the value that can be found in rather dense sets of patterns.
The proposed optimization approach can be adopted for both real values of lamination angles and integer values with a user-defined angle increment. The rather narrow spread of results, e.g., those from ANN 4 , indicates the robustness of the proposed procedure. The time consumption of one optimization case (for example, using the network ANN 4 , a lamination angle increment of '0', and 50 repetitions was one of the most time-consuming cases investigated) is 802 min for pattern generation, 15 s for GA+ANN optimization, and 22 min for FEM verification of the obtained results for a total time of slightly less than 14 h. The number of FE calls is 3413 during pattern generation and 50 during FE verification of GA+ANN results for a total of 3463 FE calls. For the network ANN 1235 , whose application generates the best-obtained results (for the increment '0') of f 1 = 37.67 Hz and Λ * = [51.5/ − 6.33/77.5], the overall time consumption is 9 min less, and the number of FE calls remains constant. In a reference case (classical optimization involving FEM), the best-obtained results are f 1 = 37.786 Hz and Λ * = [50.9/ − 4.5/77.7], and the overall time and the number of FE calls are 21 h and 9061, respectively.

Maximization of the Distance between Natural Frequencies and a Selected Frequency
The main optimization task in the proposed approach is the maximization of the distance between an arbitrarily selected excitation frequency p and the neighboring natural frequencies (see Equation (8)). The considered values of p are 30, 40, 50, 60, 70, and 80 Hz.
The minimal difference between any natural frequency and any of the arbitrarily selected excitation frequencies is significantly higher than the targeted 10% of the excitation frequency. Actually, in every case, it reaches 30% of the excitation frequency p, whereas the minimum difference |p − f i | for every i should be higher than 10% of p to avoid the resonance phenomenon.
The results obtained from the ANN 1235 network are presented in a graphical form in Figure 6. Successive rows in Figure 6 present the results of f 1 maximization (the bottom row) and the maximization of the distance between arbitrarily selected excitation frequencies (30-80 Hz) and natural frequencies calculated for MODEL1.
The green boxes represent the distance in the frequency space around a selected excitation frequency  Hz) that is free from natural frequencies and obtained when a particular set of lamination angles is applied. The blue vertical lines represent the natural frequencies calculated for the same lamination angles. It is clearly visible that the ANN 1235 network gives the expected results. Table 5 shows a kind of lookup table that enables the quick selection of lamination angles that lead to resonance avoidance for any excitation force frequency up to about 72 Hz. For example, for the excitation frequency p = 45 Hz, the suggested lamination angle set [−64.5/61/−29.4] leads to a distance from p = 45 Hz to the closest natural frequency f i of at least 17.4 Hz (or, expressed as a percentage, 32.1%). No calculations are necessary for resonance avoidance once Table 5 is ready. Both problems, namely, the maximization of f 1 and the maximization of frequency gaps (eight optimization cases in total), were successfully solved using the ANN 1235 network. The network ANN 1235 was trained and tested using only 3413 FE calls during pattern set generation and 400 FE calls during the verification of obtained results. If it is assumed that the number of FE calls necessary to perform the same tasks using the classical optimization approach exceeds 70,000 (eight optimization cases, each needing about 9000 FE calls), then, in the GA+FEM approach, the number of FE calls reaches 4,000,000. The number of FE calls in the application of the proposed GA+ANN procedures decreases by at least one order of magnitude compared with the classical optimization approach.

Optimization of MODEL1 Structure with a Different Number of Layers
The optimization of the fundamental natural frequency f 1 and the distance between an arbitrarily selected excitation frequency p and the neighboring natural frequencies was performed with the number of layers equaling 2, 3, 4, 6, and 8. In three cases, including the one described above, the classical FEM-based optimization procedure was also applied to verify the results obtained from the GA+ANN procedure (see Table 6 and Figure 7 for details). The results obtained from the GA+ANN procedure are at least as accurate as the ones obtained from the classical FEM-based procedure (for the more complicated examples, they are even slightly better), though the number of FE calls is significantly smaller. Table 6. Results of f 1 optimization using classical and proposed procedures using MODEL1 with a different number of layers.  Figure 7. Optimization of f 1 the MODEL1 structure with a different number of layers using classical minimization techniques. The results of f 1 maximization show the stabilization of the fundamental natural frequency value; further increasing the number of layers does not result in an associated increase in f 1 . The same conclusion can be drawn when maximizing the distance between natural frequencies and a selected excitation frequency.

Pattern Generation and ANN Training
The proposed CL optimization improvement (see Figure 3) was tested on the MODEL2 structure (see Figure 1c Two groups of pattern sets for ANN training were calculated using the FE ADINA code [40]. Each was composed of lamination angles Λ = [λ 1 /λ 2 |λ 3 /λ 4 |λ 5 /λ 6 ] s and the corresponding fundamental frequency of the structure f = { f 1 } or ten corresponding natural frequencies of the structure f = { f 1 , f 2 , . . . , f 10 }: where the fundamental frequencies were obtained through FEM computations. The details concerning pattern generation are summarized in Table 9. The length of time required to generate one pattern is shorter than that in the previous case of MODEL1 optimization since the sparse FE mesh was applied (the number of finite elements is four times smaller than that in the case of the dense FE mesh).  Two deep networks were trained for f 1 maximization, and the difference between them is the number of learning patterns: ANN 1 3 was trained with 4096 patterns, while ANN 1 123 was trained with 8,921 patterns. Another two, ANN 10 123 and ANN 10 1234 , were trained for the maximization of the distance between natural frequencies and a selected frequency.

Maximization of the Fundamental Natural Frequency f 1
The maximization (optimization) of f 1 was performed by applying the procedure shown in Figure 3, with the following assumptions: • the lamination angle increment is expressed only in real numbers, and other approaches are not tested, • the usefulness of the two networks (ANN 1 3 and ANN 1 123 , see Altogether, the optimization procedure was repeated 19 × 50 = 950 times. The results obtained are shown in Figure 8 and Table 11. The total number of GA generations (see Table 11, the third column) was not only G = 100 (as it was for every case of MODEL1 optimization) but also G = 50 and G = 75. The number of generations was increased to G = 200 in only one case (the last row of Table 11) in order to check whether such an extension of the procedure would be expedient. Since there are no observable improvements with this increase, G = 100 is treated as the appropriate value.  Figure 8. Optimization of f 1 using MODEL2 with four symmetric layers of laminate.  Figure 8 shows a clear improvement in the results, especially for ANN 1 3 , which was trained on the data set P 3 , which is half the size of the training set P 1 + P 2 + P 3 used for the second network, ANN 1 123 . The final results obtained from both networks are comparable. However, the time required to generate the necessary pattern set, train the networks, and perform the optimization procedure is significantly different: 792 min (13.2 h) for ANN 1 3 and 1073 min (17.9 h) for ANN 1 123 . The mechanism of improvement during three CL loops in the application of ANN 1 3 is clearly visible in Figure 9. The results obtained from ANN 1 3 and ANN 1 123 are visualized in a manner that is typical for the presentation of neural network accuracy: f ANN 1 versus f FEM 1 (values predicted versus desired). Obviously, the better the network training, the closer the points-i.e., the networks' results-to the diagonal y = x line. There are three types of points in the figure: circles represent the results of the pretrained network (trained using the initially generated pattern set P 3 ), triangles represent the results of the networks retrained using additional patterns obtained after the first CL loop, and, finally, the upside-down triangles represent the results of the networks retrained again using additional patterns obtained after the second CL loop.
Larger values on the horizontal axis in the subsequent steps of the optimization algorithm (CL0, CL1, and CL2) are indicative of additional patterns gathered after successive loops approaching the desired extreme. The decreasing distances from the diagonal y = x line mean that the retrained networks acquire a higher level of accuracy around the global extreme. Figure 9a presents a visible improvement in ANN 1 3 after each of the CL loops, and the ANN 1 123 network (see Figure 9b) clearly improves after the CL1 loop. The next loop CL2 does not significantly improve the maximum f 1 value, but it increases the network accuracy for the lower values of f 1 .  Figure 10. Optimization of the distance between natural frequencies and 120 Hz using MODEL2 with four symmetric layers of laminate.

Conclusions
This paper presents a novel method for the optimization of the stacking sequence in laminated structures. The proposed method is robust and efficient in simple optimization cases as well as in more complicated problems with more design variables. Complex boundary conditions, the complicated shape of the investigated structures, and the type of variable (continuous or discrete) do not hinder the effective use of the method. Moreover, the procedure can be successfully applied to optimization cases in which it is not possible to derive analytical formulae that combine design variables and optimized quantities.
As a result of replacing the finite element method by deep neural networks for the calculation of the optimized structure parameters, the proposed method significantly accelerates the optimization and enables the optimization to be performed in limited time on typical computer hardware. In addition to instantly calculating the objective function, deep neural networks have another important advantage: they facilitate the use of a very large number of patterns describing the optimization space. In the examples described in this paper, the number of available patterns reaches 10,000. In other cases, which are not discussed in this paper, deep networks were trained on 120,000 patterns, and the training time did not exceed 30 min.
The main idea presented in this paper, i.e., the curriculum learning of the applied deep networks, opens the door to new applications of the combined GA+ANN optimization procedure. The procedure can start with a very sparse description of the optimization space, and, in the successive iteration steps, the space description acquires a higher level of accuracy around the global minimum. The increasing accuracy of the objective space description and the possibility of deep network retraining presents the opportunity to create a very precise neural approximation of optimized structure parameters and, therefore, to significantly increase the precision of GA+ANN optimization.
The combined GA+ANN procedure was successfully applied to problems related to the avoidance of vibration resonance, which is a major concern for every structure subjected to periodic external excitations. The presented examples show two approaches to resonance avoidance: the maximization of the fundamental natural frequency (for external excitation frequencies possibly smaller than the fundamental natural frequency) or the maximization of a frequency gap around the external excitation frequencies (in other cases). In all examples presented in this paper, the necessary changes in natural frequencies are the result of appropriate changes in lamination angles only. In other words, no changes in the boundary conditions, geometry, or mass of the structures are introduced.
A further study of the proposed optimization procedure would be of interest. The proposed problems to be discussed include the following: • vibration resonance avoidance in the case of a few periodic excitations with different frequencies; • multi-criteria optimization, e.g., the maximization of the fundamental natural frequency, together with the maximization of the buckling force; • composite material design (e.g., topological constraints, varying thickness and/or stiffness); • a full variable-stiffness approach to lamination parameters that considers continuous changes in lamination parameters over the investigated domain;

Conflicts of Interest:
The authors declare no conflict of interest.