Generative VS non-Generative Models in Engineering Shape Optimization

In this work, we perform a systematic comparison of the effectiveness and efficiency of generative and non-generative models in constructing design spaces for novel and efficient design exploration and shape optimization. We apply these models in the case of airfoil/hydrofoil design and conduct the comparison on the resulting design spaces. A conventional Generative Adversarial Network (GAN) and a state-of-the-art generative model, the Performance-Augmented Diverse Generative Adversarial Network (PaDGAN), are juxtaposed with a linear non-generative model based on the coupling of the Karhunen-Lo\`eve Expansion and a physics-informed Shape Signature Vector (SSV-KLE). The comparison demonstrates that, with an appropriate shape encoding and a physics-augmented design space, non-generative models have the potential to cost-effectively generate high-performing valid designs with enhanced coverage of the design space. In this work, both approaches are applied to two large foil profile datasets comprising real-world and artificial designs generated through either a profile-generating parametric model or deep-learning approach. These datasets are further enriched with integral properties of their members' shapes as well as physics-informed parameters. Our results illustrate that the design spaces constructed by the non-generative model outperform the generative model in terms of design validity, generating robust latent spaces with none or significantly fewer invalid designs when compared to generative models. We aspire that these findings will aid the engineering design community in making informed decisions when constructing designs spaces for shape optimization, as we have show that under certain conditions computationally inexpensive approaches can closely match or even outperform state-of-the art generative models.


Introduction
The design process is a critical phase for any industry, and it can be revolutionized by incorporating state-of-the-art intelligent methods.This integration not only automates design processes but also aids designers in creating innovative and optimized solutions for free-form functional surfaces, such as wings, turbine blades, and ship hulls.Optimizing such surfaces often plays a crucial role in enhancing their functional performance; see for example [1,2,3,4].An effective design parameterization, facilitating high levels of intuitiveness, flexibility, and representational accuracy, is a crucial prerequisite for such shape optimization approaches.Intuitiveness is essential for enabling designers to articulate the design logic, while flexibility is crucial for accommodating intricate design specifications.Representational accuracy ensures that a concise set of design parameters can effectively capture an expansive design space, encompassing physically optimal solutions across a diverse range of design conditions and constraints [5,6].However, using traditional methods to create a design space that accommodates these three qualities often results in prohibitively high dimensionality and increased complexity.
In the realm of engineering design, the methods and processes for generating appealing and optimized designs have continuously evolved, mirroring technological advances and paradigm shifts in design approaches.For instance, the recent growth of generative methods within engineering design disciplines has contributed significantly to the automation of the design generation process.These models have the ability to extract and capture the underlying data distribution of the design space, enabling them to generate conventional as well as novel design samples; see [7,8,9].Notably, deep generative models (DGMs).such as generative adversarial networks (GANs) [10], variational autoencoders (VAEs) [11], and deep reinforcement learning techniques, have found applications in diverse domains such as microstructural design [12], 3D modeling [13], and aerodynamic shape design and optimization [14].
Despite of these developments, conventional generative models, primarily focused on learning the distribution of an existing design space, often encounter significant challenges when applied in engineering design synthesis.These challenges relate to reduced diversity, sub-optimal performance, and a lack of novelty, which can be primarily attributed to limitations of the employed design space [15,16,17].To address these issues researchers have proposed a series of advanced algorithms [18,19,20].Notably, Chen et al. [21] proposed the performance augmented diverse generative adversarial network (PaDGAN), incorporating a loss function based on determinantal point processes (DPPs) [22,23].This approach aims to synthesize high-performance and diverse designs while extending the boundaries of the existing design space for the generation of novel designs.
While generative approaches show promise, they often come with significant computational costs when dealing with complex designs.In contrast, non-generative approaches, such as Principle Component Analysis (PCA) / Karhunen-Loève Expansion (KLE) [24,25], can be tailored, by appropriate methodological expansions and augmentations, to approach (and in some cases, even outperform) the performance of generative models while being computationally more efficient, as we will describe in sequel.The dichotomy between generative and non-generative models becomes evident in their approaches to data representation and design generation.Generative models focus on capturing the underlying data distribution and generating novel and diverse design samples, while non-generative models solely focus on extracting latent features from the design space without explicitly modeling the underlying data distribution.Classic non-generative methods face limitations in preserving intricate shape complexity and underlying geometric structures.This leads to latent subspaces which may not permit the efficient generation of diverse and valid shapes during shape optimization [26].The compromised representational capacity impedes optimizers and wastes computational resources in the exploration of infeasible designs and/or shapes lacking novelty.Moreover, these techniques rely predominantly on geometric features, neglecting crucial quantities of interest, pertaining to physics and performance, which actually drive the design optimization process.Consequently, along with the lack of a prior probability distribution, the generated designs tend to be either close replicates of the original dataset members or inter-member interpolants lying in their respective neighborhoods.Therefore, there is a wide variety of challenges to be addressed before contemporary design process objectives can be met.
While PCA / KLE techniques may be computationally efficient, their linear nature and susceptibility to generating less diverse design spaces with a relatively high number of invalid designs make them less effective when compared to other modern non-generative models.For example, auto-encoders have demonstrated their ability to produce diverse design spaces with a low number of invalid designs while also capturing nonlinearities within datasets [27,28,29].To address the above-mentioned limitations of conventional linear non-generative approaches, Khan et al. [26] introduced an augmented shape signature vector (SSV) coupled with KLE-based approach and managed to improve the original design space representation by incorporating both geometric and physical information in the design description.Enhanced performance has been demonstrated in their study without sacrificing computational efficiency.A further step in the same direction was performed in Masood et al. [30] where the effect of different shape-discretization methods was highlighted and by using a similarly augmented SSV they showcased the positive impact of an enhanced data representation which tackled the problems of invalid designs and lack of diversity.
These recent results motivated us to conduct a comparison between a representative of the state-of-the-art generative models with the enhanced KLE-based non-generative model mentioned above.Specifically, non-generative models are represented by the SSV-KLE-based approach, which is an enhanced linear shape-supervised dimension reduction approach (see §2.1), whereas PaDGAN (see §2.2) is the generative model of choice.PaDGAN is a nonlinear method with a nominal two-fold advantage over non-generative models, i.e., it captures nonlinearities and learns the underlying data distribution.The comparison is performed on airfoil design spaces which align very well with our research aims, as airfoil design requires rich design spaces with adjustable parameters influencing performance.At the same time, due to their 2D nature, their performance evaluation does not require prohibitively costly computations and therefore permits a thorough investigation into how generative and non-generative models cope with the intricacies of complex design spaces, and performance-based shape optimization within them.Thus, we can cover both design space quality and design performance assessment that produce valuable insights into the respective models' capabilities and limitations.The effectiveness and efficiency of both generative and non-generative models are significantly affected by the representation of the design dataset which plays a pivotal role in each model's capacity to capture relevant features and patterns within the design space.We aspire to demonstrate that, with appropriate data representation, non-generative models can achieve results on par with those of generative models.Our comparison will be facilitated by the following major steps: • Generation of datasets with varying shape signature vectors (with and without augmentation with performancebased components).
• Performance of varying shape discretizations to quantify their effects as well as identify the ones that lead to data representations with enhanced quality.
• Deploy both generative and non-generative models on the created datasets.
• Perform a comprehensive analysis of latent space quality to evaluate the efficacy of the implemented models in design optimization.
This paper is divided in two main sections: section 2, where the employed models and comparison criteria are discussed, and section 3, containing the produced results and their analysis.Specifically, we begin by describing the enhanced SSV-KLE-based approach in section 2.1, followed by the presentation of the PaDGAN model in section 2.2.In the same section ( §2), the signature vectors, the respective enhancements as well as the dataset generation procedures are also discussed.Section 2 concludes with the description of the quality metrics employed in the design space evaluation; see section 2.4.The constructed models' and datasets' comparisons are subsequently presented and discussed in section 3, followed by a summary of main observations and future research directions in section 4.

Methods
This section begins with the presentation of the selected non-generative approach in section 2.1, followed by the employed generative model in section 2.2.We then describe the dataset generation process for the two datasets in section 2.3, and conclude with the presentation of the quality metrics employed in the models' comparison in section 2.4.

Shape-Supervised Dimension Reduction (SSDR)
The non-generative SSDR employed in this work is adopted from [30], which combines a Karhunen-Loève Expansion (KLE) approach with a shape signature vector (SSV -see also [31]) that is augmented with physics-informed quantities (mainly, Geometric Moments; see section 2.1.1)and considers varying discretization methods in shape encoding.In this context, a rich and diverse space of foil-profile designs, denoted as C, is assumed with each design being represented or modified using a design vector v ∈ V ⊆ R n .The design space V is constrained by an appropriate set of bounds which limit the space to geometrically and physically valid foil profiles.
Vector v ∈ V facilitates the definition of a shape modification procedure θ † = θ + v θ, where θ denotes the initial foil geometry, discretized into a set of points that are encoded into this vector of point coordinates, whereas vector θ † corresponds to the resulting vector encoding of the shape after applying the modification procedure.For the generation of the augmented SSV, we combine the geometry, vθ, with a vector of physics-informed quantities, µ(vθ), to form the final unique SSV, ϑ: where the function p (ϑ) incorporates both geometrical and physics-informed information, and n p = n + n µ , with n corresponding to the dimension of V, and n µ to the number of physics-informed quantities employed in the augmentation.
To reduce the computation cost, quantities that are related to performance instead of actual performance metrics can be used.In this work, we mainly use the foil's geometric moment invariants (see section 2.1.1)but performance metrics, such as lift and drag coefficients are also used.Finally, the KLE approach allow us to determine an appropriate set of orthonormal basis functions, {ϕ i (ϑ)} κ i=1 , which will be used in the approximation of the initial design space, i.e., where {ϕ i (ϑ)} κ i=1 span the latent space U, and u = (u 1 , u 2 , . . .u i , . . ., u κ ) is the vector of latent parameters and κ is the number of eigenvectors that retain the required percentage of total variance in the given dataset.For the calculation of the eigenvectors / basis functions, the approach discussed extensively in [25,26,30] is adopted in this work.The interested reader may specifically study the full derivation of this approach for the case of airfoil design spaces in [30].
Apart from the KLE method mentioned above, we still need to briefly describe the Geometric Moments Invariants which are mainly used for the augmentation of the SSV (see section 2.1.1,and the bounds, (u low , u high ), we employ for the resulting latent spaces; see section 2.1.2.

SSV augmentation -Geometric Moments
As mentioned before, SSV augmentation is performed with the use of a series of physics-informed quantities that lead to significant quality enhancements in the resulting latent spaces as demonstrated in [26].This approach address the limitations of conventional dimensionality reduction approaches, which often fail to preserve the full complexity of shape and the underlying geometric structure.One obvious approach for the application at hand is to use performance metrics, as lift and drag coefficients, to augment the SSV.However, such metrics can become computationally expensive, and therefore, following the insights of Khan et al. [26] and Masood et al. [30], we introduce geometric moment invariants as physical information substitutes.This addition not only encompasses additional integral geometric characteristics but also incorporates relevant physical properties of the designs, as geometric moments exhibit a strong correlation with common performance metrics in airfoil design.
If we use Ω to denote the 2D domain enclosed by a given 2D foil profile, the r th -order moments can be calculated using the following general equation.
(3) In this expression, the "density" function ρ(x, y) assumes the value 1 when (x, y) ∈ Ω and 0 otherwise.However, as one may easily observe, the moments in Eq. (3) depend on shape's rigid motions whereas the common relevant performance metrics, i.e., lift and drag coefficient, are invariant to translations and uniform scaling.For that reason, appropriate scale and translation invariant moments should be used if we want to avoid introducing noise and nonrelevant information in the SSV.Moments that are invariant to translations, rotations and scaling have been presented in [32].Since, rotational-invariance is unwanted, we only employ of normalized version of the central moments which deliver the needed invariance with respect to uniform scaling and translations.Specifically, central moments are defined as where c = (c X , x y ) corresponds to the centroid of Ω.Finally, we proceed with normalization to eliminate the scaling influence.This can be done by dividing with any of the moments, but picking a low order one is computationally more stable.Hence, if we pick M0,0 for the normalization, we finally get with µ p,q being the main quantities augmenting the SSV in this work.For a more detailed discussion regarding geometric moments and their invariants, the interested reader may refer to [32,30,26].

Latent Space Bounds
Design space bounds are generally easy to be determined, especially when their generation is performed by parametric models that employ parameters with physical meaning.These bounds are of utmost importance as they limit the design space to regions producing valid geometrical profiles, hence excluding regions that would produce infeasible and/or invalid designs which would obviously impede the design optimization process.However, determining the bounds of latent parameters is a daunting task as latent parameters have no physical interpretation.Nevertheless, setting appropriate bounds for the latent space is still a crucial step as regions with infeasible or invalid shapes need to be excluded or minimized so that optimizers are not trapped in irrelevant design regions.At the same time, overly tight bounds may negatively effect the design space since they undermine the potential of generating rich spaces with novel designs.Specifically, in this work, although it is relative easy to derive bound for the parameters of the v vector in V, the same cannot be said for the latent vector u ∈ U.Although various methods are proposed in the pertinent literature it is still hard to determine the values of (u low , u high ) in a way that would guarantee the satisfaction of all design requirements, i.e., diverse and rich design space with no invalid/infeasible designs.One approach entails the projection of the original design space bounds to the latent space which although feasible may over-constrain the latent space and exclude large useful regions.Another approach, which is computational inexpensive and is commonly used in the pertinent literature, involves the use of standard deviation for the mean design, placed at the center of the latent space.This approach achieves a good compromise between the aim of contracting regions with invalid designs and the requirement of a rich and diverse design space.Specifically, this approach involves the eigenvalues {λ i } κ i=1 identified when calculating the basis functions in Eq. (2).
where α is a commonly a whole number ranging from 1 to 3 determining the number of standard deviations around the mean space which will be used in the definition of the design space.We should also note here that variance is represented by the sum of all eigenvalues, i.e., σ 2 = ∞ i=1 λ i , which is also used for the determination of the number of eigenvectors sufficient for capturing the required percent of total variance, i.e., where β% is the required variance in the latent space.

PaDGAN: Performance Augmented Diverse Generative Adversarial Network
Traditional GANs [33] consist of two neural networks: a generator G and a discriminator D that are trained simultaneously in an adversarial mode with the following objective function, including both generator and discriminator loss terms.
where x represents a sample of real data, drawn from the data distribution P V , z is a random noise vector drawn from the noise distribution P z , while D(x) represents the discriminator's output when evaluating real data distribution and D(G(z)) is the discriminator's output when evaluating generated data.In other words, G aims to minimize the objective, whereas D aims to maximize it.Conventional GANs do not perform well when the real-world functional performance of designs and their physical feasibility for fabrication is taken into consideration [15].Besides that, GANs often suffer from mode collapse [19], which means that G focuses on producing a limited set of designs deceiving D without being able to produce the full range of To address these issues, PaDGAN algorithm [21] measures diversity and quality during training by incorporating a loss function which is based on a performance-augmented Determinantal Point Process (DPP).DPPs are probabilistic models which are designed to efficiently subsample large sets of data and are well aligned with the objective of promoting design diversity without sacrificing quality.To model diversity and quality simultaneously, the performance-augmented DPP loss gives a lower value for both high-performance and diverse designs.Specifically, if we consider a DPP kernel matrix L B for a generated set of designs, B, each element can be written as where k(x i , x j ) is the similarity kernel between two designs, x i and x j , and q(x) is the performance function evaluated for x.The exponent γ 0 is added to control the contribution of the design's performance, i.e., a value of γ 0 = 0 will obviously eliminate performance contributions while a large exponent value will promote high-quality designs and undermine the diversity term's effect.Using now Eq. ( 8), the performance-augmented DPP loss function can be written as where λ i is the i th eigenvalue of the kernel matrix L B for the design set B. Finally, by including this loss term to the initial GAN objective function (see ( 7)), we derive the PaDGAN objective function: where γ 1 controls the contribution of the performance-augmented DPP loss of the generator.

Datasets generation
This study utilizes two datasets which are both derived from the publicly available UIUC foil designs database [34].
In both cases the approximately 1600 foil profiles residing in the UIUC database are enriched with a large number of artificial designs that are produced either by perturbations of the parametric model described in the sequel (dataset D 1 ), or synthesized by the Bézier-GAN approach described in [35] (Dataset D 2 ).

Parametric Model
The parametric model for airfoil/hydrofoil generation, initially presented in [2] and subsequently extended to encompass a broader range of designs in [36,37], has been extensively used in the generation of D 1 .We chose to utilize the parametric model proposed by Kostas et al. in [36] because it is specifically designed to meet the particular requirements of design optimization, i.e., guaranteed generation of valid foil geometries using parameters with physical interpretation and high representational capacity as it can approximate within Kulfan tolerance [38,6] all profile designs residing in UIUC database.The adopted foil parametric model in this work generates each foil profile instance as a cubic NURBS curve of order 4 with 13 control points from a nondimensionalized design vector p ∈ P ⊂ R 17 with P ∈ [0, 1] 17 , as is depicted in Fig 1 .For a detailed description of the construction and parametric definition of the foil profile, readers are encouraged to refer to [36].
The process initiates by approximating the foil profile designs in the UIUC database with the abovementioned parametric model which results in approximately 1600 parameter vectors p. Subsequently, for each design vector, five random perturbations are generated within ±5% of the original design vector's parametric values.Hence, following the exclusion of a small number of inappropriate or nearly identical designs found in the UIUC database, a core set comprising 1263 foil designs was identified for processing.For each of these base designs, as mentioned above, five random shape perturbations were generated, resulting in a total of 1263 + 5 × 1263 = 7578 foil designs constituting the first design dataset, D 1 .

Augmented Airfoil
The second dataset, D 2 , is once again based on the UIUC airfoil database, but this time the additional artificial designs are produced by employing the BézierGAN [39] which was trained using the UIUC dataset as described in [35].Specifically, BézierGAN produces smooth curves by synthesizing the control points, weights, and parametrization of rational Bézier curves which correspond to artificial foil profiles.At the last stage, these profiles are discretized to generate the corresponding SSVs needed in this work.This second dataset contains a total of 38802 foil designs.

Discretization
For both datasets, D 1 and D 2 , the geometric component of the SSVs is produced by discretizing the corresponding smooth profile curves which can be generally represented as parametric NURBS curves.The process of discretization involves transforming the continuous foil profile representation into a polygonal approximation, which can then be stored as a vector of point coordinates for further processing.However, as demonstrated in Masood et al. [30], this discretization, i.e., the point distribution on the profile curve, has a significant impact on the quality of the produced latent space.Therefore, the following four distinct discretization methods for producing N points on the foil profile are explored in this work: 1. Uniform Parametric Spacing: We calculate N parametric values, t 1 , . . ., t N , uniformly distributed over the curve's parametric domain.The resulting N points, {p(t i )} N i=1 are subsequently used in the curve encoding; see Fig. 2a.

Cosine Spacing:
A re-parameterization of all NURBS curves using the cosine function is performed.This reparameterization results in concentrating the generated curve points near the leading and trailing edges of the profile; see Fig. 2b.
3. Curvature-Based Spacing: In this approach, the profile's curvature is utilized to determine the distribution of parametric values.More precisely, parametric points are distributed to ensure an equal curvature integral across all parametric intervals.Consequently, this method leads to a significant point concentration near regions of high curvature, e.g., the leading edge region; see Fig. 2c.Obviously, apart from the point distribution, their number, N , plays an equally significant role in both shape's encoding as well as when evaluating a design's performance.Therefore, taking into account the requirements of the computational package XFOIL [40,41], used in analyzing foil performance, along with the need of an accurate geometric representation of the dataset, we selected a value of N = 200 which achieves a generally low approximation error and it is also sufficient for performing the evaluation in XFOIL.discretization of the foil geometry are considered.Therefore, the corresponding latent spaces included in Table 2 are differentiated only by the point distribution method used in profiles' discretization.
2 (D 1 ) U [1] 2 ( 1 ) [2] 2 ( 1 ) (d) Probability Density Distribution inclusion of the DPP kernel and its loss (9), in addition to the traditional GANs loss function.However, for the second dataset (D 2 ), the non-generative model exhibits a non-balanced distribution of design above and below the average value which indicates a more narrow high-performing region and a more diverse low-performing region.In contrast, when the same model is used with the first dataset (D 1 ) a more balanced distribution is obtained, with designs distributed uniformly around the average value of (84.38).Intriguingly, even in this dataset, the average performance value (82.39) of the SSV-KDE-based approach outperforms once again the enhanced generative model (PaDGAN).Finally, Fig. 3d, which depicts kernel density estimates, provides some further insights into these results.The generative models, GAN and PaDGAN, exhibit a less concentrated distribution over the region of high-performing designs.As for the non-generative model it exhibits an almost uniform distribution when D 2 is used with high concentration over symmetric and high-performing design when D 1 is employed; observe the two distinct peaks when D 1 is used.

Effect of Discretization
As previously discussed, the distribution of points along the curve significantly influences the quality of the achieved latent space.In this comparison, we focus on the D 1 dataset for both generative and non-generative models.By examining Fig. 4a, we can clearly notice a distinct influence of different discretization methods on the validity of the latent space.For example, the non-generative model, employing the uniform point spacing, exhibits a notable increase in invalid designs (3.14%).For the PaDGAN approach, the latent space with uniform parametric spacing produces the worst performing latent space with a significantly elevated percentage of invalid designs, reaching 10.01%.These results highlight the sensitivity of all models, and especially generative ones, to the employed point distribution.
With regards to diversity, we observe a rather mild effect of the discretization scheme when generative models are considered; see Fig. 4b, However, the effect of discretization schemes on the SSV-KLE-based approach is pronounced with average diversity values ranging wildly from −33.39 to −9.26.PaDGAN approach achieves the highest value when the curvature-based spacing is employed with an average diversity score of −2.44.
Furthermore, in terms of the performance indicator, Fig. 4c depicts the impact of the discretization scheme on performance values as performance values vary for D 1 in all models, although less so for the SSV-KLE-based approach.Notably, curvature-based spacing with PaDGAN achieves the highest performance with an average value of 114.05, suggesting its favorable performance and diversity characteristics in the case of generative models.Once again we record another example of generative models' sensitivity to the discretization.

Conclusion and Future Work
In this work, we compared the performance and efficiency of generative and non-generative models in the field of engineering design synthesis and demonstrated how advancements in these models can effectively revolutionize the process.The PaDGAN model, specifically designed for engineering design synthesis applications, is compared with a nongenerative linear KLE-based approach.This study illustrates that the employed discretization in shape representation significantly affects the performance of both approaches, emphasizing the importance of the representation of the design dataset.Additionally, augmenting profile encodings with integral shape characteristics and physics-informed parameters improves significantly the quality of the resulting latent spaces and the efficacy of the KLE-based.In summary, this study demonstrates that non-generative models, which are linear and cost-effective, can achieve results on par with those of generative models.Design spaces for airfoils and/or hydrofoils have been employed in this comparison and although they are important design elements for both aviation and marine industries, they are only 2D designs in nature.An obvious future extension of this work would address the 3D shape synthesis of more complicated functional surfaces, such as wings, blades,

Figure 3 :
Figure 3: Plots for each Quality Analysis Metric as discussed in §2.4 with cosine spacing and SSVs with 4 th geometric moments when augmented SSVs are required.The horizontal line within boxes represents the average value, which is also marked at the top of each box.

Figure 4 :
Figure 4: Effect of each discretization scheme on the three Quality Analysis Metrics.The horizontal line in boxes represents the average value, which is also marked at the top of each box.

4 .
Uniform Point Spacing: Finally, this approach discretizes the profile by computing segments of equal arc length on the curve; see Fig.2d.

Table 2 :
Corresponding latent spaces for each considered geometry discretization for the GAN and PaDGAN approaches.As before, subscripts denote the discretization type while D i can be either D 1 or D 2 .