Principal Component Analysis-Based Convolutional Neural Networks for Atmospheric Turbulence Aberration Correction and the Optimal Preprocessing Strategy Research

Abstract

This study proposes a principal component analysis-based convolutional neural network (PC-CNN) to correct atmospheric turbulence-induced aberrations. Unlike traditional Zernike polynomials (ZPs)-based methods (ZP-CNN), PC-CNN avoids mode aliasing and cross-coupling via the strict orthogonality of principal components (PCs). A coefficient magnification strategy is incorporated to further enhance efficacy, maximally preserving the intrinsic physical information within the PCs coefficients. A series of systematic experiments was conducted under conditions from weak to strong turbulence, characterized by $D/r_0$ from 1 to 25, where $D$ is the pupil diameter and $r_0$ is the atmospheric coherence length. Quantitative results show PC-CNN achieves a lower mean relative error (MRE) in coefficient prediction than ZP-CNN under equivalent conditions. It also yields a higher Strehl ratio, reduced speckles, and enhanced spot clarity while requiring fewer basis terms, demonstrating high stability and robustness in strong turbulence. These findings emphasize that basis function orthogonality and physically informed preprocessing are critical design principles for deep-learning-based wavefront sensor-less adaptive optics (AO), establishing a robust foundation for real-time intelligent AO systems in astronomy and free-space optical communications.

1. Introduction

Atmospheric turbulence disrupts light propagation and induces aberrations, degrading the quality of free-space optical communications and astronomical observations [1,2]. Adaptive optics (AO) systems are thus designed to correct such wavefront aberrations [3,4]. Conventional AO systems rely on wavefront sensors (e.g., Shack–Hartmann sensors and shear interferometers) for wavefront reconstruction, but these suffer from inherent limitations, such as restricted resolution, large volume, and low optical energy utilization [5,6].

Wavefront sensor-less AO (WFS-less AO) systems have been proposed to address the drawbacks of wavefront sensors, which reconstruct aberrations via adaptive control methods [7,8,9,10]. They are generally categorized into model-free and model-based approaches. Model-free methods, such as stochastic parallel gradient descent [11,12], genetic algorithms [13], simulated annealing algorithms [14,15], and Gerchberg–Saxton algorithms [16,17,18], adopt stochastic global or local search strategies for aberration detection and reconstruction.

A limitation of model-free methods is extensive iterations, leading to slow convergence and vulnerability to local optima, making real-time correction challenging [19]. Consequently, model-based methods have aroused widespread interest for their ability to accelerate convergence significantly, including physical and intelligent models. Physical model-based methods decompose wavefront aberrations into Zernike polynomials (ZPs) via optimization algorithms, such as the least-squares method [20,21,22]. With the advancement of artificial intelligence, intelligent models have become prevalent for improving wavefront reconstruction speed and accuracy [23], relying mainly on algorithms like convolutional neural network (CNN) [24,25,26,27,28,29,30,31], generative adversarial networks [32,33,34], and reinforcement learning [35]. Among these, CNNs are particularly suitable for image-based wavefront sensing due to their proficiency in extracting local features and learning complex high-dimensional nonlinear mappings from large datasets [25,26,27]. Li et al. [28] proposed a method using a simple artificial neural network, while Lohani et al. [29] applied a CNN to classify distortion degrees.

In general, ZPs are widely used as basis functions for wavefront aberration characterization in both types of AO systems. ZPs are also the most adopted basis functions in physical model-based methods [20,21,22], and intelligent models typically predict Coefficients of ZPs for aberration correction, requiring ZPs for wavefront fitting. Thus, the effect of aberration correction heavily depends on ZPs’ features.

Notably, Siddik et al. identified sign ambiguity as a key challenge in deep learning-based Zernike coefficient prediction, and they verified that AlexNet-based deep learning models fail to directly map the features of degraded images to such coefficients [36,37]. Moreover, ZPs also suffer from mode aliasing and cross-coupling, which reduce wavefront reconstruction and correction reliability [38]. Mode aliasing misclassifies high-order terms as low-order ones, while cross-coupling causes inter-mode interference, leading to inaccurate mode contribution estimation. These issues stem from the loss of strict orthogonality of ZPs under non-ideal conditions, such as mechanical noise, discrete sampling [39,40,41]. Solving mode aliasing requires reducing fitting order, causing underfitting and compromised accuracy [36]. Meanwhile, computational complexity surges with more ZP terms, hindering coefficient prediction [42].

This study proposes a CNN model that integrates principal component analysis (PCA) to address this research gap and overcome the limitations of ZP-based aberration correction arising from mode aliasing and cross-coupling. As a robust statistical analysis tool, PCA enables the discretized and physically interpretable representation of aberrations. The findings of our previous work indicated that PCA-based feature extraction models exhibit superior information compression capability compared to Zernike polynomials, making them a more optimal strategy for characterizing and reconstructing atmospheric turbulence aberrations. Existing studies have further validated that principal components (PCs) can accurately characterize such aberrations [43,44], laying a solid theoretical foundation for its application in this work. Building on our previous research into PCA-based feature extraction, this work verifies that ZPs deviate from strict orthogonality in practical applications and suffer from mode aliasing and cross-coupling. In contrast, PCs retain rigorous orthogonality and thus mitigate these issues effectively. We conducted a systematic comparison between the PCA-integrated CNN (PC-CNN) and the ZPs-integrated CNN (ZP-CNN) across varying turbulence strengths and further explored optimized preprocessing strategies to improve the practical applicability of PC-CNN. The results demonstrate that PC-CNN outperforms ZP-CNN by yielding lower aberration coefficient prediction errors and enhancing the energy concentration of corrected light spots. This performance superiority stems from the rigorous orthogonality of PCs, which ensures that the CNN can learn the intrinsic features of atmospheric turbulence aberrations effectively. This work advances the development of WFS-less AO and provides valuable insights for the real-time intelligent aberration correction in free-space optical communication and astronomical AO systems.

2. Methods

2.1. Imaging Theory and Optical Quality Evaluation

Atmospheric turbulence is the primary source of aberrations encountered during the propagation of laser beams through the atmosphere and in ground-based astronomical observations. Kolmogorov’s analysis assumes that the random fields of turbulent flow are locally homogeneous and isotropic, and the phase power spectral density (PSD) that satisfies the −11/3 power law is called a Kolmogorov spectrum. In this study, the modified Von Karman PSD is employed as the simplified PSD, incorporating the effects of both inner scales (high-frequency) and outer scales (low-frequency) of atmospheric turbulence.

$${\Phi }_{\varphi }^{mvK}(\kappa )=0.49{r_0}^{-5/3}{\displaystyle \frac{\exp (-{\kappa }^2/{\kappa }_m^2)}{{({\kappa }^2+{\kappa }_0^2)}^{11/6}}},$$

(1)

where $\kappa =2\pi (f_x\widehat{i}+f_y\widehat{j})$ is the angular spatial frequency in rad/m. ${\kappa }_m=5.92/l_0$ is used to generate the phase screen data set using Fast Fourier Transform (FFT) to improve the agreement between theory and experimental measurements [45]. ${\kappa }_0=2\pi /L_0$. $L_0$ represents the outer scales of turbulence and $r_0$ is the atmospheric coherence length [46]. A sub-harmonic compensation technique was integrated into the standard FFT-based phase screen generation process during our simulation. This supplementary procedure specifically addresses the inherent low-frequency deficiency of standard FFT methods, ensuring that low-order aberrations are faithfully represented and the rigorous statistical properties of the modified Von Karman spectrum are fully preserved across all spatial frequencies.

Key parameters were specified as follows to accurately simulate real-world conditions and facilitate subsequent experimental validation: A pupil diameter $D=0.3$ m, wavelength $\lambda =530$ nm, and propagation distance $L=1000$ m were selected to align with planned validation experiments. The turbulence scales $l_0=0.005$ m and $L_0=10$ m reflect values derived from our laboratory’s long-term observational data. Modeling turbulence as a single pupil-plane phase screen decouples phase errors from complex diffraction. This facilitates a rigorous assessment of the basis functions’ representational capacity within the phase-only correction framework. Furthermore, the grid resolution parameter $N=256$ was chosen to optimize both computational efficiency and numerical accuracy.

In most AO systems, ZPs are typically employed to characterize and correct these aberrations. According to the parameters chosen above, the phase aberrations ${\phi }_1(\rho ,\theta )$ can be represented as a combination of ZPs $Z_j(\rho ,\theta )$ with coefficient $a_j$ according to Noll [47],

$${\phi }_1(\rho ,\theta )={\displaystyle \sum _{j=4}^n{a}_j{Z}_j(\rho ,\theta )},$$

(2)

where $\rho$ and $\theta$ are radial and azimuthal variables in polar coordinates. In this study, up to 496 terms of ZPs were used to fit initial phase aberrations. The first three terms of ZPs (piston, X-tip, and Y-tilt) were ignored and aberrations from 4th (defocus) and above were considered. In other words, the values of the piston, X-tip and Y-tilt terms were set to 0 to generate a new phase aberration data set for subsequent studies.

We proposed a PCA-based model in the prior work to address limitations associated with ZPs-based wavefront restoration [45]. And the phase aberration can also be represented as a combination of PC terms $P{C}_j$ with coefficient $b_j$. The representation of the phase aberrations fitted using the first $n-3$ terms of PCs is shown below expression to be consistent with the phase aberrations described above, using from $4^{th}$ to $n^{th}$ ZPs and using the equal terms to control variable:

$${\phi }_2(\rho ,\theta )={\displaystyle \sum _{j=1}^{n-3}(b_{j}P{C}_j)},$$

(3)

In Kolmogorov spectrum cases, $D/r_0$ is used to describe the strength of turbulence based on Lane’s derivation [48]. The higher the value of $D/r_0$, the stronger the atmospheric turbulence. It is necessary to generate datasets encompassing varying turbulence strengths to conduct a more comprehensive analysis of the effectiveness of PCs-based method across diverse turbulent conditions. $D/r_0=5$, 15 and 25 are chosen to represent relatively weak, moderate and strong turbulence strength, respectively.

The point spread function (PSF) is the result of an optical system imaging a point source, and it is a direct and complete representation of the effect of the wavefront aberration on the imaging quality. The PSF, denoted as $h$, can be mathematically expressed as [49]:

$$h={\displaystyle \frac{{\left|\mathcal{F}\left(P(x,y)e^{i\phi (x,y)}\right)\right|}^2}{{\left|\mathcal{F}\left(P(x,y)\right)\right|}^2}},$$

(4)

where $P(x,y)$ is the pupil function with the Cartesian coordinate $(x,y)$ in the pupil plane, $i=\sqrt{-1}$ is the imaginary symbol, and $\phi (x,y)$ is the phase aberrations with the unit of rad. Further, image quality evaluation serves as a critical basis for assessing the performance of aberration correction. In this study, the Strehl Ratio (SR) and spot diameter are employed to quantitatively evaluate the quality of the PSF of the light spot. This dual-evaluation approach not only facilitates the effective learning of image features by the CNN but also provides an intuitive visualization of the aberration correction effect.

The SR is a key indicator for characterizing the intensity performance of optical systems, defined as the ratio of the maximum far-field peak intensity of an actual light beam to that of an ideal beam with the same power and uniform phase. The value of SR equals 1 indicates an ideal aberration correction effect:

$$SR={\displaystyle \frac{h_{\max }}{h_{ideal}}},$$

(5)

In addition to intensity information, the PSF primarily describes the response distribution of an imaging system to an ideal object point. The corresponding spot morphology directly reflects the imaging resolution and energy concentration of the system. The spot diameter can be defined as the width of the spot at the position where the light intensity reaches $1/e^2$ of its peak value [50]. The beam’s focusing performance, spot size, and spatial characteristics can be described by this method. The PSF is first normalized. Subsequently, regions exceeding the $1/e^2$ peak intensity threshold are filtered to simplify calculations. The area of the grid region $S_{pixel}$ is calculated, and the equivalent circle diameter is determined, which represents the spot diameter. The peak intensity of the PSF is $I_{\max }$ and the equivalent circular spot diameter can be expressed by the following formula:

$$D_{eq}=2\sqrt{{\displaystyle \frac{S_{pixel}}{\pi }}},$$

(6)

In this work, CNN was selected from deep learning as an exemplary intelligent model, incorporating it with PCs to intelligently correct phase aberrations induced by atmospheric turbulence. CNN represents a deep learning approach selected for its established efficacy in processing two-dimensional data and capacity for direct feature extraction from images [25,28]. A typical convolutional neural network comprises a series of convolutional layers, pooling layers, and fully connected layers. Convolutional layers are effective in extracting image features. Typically, a pooling layer is appended to a convolutional layer, reducing the number of parameters, the computational complexity, and the data size while preserving the significant image data. The fully connected layer is usually at the end of the network and transforms the feature maps into outputs.

2.2. CNN Architecture Design

The CNN architecture utilized in this study is designed based on the classic AlexNet based on the previous successful utilization in relevant study. And compared with overly deep complex networks, the modified AlexNet has a moderate number of parameters and moderate computational complexity, which matches the lightweight design goal of our proposed PC-CNN model. All simulations and the network were all completed using MATLAB R2022b software, which ensures the efficiency of simulation and training. The original PSF matrices are cropped to amplify the features of the PSF for facilitating CNN learning from $256 \times 256$-sized to $121 \times 121$, which emphasizes the proportion of valid spot information in the input data. Tailored for the specific task of predicting atmospheric turbulence aberration coefficients, it constructs a regression mapping model from a $121 \times 121$ single-channel matrix to a one-dimensional $1 \times m$ coefficients of $m$ terms output. Since the influence of ZPs on spot morphology is grouped by spherical aberration terms, particularly Z₁₁ and Z₃₇, as previously demonstrated in [43,44]. The coefficients selected in this paper correspond to ZPs from Z₄ to Z₁₁ or Z₄ to Z₃₇, with a total of $m$ = 8 or 34 terms. The number of PCs remains consistent with the number of ZPs. This architecture adopts the core design principles of AlexNet, specifically hierarchical feature extraction using multi-scale convolutional kernels and modular stacking. Through structural optimization and task-specific adaptations, it has evolved into a specialized network that balances feature representation capacity with computational efficiency as shown in Figure 1.

Regarding the design of the feature extraction module, the AlexNet’s progressive strategy for reducing convolutional kernel sizes was strictly followed: the initial layer utilizes a large $11 \times 11$ convolutional kernel to capture global features and conduct spatial down sampling, the intermediate layers enhance local feature correlation modeling through $5 \times 5$ convolutional kernels, and the final stage employs stacked small $3 \times 3$ convolutional kernels for fine-grained feature analysis, ultimately constructing a hierarchical feature representation framework. Each convolution unit integrates a Batch Normalization layer and a ReLU activation function to form a standardized processing module. This design can mitigate the vanishing gradient phenomenon, accelerate the model convergence process, and introduce the batch normalization layer.

In this work, the configuration of the hyperparameters of the network is demonstrated in

Table 1. The configuration of hyperparameters.

Hyperparameters	Configuration
Optimizer	Adam
Learning rate	$1 \times {10}^{-4}$
Batch size	64
Max training epochs	500
Learning rate drop	0.5
Dropout rate	0.1
Learning rate drop period	30

.

Compared with the original AlexNet architecture, this design accomplishes model lightweighting without compromising core predictive performance, substantially enhancing the inference efficiency in converting two-dimensional matrices into one-dimensional coefficients. It is especially well-suited for application scenarios demanding stringent computational timeliness, such as real-time correction of atmospheric turbulence-induced aberrations.

2.3. Dataset Construction and Preprocessing

This paper adopts $D/r_0$ values from 1 to 25 with an interval of 2 to characterize turbulence strength from weak to strong. For each turbulence strength, 2500 sets of phase data consistent with the Kolmogorov spectrum are randomly generated. The datasets corresponding to the 13 turbulence strengths are integrated into a comprehensive dataset containing 32,500 samples to enhance the generalization capability of CNNs in practical scenarios. Meanwhile, data corresponding to different turbulence strengths are partitioned into training, validation, and test sets in equal proportions to ensure fairness in model training across data of varying turbulence strengths. The training set comprised 70% of the data, with the test and validation sets each accounting for 15%.

Subsequently, the PSF data is normalized to eliminate the influence of absolute intensity, retaining only the relative distribution characteristics of intensity. For the target coefficient data, the min-max normalization method is applied for preprocessing to ensure the data meets the requirements of model training.

3. Simulation Results and Analysis

3.1. PSF Grading Evaluation

The PSF quality of the test set is evaluated using the SR and the spot diameter to intuitively analyze the model’s training effect, prediction performance, and correction efficacy on aberrations induced by atmospheric turbulence. Samples with $D/r_0$ value of 5, 15, and 25 are chosen as representations of weak, moderate, and strong turbulence, and three PSFs (the best, medium, and the worst) are shown in Figure 2. Then, the worst-performing PSFs are selected to provide support for the intuitive single-sample comparison of the model’s correction effects in subsequent analyses of this study.

As demonstrated in Figure 2, a decline in PSF quality leads to a more diffuse and less concentrated distribution of spot energy. An increase in turbulence strength has been shown to result in more severe phase disturbance, with lower SR values indicating greater light energy loss caused by turbulence. Furthermore, it has been demonstrated that larger spot diameters are indicative of greater imaging spot diffusion and distortion.

3.2. Comparison of Prediction and Correction Performance Between CNN Models Based on PCs and ZPs

As illustrated in Figure 3, the correlation matrix plots of ZPs and PCs are presented. As illustrated in (a), the correlation matrices of the first 29 terms of ZPs reveal that, in practice, ZPs are not strictly orthogonal. This indicates that correlations do exist between the terms, and the fewer terms utilized, the more the correlations between the terms interfere with the aberrations fitting. It is evident from (b), the correlation matrices of the first 29 terms of PCs, that PCs are strictly orthogonal, so the absence of interference between terms is demonstrated. Furthermore, the autocorrelation coefficients of PCs demonstrate statistically significant ordering. This finding indicates that the lower-order terms are more effective in correcting aberrations. It can thus be concluded that there are two key advantages of PCs over ZPs. Firstly, the strict orthogonality of the PCs can effectively inhibit mode aliasing and cross-coupling effects. Secondly, PCs obtained by feature decomposition are arranged in descending order of the eigenvalues, and the order of the terms directly characterizes the contribution weights of the corresponding components in the fitting of the aberration. This configuration facilitates optimal reconstruction accuracy with constrained numbers of terms.

Employing the same convolutional neural network architecture as shown in Figure 1, models were trained on the same PSF dataset to predict PCs and ZPs, respectively. Subsequently, the trained models are used to predict coefficients from the test set PSF, with the comparison of coefficient prediction accuracy shown in Figure 4. The top panel presents a point–line plot of the true and predicted values for a random single sample of PCs and the root mean square error (RMSE) of each coefficient prediction across the entire test set of PCs. While the bottom panel presents the same single sample plot and RMSE of ZPs. RMSE is a measure of absolute error with the capacity to amplify significant deviations of predicted values from true values. This process serves to highlight items requiring additional attention. The calculation is as follows:

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{(y_{i,true}-y_{i,pred})}^2}},$$

(7)

The comparison of RMSE indicates that the prediction errors of each PC term are relatively uniform without obvious preference, indicating that no mutual interference occurs between terms to affect the prediction performance. Conversely, the ZP predictions demonstrate a pronounced tendency for discrepancies in prediction errors across different terms.

Among low-order terms, the defocus (Z₄), astigmatism (Z₅, Z₆), spherical aberration (Z₁₁), secondary astigmatism (Z₁₂, Z₁₃), and tetra foil (Z₁₄, Z₁₅), marked by red circles, exhibit much larger prediction errors than their adjacent terms. This phenomenon arises from the mode aliasing and cross-coupling effect of ZPs, resulting in the incorrect estimation of two aberrations that are similar in morphology or can compensate each other in phase. The observed mode combinations are consistent with the combinations of ZPs reported in previous research [37], with additional supplementary findings. Therefore, the CNN model based on the PCs avoids the problems of ZPs, leading to simpler training processes and superior prediction performance.

Moreover, the relative error assumes particular significance when comparing the prediction accuracy of PC-CNN and ZP-CNN under equivalent conditions. The mean relative error (MRE) is utilized to characterize the relative error of the prediction coefficients. As illustrated in Figure 5, a bar chart has been employed to compare the MREs of models predicting 8 and 34 terms of coefficients of PCs and ZPs when $D/r_0$ is 5, 15, and 25, respectively. The MRE is calculated as follows:

$$MRE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\left|\frac{y_{i,true}-y_{i,pred}}{y_{i,true}}\right|},$$

(8)

The MRE comparison results reveal that PC-CNN demonstrates superior stability and accuracy in coefficient prediction in comparison to ZP-CNN. Firstly, the accuracy of predictions is shown to decrease in proportion to the increase in turbulence strength. It is evident that the ZP-CNN demonstrates a higher degree of sensitivity to turbulence strength. When $D/r_0=5$, the MRE of prediction of 34 terms by PC-CNN falls below 0.3, whereas the maximum MRE of prediction of 34 terms by ZP-CNN on the same test set reaches 0.5. Even when $D/r_0=25$, the predicted MRE of PC-CNN is marginally lower than that of ZP-CNN at $D/r_0=5$. Secondly, the accuracy of predictions decreases with an increasing number of terms. When predicting 8 coefficients, the maximum MRE of PC-CNN was approximately 0.16, and for ZP-CNN, it was approximately 0.34. However, when predicting 34 coefficients, the maximum MRE of PC-CNN exceeded 0.4, and ZP-CNN’s maximum MRE even surpassed 0.8. The accuracy of predicting 8 terms is significantly higher than that of predicting 34 terms under the same turbulence strength. Furthermore, in accordance with the RMSE trend in Figure 4, the prediction accuracy of Coefficients of PCs exhibits a high degree of stability, demonstrating consistent prediction performance across all components. In contrast, the prediction error of ZP increases significantly with higher orders.

Subsequently, the predicted coefficients are employed to reconstruct the phase and perform aberration correction. The same evaluation metrics (SR and spot diameter) are adopted to quantify the performance of the corrected PSFs, thus enabling an intuitive demonstration of the correction effect. Figure 6 presents the statistical comparison results of the SR and spot diameter under different turbulence conditions and with varying numbers of terms. In the figure, “Original” denotes the uncorrected raw PSF data with inherent aberrations.

Comparison results demonstrate that the PC-CNN model delivers superior aberration correction performance relative to the ZP-CNN model. For the same turbulence strength, the correction effect of the 8 terms of PCs outperforms that of the predicted 34 terms of ZPs. After aberration correction, the PC-CNN model yields higher SR values, smaller spot diameters.

Figure 7 shows comparative images of PSFs before and after correction for the single sample with the worst quality rating in the test set under $D/r_0=5$, 15, and 25. The employment of the correction effect of the worst-quality PSF for the purpose of comparison enables an intuitive assessment of the performance of PC-CNN and ZP-CNN under extreme conditions.

Figure 7a displays the original PSF with aberrations before correction. Figure 7b–d illustrate the PSF after correction using 8 terms of PCs, 34 terms of PCs, and 34 terms of ZPs predicted by the model. The comparison of results demonstrates that under relatively weak-to-moderate turbulence at $D/r_0=5$ and 15, the prediction and utilization of 8 terms of PCs results in a superior correction effect in comparison to 34 terms of ZPs. The SR of the corrected PSF demonstrates a substantial enhancement, characterized by a diminished spot size and a more distinct central spot. In conditions of strong turbulence at $D/r_0=25$, the correction of 8 terms of predicted PCs yielded limited effects. However, an increase to 34 terms of predicted PCs has been shown to increase SR from 0.012 to 0.018, accompanied by a substantial reduction in speckle. The spot pattern shifted from a multi-peak distribution to a single dominant peak. Conversely, the utilization of 34 terms of predicted ZPs by ZP-CNN resulted in a corrected SR of 0.016, exhibiting a comparatively faint main spot. It is noteworthy that in conditions of weak to moderate turbulence, the correction effect of 34 terms of PCs exhibits no significant enhancement in comparison to that of 8-term PCs. The rationale underlying this phenomenon is that PCs denote the fundamental constituents of aberrations, with lower-order terms signifying a greater proportion of the aberration they represent. In conditions of weak to moderate turbulence strengths, the principal features of aberration can be significantly corrected by the first 8 terms of PCs. Moreover, the accuracy of CNN predictions for additional Coefficients of PCs diminishes, introducing further prediction errors. Consequently, the enhancement achieved through the increase in the number of PCs is not sufficiently significant, as determined by the statistical properties of PCs.

As demonstrated in Figure 7, the PC-CNN model enables enhanced correction accuracy with a reduced number of terms, thus facilitating a more efficient correction.

3.3. Optimization Analysis of Preprocessing Methods

In accordance with the research findings outlined in Section 3.2, the PC-CNN demonstrates superior and more stable performance in the correction of aberrations induced by atmospheric turbulence. The model exhibits proficiency even under relatively extreme conditions. Consequently, the subsequent investigation will focus on the enhancement of the PC-CNN model.

In earlier studies, the model employed min-max normalization as the primary preprocessing method. This approach involves scaling values to a fixed interval [0, 1], with the objective of eliminating dimensional differences and unifying the value range. The advantage of min-max normalization is its adaptability to different environments. The labels, i.e., the coefficients to be learned by the model, maintain a fixed value range regardless of turbulence strength. However, it is important to note that this approach has limitations, as it is inherently dependent on extreme values. In the event of the presence of outliers in the data, the scaling range of effective data will be compressed, and data that lies beyond the outliers will be oversmoothed. This will have a detrimental effect on the model’s ability to learn features.

Another frequently employed technique in the domain of preprocessing is z-score standardization, which involves the transformation of data into a distribution with a mean of 0 and a standard deviation of 1. The objective of this process is to eliminate distribution shifts whilst preserving the degree of dispersion. The advantages of this approach precisely complement the shortcomings of min-max normalization, which is not affected by extreme values and preserves the statistical distribution characteristics of the data.

Furthermore, given that the distribution and sign of Coefficients of PCs contain crucial physical information, a methodology is proposed that only magnifies the coefficients without altering the original distribution of coefficients. The present study analyses the statistical outcomes of coefficient predictions and the SR and spot diameter of PSFs, both prior to and following the correction. The results of this analysis are presented in Figure 8 and Figure 9.

As illustrated in Figure 8, the statistical outcomes of the root mean square error (RMSE) for coefficient predictions derived from models that have undergone training with three different preprocessing methods are presented. It is evident that while the performance trends of the various preprocessing methods are consistent, their corresponding RMSE values exhibit significant discrepancies: the method with coefficient magnification yields the smallest prediction error, followed by the z-score normalization method, whereas the min-max normalization method results in the largest error. It is interesting to note that the z-score method outperforms the min-max method in terms of prediction accuracy for terms with inherently low prediction errors. However, it exhibits a slightly lower prediction accuracy than the min-max method for terms with relatively large errors, such as the first three terms.

This phenomenon can be attributed to the following reasons in relation to the regression task of Coefficients of PCs. Firstly, it is evident that the sign and distribution of Coefficients of PCs contain critical physical information. In contrast, min-max normalization completely eliminates the physical significance of the sign, thereby converting the difference between positive and negative values into a simple magnitude difference. Consequently, the model is deprived of the key feature that the sign corresponds to the direction of aberration during training. Secondly, it is important to note that simulated turbulence data generally contains extreme values, including individual coefficients of extremely large aberration, which are induced by strong turbulence. The scaling ratio of min-max normalization is entirely determined by the minimum and maximum values of the dataset; these extreme values compress the distribution range of most valid data, severely weakening the data’s discreteness and making it difficult for the model to distinguish between samples. Moreover, min-max normalization presupposes the stability of the extreme value interval of the data, yet the physical parameter distribution of atmospheric turbulence is dynamically changing and unpredictable. The extreme values differ significantly across different turbulence strengths, meaning that the extreme values of the training set cannot cover those of the test set. This phenomenon results in scaling overflow during model testing, leading to a significant decline in prediction accuracy.

Conversely, the z-score standardization has the capacity to mitigate these limitations by centering the standardized data on the mean value, with positive and negative values corresponding to the direction of deviation from the mean. Furthermore, z-score standardization is predicated on statistical distribution characteristics as opposed to extreme values, such that the presence of a small number of outliers will not have a significant effect on the calculation, and the distribution pattern of valid data is preserved. The method of coefficient magnification preserves the original information of the data to the greatest extent, thereby ensuring optimal effectiveness in model training.

As illustrated in Figure 9, the statistical outcomes of the aberration correction are presented. In accordance with the findings of the coefficient prediction study, the method that employs coefficient magnification attains the maximum SR and the minimum spot diameter after aberration correction. It is noteworthy that the min-max normalization method has been shown to yield marginal superior outcomes in terms of SR enhancement compared with the z-score method. Conversely, the z-score method has been observed to demonstrate superiority over the min-max method in reducing the spot diameter. When considered in conjunction with the findings presented in Figure 8, the underlying rationale can be explained as follows: the z-score method demonstrates enhanced accuracy in predicting data distribution and reduced overall numerical variation in coefficients lower RMSE. The phase reconstructed using the coefficients predicted by the z-score method is closer to the true phase, which can more effectively compensate for the wavefront aberration caused by atmospheric turbulence and suppress the spatial diffusion of the light spot more precisely, thus resulting in a more significant reduction in the corrected spot diameter. In contrast, the essence of aberration correction lies in the weighted compensation of coefficients at different orders. Specifically, low-order coefficients have a more significant impact on the overall morphology of the light spot, while high-order coefficients mainly affect the speckle distribution. The min-max normalization method compresses all coefficients into the interval of [0, 1], thereby forcibly highlighting the differences in relative weights among coefficients at different orders. This results in an increased numerical proportion of low-order coefficients, thereby enabling the network to prioritize the compensation of low-order aberrations during the training process. Despite the fact that the min–max normalization method achieves slightly lower coefficient prediction accuracy, the relative weights of the predicted coefficients are more consistent with the actual requirements of aberration correction. Consequently, the reconstructed phase is able to converge the optical energy to the central spot more effectively, thereby obtaining a higher SR.

4. Discussion

This paper proposes a PC-CNN method combining PCA with CNN-based intelligent algorithms to investigate atmospheric turbulence aberration correction models and demonstrates the feasibility of this intelligent correction approach. Key findings and their underlying physical mechanisms are discussed as follows.

Firstly, the PC-CNN method proposed herein demonstrates significant performance advantages in coefficient prediction and aberration correction compared to the ZP-CNN method, which combines CNN with traditional ZP models. With regard to the accuracy of prediction, the MRE of the PC-CNN model is lower than that of the ZP-CNN model under conditions of equivalent turbulence strength. When $D/r_0=5$, the MRE of prediction of 34 terms by PC-CNN falls below 0.3, whereas the maximum MRE of prediction of 34 terms by ZP-CNN on the same test set reaches 0.5. Even when $D/r_0=25$, the predicted MRE of PC-CNN is marginally lower than that of ZP-CNN at $D/r_0=5$. This is due to the strict orthogonality of PCs, which prevents mode aliasing and cross-coupling issues, thus avoiding additional interference in coefficient prediction. However, it has been determined that mode aliasing and cross-coupling are significant factors contributing to the accuracy limitations of ZP-based model. This has been shown to cause misjudgment of high-order terms as low-order terms and incorrect estimation of mutually compensable aberrations (particularly defocus, astigmatism, spherical aberration terms). The hypothesis is substantiated by the observed trend in the RMSE for coefficient predictions across ZP-CNN. In contrast, the RMSE for each coefficient in PC-CNN demonstrates a more uniform distribution of prediction errors.

With regard to the correction accuracy, the PC-CNN utilizing 8 predicted terms generates superior correction results in comparison to the ZP-CNN employing 34 predicted terms under conditions of low-to-moderate turbulence strength. The SR of the ZP-CNN is enhanced from 0.081 to 0.150, with a relative enhancement of approximately 85.19%. In contrast, the proposed PC-CNN has been shown to achieve a more significant SR improvement, with an increase from 0.081 to 0.169 (a relative enhancement of 108.64%) under $D/r_0=5$. The corrected SR value of the PC-CNN is 12.67% higher than that of the ZP-CNN. The initial SR is reduced to 0.031 due to severe wavefront distortion under moderate turbulence conditions ($D/r_0=15$). The ZP-CNN only improves the SR to 0.043 (a relative enhancement of 38.71%) using 34 terms of ZPs. On the contrary, the PC-CNN effectively improves the SR to 0.056 with a relative enhancement of 80.65% with only 8 terms of PCs. The corrected SR of the PC-CNN is 30.23% higher than that of the ZP-CNN under $D/r_0=15$, indicating that the PC-CNN exhibits greater robustness against increasing turbulence strength. When the turbulence strength is further enhanced with $D/r_0$ reaching 25, the proportion of high-frequency aberration components in the wavefront distortion increases significantly, which poses greater challenges to the aberration correction model. In this scenario, the correction effect of the PC-CNN with only 8 terms of PCs is limited, since the limited number of PCs cannot fully characterize the complex high-frequency aberration features induced by strong turbulence. However, increasing the number of PCs to 34 terms enables the model to capture more detailed high-frequency aberration information, thereby improving the SR from 0.012 to 0.018 with a relative enhancement of 50%. For comparison, the ZP-CNN with the same 34 terms of ZPs only improves the SR to 0.016 (a relative enhancement of 33.33%). More importantly, PSFs of the corrected spots show that the ZP-CNN produces more speckles and lower spot sharpness, while the PC-CNN yields fewer speckles and higher spot clarity. It is worth noting that the modest improvement in SR under strong turbulence ($D/r_0=25$) is not caused by the representational limits of PCs. As demonstrated in our previous work [43,44], PCs can accurately reconstruct severe phase distortions. The current bottleneck lies in the prediction accuracy of the standard CNN. When dealing with complex high-frequency aberrations, the network’s prediction accuracy for high-order PC coefficients decreases, which introduces residual errors. Future research will focus on advancing the network design to overcome this limitation within the proposed framework. Specifically, incorporating attention mechanisms and physical prior constraints can effectively improve the prediction accuracy for high-order features. This path will further enhance the correction capability of the PC-based model under extreme turbulence conditions.

Secondly, the model integrated with the coefficient magnification strategy demonstrates optimal performance for the PC-based regression model. This superiority is attributable to its capability to maximally preserve the intrinsic physical information encoded in the coefficients of PCs. The sign of the coefficients directly characterizes the direction of wavefront aberration (e.g., positive values correspond to wavefront convexity and negative values to wavefront concavity), while the amplitude distribution of the coefficients quantitatively reflects the contribution of different aberration components.

Thirdly, the computational efficiency of the PC-CNN was quantitatively evaluated. For a test set of 2500 frames, the total forward-pass inference time was 2.426 s, yielding an average latency of 0.97 ms per frame (equivalent to an algorithmic frequency > 1000 Hz). While a closed-loop AO system involves additional hardware-level latencies, this sub-millisecond processing speed validates the lightweight architecture of the PC-CNN and its capacity to meet the stringent bandwidth requirements of real-time correction.

Furthermore, regarding the rapid temporal evolution of turbulence, the data-driven PCs encapsulate the macroscopic spatial covariance of the environment. This statistical robustness ensures the spatial basis remains effective without real-time recalculation, even under significant instantaneous fluctuations.

5. Conclusions

This study presents a reliable and computationally efficient atmospheric turbulence aberration correction method based on PCA and CNN, which achieves superior correction accuracy and robustness compared to the traditional ZP-CNN method. Specifically, the proposed PC-CNN method leverages the strict orthogonality of PCs to eliminate mode aliasing and cross-coupling issues inherent in ZPs, enabling uniform and lower-error prediction of aberration coefficients. When integrated with a coefficient magnification strategy, the PC-CNN further preserves the intrinsic physical information of Coefficients of PCs, where coefficient signs denote wavefront aberration directions and amplitude distributions reflect the contribution of different aberration components.

By systematically validating the critical role of basis function selection and physically informed preprocessing strategies in optimizing model performance, this work provides actionable guidance for the design of high-performance turbulence aberration correction models. The findings reveal that the PC-CNN demonstrates consistent correction performance with a reduced number of terms in comparison to the ZP-CNN across a wide range of turbulence strengths. These results demonstrate the PC-CNN’s adaptability to complex practical turbulence conditions, laying a solid foundation for enhancing the quality of optical imaging systems affected by atmospheric turbulence. While the proposed PC-CNN framework demonstrates robustness under relatively strong turbulence, it is important to acknowledge the physical limitations imposed by scintillation, which remains an important avenue for future research.

This work provides novel insights and research directions for the real-time intelligent correction of atmospheric turbulence aberrations, further facilitating the practical deployment of deep learning in AO systems. Future research will concentrate on three fundamental aspects. Firstly, the optimization of deep learning performance by adopting advanced network architectures (e.g., attention mechanisms, ResNet, lightweight networks) and fine-tuning hyperparameter configurations. Secondly, the integration of enhanced physical prior constraints tailored to PCA-derived features to further improve high-frequency aberration correction accuracy under extreme turbulence. Thirdly, the acceleration of model inference speed to meet the real-time requirements of AO systems for astronomical observation and free-space optical communications. The present work demonstrates that basis function orthogonality and physically informed preprocessing are critical design principles for deep-learning-based sensor-less AO systems. This provides a robust operational framework for developing next-generation AO technologies.