Bagged Tree Model to Retrieve Planetary Boundary Layer Heights by Integrating Lidar Backscatter Profiles and Meteorological Parameters

Abstract

The planetary boundary layer (PBL) is the part of the troposphere in which the soil’s influence is noticeable. It plays an important role in the fields of air pollution, meteorology, weather forecasting, and climate. Continuous observation of lidar makes obtaining the day–night PBL height (PBLH) with a high temporal resolution possible. A high-precision PBLH retrieval method is the key to achieving this goal. In this study, we propose a new method based on a bagged tree model to retrieve the PBLH from micro-lidar backscatter profiles. With the radiosonde measurements taken as the true reference, lidar features (the ten maximum slopes identified by the maximum gradient method) and four meteorological parameters (atmospheric pressure, temperature, relative humidity, and wind speed) serve as characteristic variables. The PBLH retrieval model is evaluated using a 10-fold cross-validation (CV) method and then compared with the four traditional methods (i.e., maximum gradient, maximum standard deviation, wavelet covariance, and the ideal profile method). The correlation coefficient (R) between the retrieved PBLHs and the radiosonde measurements is 0.89, which is much bigger than the R (0.2–0.48) from the four traditional methods. Moreover, the root mean square error and mean absolute error for the retrieved PBLH are 0.3 km and 0.2 km, respectively, which are lower than those of the four traditional methods (0.5~0.6 km for RMSE and 0.4–0.5 for MAE). Cases with different conditions show that this new method is almost undisturbed by cloud and suspended/thick aerosol layers. It can also be used to retrieve shallow PBL in cases in which using traditional methods would be difficult. Long-term analysis of averaged PBLHs retrieved by the proposed model from 2013 to 2016 shows that the hourly PBLH rises at sunrise and sets at sunset, and that the monthly PBLH in summer is higher than that in winter. The results suggest that the proposed method is better than the four traditional methods and available for use in conditions such as existing cloud layers and multiple-layers.

1. Introduction

The planetary boundary layer (PBL) is defined as the nearest atmospheric layer to the earth’s surface [1]. The height of PBL (PBLH) is the thickness of PBL, a key factor in determining the regional scale of environmental pollution, weather, climate, and air pollution models [2,3]. A diurnal variation in time and space for the PBLH is typically from several kilometers to hundreds of meters [4]. Day–night PBLH observations can monitor the pollutant change in real time and provide the PBLH for weather and climate forecasting. Previous studies show that the PBLH derived from a radiosonde has a high accuracy and is considered as the validation data for PBLHs derived from other instruments and models [5,6]. However, a radiosonde has a low temporal resolution approximately twice a day [7]. As an active sensor, light detection and ranging (LiDAR) can be applied to retrieve aerosol layers [8], wind fields [9], and leaf biochemical properties [10], amongst others. Continuous observation of aerosol lidar makes obtaining the day–night PBLH with a high temporal resolution possible [5,11].

PBL has a higher aerosol burden than that of the free atmosphere above it [12,13]. The backscatter signal has strong attenuation, and its height is the PBLH [14]. Many algorithms are used to retrieve the PBLH, such as the maximum gradient (MG) [15], maximum standard deviation (MSD) [16], wavelet covariance transformation (WCT) [17,18], and ideal profile fit (IPF) methods [14]. These algorithms are prone to error under complex conditions, such as the presence of clouds or noise. Several algorithms perform well in the daytime and are prone to error at night owing to the residual layer [19]. Machine learning is widely used in various fields with classification and regression functions. Some machine learning models are introduced to simulate the PBLH. Kumar, et al. [20] developed a tool to automatically identify different structures of PBL based on machine learning technology. Rieutord, et al. [21] used two machine learning algorithms (k-means and AdaBoost for PBL) to derive the PBLH based on the training sets made by human experts. These two algorithms perform well in general, but they still have limitations, such as misjudgments. Ye, et al. [22] used training data, which are radiance data from an emitted radiance interferometer, and the true values of PBLHs measured by radiosondes, which are then used by a support vector machine to derive the PBLH. This method reduces the influence of clouds on PBLH retrieval and proves the feasibility of machine learning. de Arruda Moreira, et al. [23] proposed gradient boosting regression trees to improve the estimation of the PBLH derived from a ceilometer. A non-supervision Mahalanobis transform K-near-means algorithm was proposed to estimate the PBLH, which can identify available PBL under the cloud and residual layer conditions [24]. However, multiple aerosol layers influence the estimation for the PBLH. Therefore, it is promising to explore a new method for PBLH estimation based on machine learning.

The accuracy of PBLH modeling in machine learning can be improved by exploring different parameters and models. The bagged tree (BT) algorithm of supervised machine learning is used to train a model for the retrieval of the PBLH. The input data are the first 10 maximum variances of the backscatter signal from the micro-pulse lidar (MPL) [25] and meteorological parameters from the surface meteorology systems (MET) [26] of the Atmospheric Radiation Measurement (ARM) facility, and the output data for the training set are PBLHs derived from the radiosonde [27] and are also used as the reference. The structure of this paper is as follows: The second section describes site and data. The third section details the methods of BT model, MG, WCT, MSD, WCT, and IPF. The fourth section analyzes the experimental results, and the fifth section summarizes the study.

2. Site and Data

2.1. Site

This study applied three kinds of data from the ARM: the backscatter signal profiles from the MPL, the meteorological parameters from the MET, and the PBLH derived from the radiosonde. Figure 1 shows the locations of the instruments. The red triangle indicates the sites of the MPL, the radiosonde, and the MET at Lamont, Oklahoma (Lat: 36.607322, Lon: −97.487643), the Southern Great Plains (SGP).

2.2. Radiosonde

The ARM facility provided the PBLH value-added product from radiosonde profiles [28]. The variables from the radiosonde profiles contained the pressure, dry bulb temperature, relative humidity, wind speed, and wind direction, which were used as input data to retrieve PBLHs. In this study, PBLHs for the output training data were derived using the bulk Richardson number method [2] using a critical threshold of 0.25, specifically reported by Sivaraman, et al. [27].

2.3. MPL

MPL is an eye-safe lidar operating at 532 nm and an active ground-based automatic observation instrument. It works by sending short pulses of laser light and detecting and collecting light backscattered by atmospheric particles. The distance to the scatterer can be inferred from the time delay between transmitting and receiving the signal. The produced MPL contains the return signal in the co-pol and cross-pol channels [25]. The range resolution is 15 m, and the averaging temporal resolution is approximately 10 s.

2.4. Meteorology Measurements

The meteorological parameters from the MET of the ARM facility at the SGP site were used as input data for the training set. The variables atmospheric pressure, temperature, relative humidity, and wind speed were used in the study as the input data for the training set. The accuracy of pressure, temperature, RH, and wind speed measurements were 0.01 kPa, 0.01 °C, 0.1% RH, and 0.01 m/s, respectively. The measurement interval of atmospheric pressure was one minute, and those of the other variables were one-minute averages. The measurement levels of atmospheric pressure, temperature, relative humidity, and wind speed were are 1, 2, 2, and 10 m, respectively, and the resolutions were 0.01 kPa, 0.001 dec C, 0.1%, and 0.001 m/s, respectively [29].

3. Methods

3.1. Bagged Tree (BT)

BT is an ensemble of decision trees for either classification or regression. Many training sets were taken from the total training set, and each training set was used to create a separate prediction model. The prediction result is the average predictor for every decision tree, which reduces the variance. Studies show that integrating decision trees provides a higher accuracy than that of a single tree [30,31]. The BT model is defined as

$${\hat{\mathrm{f}}}_{bag}=\frac{1}{B}{\displaystyle \sum _{b=1}^B{\widehat{f}}^b}(x)$$

(1)

By generating B different self-help training datasets, the prediction results ${\widehat{f}}^b(x)$ (b = 1, 2, …, B) were obtained from the self-service training dataset. A clustering prediction ${\hat{\mathrm{f}}}_{bag}$ was obtained by averaging B training results. The k-fold cross-validation (CV) randomly divided a dataset into k disjoint folds with roughly the same number of instances. Each fold, in turn, served as the test level, and other folds served as the model generated by the training dataset. The average of the correct rate of the k times results was used as an estimate of the accuracy of the algorithm. In this study, the number of decision trees (B) was 200, and the minimum leaf size was 5, k = 10.

Figure 2 displays the workflow for PBLH estimation. The key step is choosing the feature of the lidar profile. Specifically, the variance profile is calculated from the lidar signal, and the 10 maximum values of the variance profile work as a part of characteristic variables. The meteorological parameters, namely, atmospheric pressure, temperature, relative humidity, and wind speed, are the other part. The characteristic variables are also called test sets. The test set and the PBLH derived from the radiosonde form the training set, the size of which is 22,860. This step is flexible in order to select feature variables helpful in constructing the PBLH retrieval model for other lidar signals. The BT model used to retrieve the PBLH can be built and trained by the training dataset. Then, the output of the PBLH can be estimated by the trained BT model.

3.2. Traditional Methods

3.2.1. Maximum Gradient (MG)

The gradient of the signal can be calculated by moving the window center according to the specified window size, which is defined as follows:

$$B_{grad}\left(z\right)=\frac{dB(z)}{dz}$$

(2)

where $B(z)$ is the backscatter signal of MPL, z is the height above the ground, and $B_{grad}\left(z\right)$ indicates the gradient of $B(z)$. The height of the greatest attenuation can be regarded as the PBLH.

3.2.2. Maximum Standard Deviation (MSD)

The signal standard deviation is defined as follows [32]:

$$B_{std}\left(z\right)=std\left[B(z)\right]$$

(3)

where $B(z)$ is the signal profile, z is the height above the ground, and $B_{std}\left(z\right)$ refers to the standard deviation of $B(z)$. The height corresponds to the maximum $B_{std}\left(z\right)$ can be regarded as PBLH.

3.2.3. Wavelet Covariance Transformation (WCT)

The WCT method is applied to estimate PBLH by detecting step changes in signals. The Haar wavelet function h is defined as follows [33]:

$$h\left(\frac{z-b}{a}\right)=\left\{\begin{array}{l}+1, b-\frac{a}{2}\le z\le b,\\ -1, b\le z\le b+\frac{a}{2},\\ 0, \mathit{elsewhere}.\end{array}\right.$$

(4)

where z is the height above the ground, b is the center position of the h function, and a is the extension of the function. The covariance transform of the Haar function $W_X(a,b)$ is defined as

$$W_X(a,b)=\frac{1}{a}{\displaystyle {\int }_{z_b}^{z_t}B\left(z\right)h\left(\frac{z-b}{a}\right)}dz$$

(5)

where $B\left(z\right)$ is the signal profile, and $z_b$ and $z_t$ are the lower and upper limits of $B\left(z\right)$, respectively. A local maximum in $W_X(a,b)$ with a coherent scale of a, b is usually considered as the PBLH.

3.2.4. Ideal Profile Fit (IPF)

Steyn, et al. [14] presented the IPF method to estimate PBLH. The method has been widely used in PBLH detection. The function of an idealized backscatter profile is defined as

$$B(\mathrm{z})=\frac{B_m+B_{\mu }}{2}-\frac{B_m-B_{\mu }}{2}erf\left(\frac{z-z_m}{s}\right)$$

(6)

where B(z) is the idealized backscatter profile, B_m is the backscattering of the average of mixed layer backscatter, B_u is the average backscattering in the air directly above the mixed layer, $z_m$ is the mixed layer depth as well as the PBLH in this study, and s is related to the thickness of the entrainment layer. Four ideal backscattering profile parameters are determined by minimizing the root mean square deviation between B(z) and the original backscatter profile. The thickness of the entrainment zone is equal to 2.77 s, where s is the thickness of the layer in which the layered air and overlying air are mixed.

3.3. Evaluation Approaches

The sample-based 10-fold CV was utilized in this study to verify the PBLHs located by the proposed BT models. The sample-based CV, a widely used validation approach, involves randomly selecting 90% of the samples for modeling and using the remaining 10% for verification. All samples are guaranteed to be tested by repeating the process 10 times. The performance of the BT models was quantitatively evaluated using the following statistical metrics: linear regression equation (slope and intercept), correlation coefficient (R), root-mean-square error (RMSE), and mean absolute error (MAE).

4. Results and Discussion

In this section, the performance of the BT model was evaluated by the 10-fold CV. The PBLHs derived by the four traditional methods are given as a comparison to demonstrate the excellent ability of the BT model. The long-term PBLHs in the study station from 2013 to 2016 were statistically analyzed at the end.

4.1. Model Validation

Figure 3 shows the PBLHs derived from the BT model and measurements from the radiosonde. The x- and y-axes represent PBLHs from the radiosonde and the BT model, respectively. The color represents the sample distribution density of each PBLH. Blue represents the smaller density, and red represents the larger density. Figure 3a shows that after model training with the training set, the PBLHs of the training set were predicted and compared with the PBLHs of the corresponding radiosonde. The correlation coefficient (R), root mean square error (RMSE), and mean absolute error (MAE) are 0.97, 0.2 km, and 0.1 km, respectively, and they are highly consistent with the PBLHs of the radiosonde with low error. Figure 3b shows the results of the sample-based 10-fold CV. Compared with the radiosonde measurements, the high R of 0.89, low RMSE of 0.3 km, and MAE of 0.2 km indicate that the model has a strong generalization ability. The results based on the 10-fold CV method demonstrate the strong generalization ability of the proposed model.

4.2. Comparison with Other Methods

MPL measurements from 2013 to 2016 were applied to retrieve the PBLH from the four traditional methods (MG, MSD, WCT, and IPF). Due to the weak anti-interference capability of these traditional methods, only the values with deviation for PBLHs between traditional methods and radiosonde less than 500 m are shown. Figure 4 shows the PBLHs derived from the MG, MSD, and WCT, and IPF methods compared with radiosonde measurements. The x-axis of Figure 4 expresses the PBLHs derived from the radiosonde, and the y-axis of Figure 4a–d express the PBLHs derived from the MG, MSD, WCT, and IPF. The chromaticity bar represents the sample size, and the redder the color, the greater the density. The R between the retrieval PBLHs and the radiosonde measurements shows that the BT model is superior to the other traditional methods (R: 0.48, 0.41, 0.48, and 0.20). Compared with the PBLH calculated by the four traditional methods, the PBLH retrieved from the BT model is more consistent with the radiosonde. The lower RMSE and MAE show that the BT model has a lower error rate than that of the traditional methods.

Recently, PBLH algorithms have been greatly developed and improved by considering them in one and two dimensions. In one dimension, generally improving traditional methods with some constraints [19,34,35] allowed excellent results to be obtained in certain conditions. Liu, et al. [34] improved the threshold for signal to noise to estimate the PBLH from the radar wind profiler. This method was highly consistent with radiosonde measurements based on data from the summer of 2018 in Beijing, when the R was approximately 0.69. However, this method is unavailable in cloudy or dusty conditions and needs to change the threshold based on areas and seasons. Su, et al. [19] presented an algorithm to determine the PBLH by combining the traditional method and PBLH temporal variation, which can exclude the interference of the residual or multiple aerosol layers. Moreover, algorithms developed from two dimensions are similar with image processing [36,37]. Vivone, et al. [36] proposed an algorithm based on image processing techniques to continuously retrieve the PBLH and achieve better accuracy (about 30%) than that of the WCT method. However, this type of method depends on the correlation of adjacent lidar distance bins and requires continuous data not available for real-time estimates. In summary, the method proposed in this study had a better performance than that of other methods proposed in recent studies.

4.3. Case Analysis

In this section, four cases of PBLH estimation are analyzed to demonstrate the advantages of the BT model. Figure 5 and Figure 6 display the performances of the BT model under four conditions, namely, suspended aerosol layers (marked ‘A’), optical thick layers (marked ‘B’), shallow boundary layers (marked ‘C’), and impenetrable layers (marked ‘D’).

Figure 5 shows the data with suspended aerosol and optical thick layers from (a) 18 August to 24 August 2013 and (b) 1 May to 7 May 2016, respectively. The color bar represents the strength of the MPL signal, with orange indicating larger values. The blue line and red points indicate the PBLH derived from the BT model and the radiosonde, respectively. Overall, it is easy to notice the remarkable consistency between the PBLHs retrieved from the BT model and the radiosonde observations. Figure 5a,b display the suspended aerosol and optical thick layers in zones A and B, respectively. In general, many traditional methods, such as the MG, MSD, WCT, and IPF methods, are unavailable under the conditions in zones A and B [11]. Fortunately, the BT model is available for PBLH estimation for these two conditions.

Figure 6 shows the MPL measurements with shallow PBLs and impenetrable layers from (a) 18 January to 24 January 2015 and (b) 1 November to 7 November 2015, respectively. The color bar represents the strength of the MPL signal, with orange indicating larger values. The blue line and red points indicate the PBLH derived from the BT model and the radiosonde measurements, respectively. Similarly, the PBLH derived from the BT model is consistent with the radiosonde measurements. Estimating PBLHs is difficult using these traditional methods under the following two conditions: shallow PBLs in zone A of Figure 6a and impenetrable layers in zone B of Figure 6b. The BT model has an obvious ability to overcome interference from non-boundary layer signals.

4.4. Long-Term Analysis

Figure 7 shows the mean of PBLHs of different time periods from 2013 to 2016. The black and gray line indicates the mean of PBLHs and the standard deviation of PBLHs, and the color bar indicates that the MPL signals from blue to red correspond to the MPL signal strength from small to large. Figure 7a shows the variation in hourly PBLHs in a daily cycle. The PBLH begins to rise to about one kilometer from 7:00 a.m. to 12:00 a.m., and then it maintains a height of approximately 1 km from 12:00 a.m. to 6:00 p.m. From 7:00 p.m. to 9:00 p.m., the PBLH begins to drop to roughly 300 m, and then the PBLH stays at approximately 300 m from 10:00 p.m. to 7:00 a.m. The PBLH is generally lower at night and higher during the day, and an ascending phase at sunrise and a descending phase at sunset are shown. The standard deviation of the PBLH is higher in the daytime from 11:00 a.m. to 8:00 p.m. than in other periods. Figure 7b shows the variation in the monthly mean of PBLHs from 2013 to 2016. The PBLHs start to rise from January to June and go down from June to December. From the end of spring to the beginning of autumn (April to September), the PBLH is approximately 600 to 800 m, and in other months, it is approximately 400 to 500 m. That is, the PBLH in summer is higher than in winter, the PBLH in spring gradually increases, and the PBLH in autumn gradually decreases.

4.5. Uncertainty Analysis

The excellent performance of the BT model in retrieving the PBLH is obtained using the CV method in case analysis in different conditions. However, some deviations still exist in PBLH calculations conducted by the BT model. The causes of deviations are very complex, involving external inadequacy of environment (feature) variables and internal inconsistency of lidar-captured aerosol layers with the PBL. For the first reason, better feature variables are sought from the lidar signal by considering the corresponding relationship between the aerosol layers and PBLs. The heights of 10 maximum standard deviations of the MPL signal could include both the boundary layer and other layers, such as a cloud layer, elevated aerosol layer, and noise that could reduce the performance of the model in retrieving the PBLH. Figure 7a displays the case with the top of the aerosol layer being inconsistent with the radiosonde PBLH in zone A. The radiosonde PBLHs are lower than those from the BT model. The results should be higher if the top of the aerosol layer is the position of the PBLH. It is very difficult to judge which results are correct.

Moreover, the changes in the aerosol layer (the black dotted line) may be delayed with the change in the true boundary layer, as shown in zones B1, B2, and B3 of Figure 8b. That is, consistency between the PBLH and the top of the aerosol layer does not always exist. The boundary layer can change rapidly, but the aerosol layer changes slowly. To some extent, we suspect that this synchronicity also affects the performance of the traditional methods and raises the inconsistency between the PBLHs derived from the traditional methods and the radiosonde measurements. This delay mechanism is uncertain and needs further analysis. Unlike these traditional methods, the BT model is not only based on a MPL signal but also incorporates meteorological parameters, which are not easily affected by aerosol delay effects. Therefore, PBLHs derived from the BT model are more consistent with the radiosonde measurements.

5. Conclusions

In this study, a new method was proposed to estimate the long-term PBLHs by choosing the ten maximum values based on the MSD from the lidar profile and four meteorological parameters. The PBLH retrieval method was created by using the BT model and then validated by using a sample-based 10-fold CV. The high correlation of 0.89, low RMSE of 0.3 km, and MAE of 0.2 km from the three CV methods demonstrated the outstanding generalization ability of the proposed model. The PBLHs derived using the BT model were then compared with the PBLH from the four traditional methods and other studies. The proposed method was better than the four traditional methods, and, being almost undisturbed by both suspended aerosol layers and optical thick layers, it can also be used in conditions for shallow boundary layers and impenetrable layers. Finally, the long-term analysis of PBLHs displayed the diurnal and seasonal changes in PBLHs. The PBLH rises at sunrise and sets at sunset, and the monthly PBLH in summer is higher than that in winter.

This paper proposed a new method for retrieving PBLH based on machine learning with high accuracy, and it is superior to the four traditional methods and is helpful in providing real-time and high-quality PBLH observations. However, there are still deviations caused by inadequacy of environment (feature) variables and inconsistency of lidar-captured aerosol layers with boundary layers. Future work will concentrate on reducing the uncertainty caused by the two aforementioned issues to improve the performance of the method.