Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data
Abstract
:1. Introduction
2. CATS Level 2 Operational Data Products and Algorithms
2.1. CATS Operational Layer Detection Algorithm
- The CATS L2O algorithm uses the 1064 nm attenuated scattering ratio, unlike the CALIOP algorithm that uses 532 nm.
- The CATS L2O data is averaged to only two horizontal resolutions (5 and 60 km) instead of the 5 resolutions CALIOP uses.
- The CATS layer detection includes an algorithm to detect clouds embedded in aerosols.
- The CATS layer detection algorithm also utilizes depolarization to create layer boundaries.
- The molecular contribution to the total backscatter signal is much smaller at 1064 nm than 532 nm, and is nearly negligible in the upper troposphere [39].
- For absorbing aerosols, the absorption optical thickness increases with decreasing wavelength. This effect reduces the backscattered signal at 532 nm with respect to 1064 nm, such that the 532 nm backscatter is not sensitive to the entire vertical of the aerosol layer [40,41,42]. Because the 1064 nm wavelength is only minimally affected by aerosol absorption, the vertical extent of the absorbing aerosol layer is more fully captured from 1064 nm backscatter profiles rather than those from 532 nm [30].
2.2. Cloud-Aerosol Discrimination
3. Machine Learning Algorithms
3.1. Denoising Technique
3.2. CNN Technique
4. Comparison of ML Techniques with Operational CATS Data Products
- Misclass—when the CNN and CATS L2 agree there is a feature (either cloud, aerosol, or undetermined), but do not agree on the type
- True Neg—when both the CNN and CATS L2 agree there is clear air (no feature)
- False Neg—when the CNN classifies a bin as clear air and CATS L2 identifies that bin as any feature type
- False Pos—when the CNN classifies a bin as any feature type (either cloud, aerosol, or undetermined) and the CATS L2 identifies it as clear air
- True Pos—when the CNN detects a feature and its classification matches the CATS L2
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A. Details of CATS Vertical Feature Mask Algorithms
Appendix A.1. Description of Cloud-Embedded-in-Aerosol-Layers (CEAL) Method
- From the altitude bin of the maximum ATB, a search begins upward (in altitude) bin by bin until the ATB value is below the threshold.
- This bin is assigned as the top of the cloud layer.
- The base of the cloud layer is found analogously by searching downward from the maximum until the same conditions are met.
Appendix A.2. Logic for Preventing the Artificial Spreading of Clouds
- If >75% of a 60 km layer is overlapped vertically by a 5 km layer(s) then the 60 km layer is eliminated.
- If the integrated attenuated total backscatter (iATB) of a 60 km layer is greater than 0.03 km, then the 60 km layer is eliminated.
- If the ATB value of any of the 5 km bins within the collocated bounds of the 60 km bin are greater than 0.0006 (night) or 0.007 (day), then the 60 km bin is eliminated.
Appendix A.3. DeBLD Depolarization-Based Layer Delineation Algorithm
- First, define 3 threshold tests related to the 2 candidate boundaries.
- (a)
- Test D: considering the candidate boundary identified by performing wavelet analysis on the volume depolarization ratio profile—pass if the percentage of bins in the thinner sub-layer, where the volume depolarization ratio is either greater than the max or less than the min of thicker sub-layer, exceeds 70%.
- (b)
- Test A: considering the candidate boundary identified by performing wavelet analysis on the ATB profile—pass if the percentage of bins in the thinner sub-layer, where the volume depolarization ratio is either greater than the max or less than the min of the thicker sub-layer exceeds 90%.
- (c)
- Test A2: considering the candidate boundary identified by performing wavelet analysis on the ATB profile—pass if the percentage of bins in the thinner sub-layer, where the volume depolarization ratio is either greater than the max or less than the min of the thicker sub-layer exceeds 70%.
- If both Test A2 and Test D are passed and both sub-layers for each candidate boundary are at least 420 m thick ...
- (a)
- If the candidate boundaries are equal, select their value as the chosen boundary. Otherwise ...
- (b)
- If the difference between the candidate boundaries is not equal to zero and is less than 120 m (“minimum spike thickness”) ...
- Select the ATB boundary if the percentage computed for Test A is greater than the percentage computed for Test D; otherwise select the volume depolarization ratio boundary.
- If Test D is passed and the sub-layers for the depolarization boundary are both at least 420 m thick, then select the depolarization boundary.
- If Test A is passed and the sub-layers for the ATB boundary are both at least 420 m thick, then select the ATB boundary
- If neither boundary has yet been chosen, a statistical t-test is performed to determine whether the distributions of volume depolarization ratio values between the two sub-layers divided by the volume depolarization ratio at the boundary are significantly different (p-value = 1 × 10). If they are significantly different and the mean volume depolarization ratio of the thicker sub-layer is either greater or less than the min or max (respectively) of the thinner sub-layer, then the depolarization boundary is chosen. Since the distributions of volume depolarization ratio values are assumed to be approximately Gaussian, if the mean of one sub-layer is outside the boundaries of the distribution of the other sub-layer, it can be confidently concluded that the sub-layers are unique.
Appendix A.4. Cloud-Aerosol Discrimination Accuracy Tests
- Horizontal Persistence Test: Since true lofted dust and smoke layers tend to have large horizontal extent, a horizontal persistence test was added to the algorithm for earlier versions of nighttime L2O data (V2-01) to identify liquid water clouds with enhanced volume depolarization ratios of small horizontal extent and correctly classify them as clouds. However, the same test was not as effective during daytime due to the noisy daytime signals so it was not implemented in V2-01. The result is a reduction of dust mixture and smoke aerosol detection over remote parts of the Earth’s oceans in nighttime CATS L2O V2-01 data, but the issue still remained in the daytime data. A slightly modified version of the horizontal persistence test was added to the algorithm for daytime data in V3-01.
- Cloud Fraction Test: The Cloud 350 m Fraction variable was used to identify complex scenes/layers in which boundary layer cumulus clouds are mixed with aerosols. Many of these layers are now defined as “undetermined” in the V3-01 data. This variable is also very helpful in differentiating aerosols from depolarizing liquid water clouds in the lower troposphere and tests have been added to ensure any layers with a Cloud 350 m Fraction greater than 0.90 are classified as clouds and any layers with a Cloud 350 m Fraction less than 0.10 are classified as aerosols.
- Integrated Perpendicular Backscatter Test: Previous versions of the CATS CAD algorithm utilized the layer-integrated attenuated backscatter intensity in lieu of the layer-integrated attenuated backscatter color ratio that the CALIOP CAD algorithm uses. This works well for thin aerosol layers, but some optically thick dust and smoke plumes are falsely classified as clouds. To overcome this issue in the V3-01 data, a test using the layer-integrated perpendicular backscatter has been employed. The multiple scattering from ice and liquid water clouds results in layer-integrated attenuated backscatter values that are significantly higher than aerosols. For cloud and aerosol layers with low Feature Type Scores (−5 to +5), a threshold value of 0.004 km sr is used to differentiate clouds and aerosols. This test also improves the discrimination of UTLS aerosols and thin ice clouds.
- Relative Humidity Test: In previous versions of the CATS data products, dust plumes in the upper troposphere, which can reach as high as 12 km as they are transported from Asia over the northern Pacific Ocean and have volume depolarization ratios greater than 0.25, were classified as ice clouds. To better identify these layers, a relative humidity test was added to the CATS CAD algorithm that identifies horizontally persistent layers with top altitudes greater than 10 km, mid-layer temperatures less than −20 C, and relatively weak backscatter intensity (layer-integrated perpendicular backscatter less than 0.001). If the mean MERRA-2 relative humidity for the layer is less than 45%, then the layer is classified as an aerosol and assigned a Feature Type Score of −6.
Appendix B. CATS Denoising and CNN Architecture
Appendix B.1. Denoising Methods
- Principal Component Analysis—PCA is a widely used statistical technique for the problem of dimensionality reduction [58]. PCA uses an orthogonal basis-transformation technique that maps a collection of n-dimensional vectors V ⊂n into a new basis B = {e1, e2, ..., en} of principal components so that the projection of V onto the first principal component e1’ has the largest possible variance, and each additional principal component accounts for as much variance as possible. Then the first k < n principal components of v ∈ V provide a lower-dimension representation of V. Intuitively, the first principal component is the best 1-dimensional representation of V, and the first k principal components represent the best k-dimensional representation of V. In practice, PCA is performed using a singular value decomposition of the covariance matrix of V. The result is a fast algorithm that operates without labeled training data.
- Wavelet—The Wavelet denoising technique is described in Chang et al. (2000) [50]. The input image is decomposed using a discrete wavelet transform. Thresholds, based on the noise, are used on the higher-resolution wavelet coefficients to remove noise, while leaving the lower-resolution coefficients unmodified. The image is then recomposed from the wavelet coefficients under threshold.
- Butterworth—A Butterworth filter is applied in the spectral domain. For denoising, the low-pass variant of the Butterworth filter is used. To apply a Butterworth filter, first a Fast Fourier Transform (FFT) is applied to the image. Then the resultant frequencies are filtered with the Butterworth low-pass transfer function shown in Equation A1. In this equation, H(u, v) represents the filtered frequency array, D(u, v) is the distance from point (u, v) to the center of the filter, is the cutoff frequency, and n is the order. Once filtered, the frequency array is converted back to the spatial domain using an inverse FFT [52].
- Gaussian filtering removes noise by convolving a 2D kernel, representing a 2D Gaussian function, with a given image. This is particularly effective at removing Gaussian noise, but comes at the expense of blurring the image [59]. Since lidars use photon-counting detectors to collect light, the signal noise is a Poisson distribution, however, Gaussian filtering was still tested for comparison.
Appendix B.2. Wavelet Parameter Determination Detail
Appendix B.3. CATS CNN Architecture
References
- Stephens, G.L. Cloud Feedbacks in the Climate System: A Critical Review. J. Clim. 2005, 18, 237–273. [Google Scholar] [CrossRef] [Green Version]
- Rajeevan, M.; Srinivasan, J. Net Cloud Radiative Forcing at the Top of the Atmosphere in the Asian Monsoon Region. J. Clim. 2000, 13, 650–657. [Google Scholar] [CrossRef]
- Ackerman, T.P.; Liou, K.N.; Valero, F.P.J.; Pfister, L. Heating Rates in Tropical Anvils. J. Atmos. Sci. 1988, 45, 1606–1623. [Google Scholar] [CrossRef]
- McFarquhar, G.M.; Heymsfield, A.J.; Spinhirne, J.; Hart, B. Thin and Subvisual Tropopause Tropical Cirrus: Observations and Radiative Impacts. J. Atmos. Sci. 2000, 57, 1841–1853. [Google Scholar] [CrossRef]
- Campbell, J.R.; Lolli, S.; Lewis, J.R.; Gu, Y.; Welton, E.J. Daytime Cirrus Cloud Top-of-the-Atmosphere Radiative Forcing Properties at a Midlatitude Site and Their Global Consequences. J. Appl. Meteorol. Climatol. 2016, 55, 1667–1679. [Google Scholar] [CrossRef] [PubMed]
- Yu, H.; Kaufman, Y.J.; Chin, M.; Feingold, G.; Remer, L.A.; Anderson, T.L.; Balkanski, Y.; Bellouin, N.; Boucher, O.; Christopher, S.; et al. A review of measurement-based assessments of the aerosol direct radiative effect and forcing. Atmos. Chem. Phys. 2006, 6, 613–666. [Google Scholar] [CrossRef] [Green Version]
- Yorks, J.E.; McGill, M.; Rodier, S.; Vaughan, M.; Hu, Y.; Hlavka, D. Radiative effects of African dust and smoke observed from Clouds and the Earth’s Radiant Energy System (CERES) and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) data. J. Geophys. Res. Atmos. 2009, 114. [Google Scholar] [CrossRef]
- Charlson, R.J.; Schwartz, S.E.; Hales, J.M.; Cess, R.D.; Coakley, J.A.; Hansen, J.E.; Hofmann, D.J. Climate Forcing by Anthropogenic Aerosols. Science 1992, 255, 423–430. [Google Scholar] [CrossRef] [PubMed]
- Haywood, J.; Boucher, O. Estimates of the direct and indirect radiative forcing due to tropospheric aerosols: A review. Rev. Geophys. 2000, 38, 513–543. [Google Scholar] [CrossRef]
- Lelieveld, J.; Evans, J.S.; Fnais, M.; Giannadaki, D.; Pozzer, A. The contribution of outdoor air pollution sources to premature mortality on a global scale. Nature 2015, 525, 367–371. [Google Scholar] [CrossRef]
- McGill, M.; Hlavka, D.; Hart, W.; Scott, V.S.; Spinhirne, J.; Schmid, B. Cloud Physics Lidar: Instrument description and initial measurement results. Appl. Opt. 2002, 41, 3725–3734. [Google Scholar] [CrossRef] [Green Version]
- Yorks, J.E.; Hlavka, D.L.; Vaughan, M.A.; McGill, M.J.; Hart, W.D.; Rodier, S.; Kuehn, R. Airborne validation of cirrus cloud properties derived from CALIPSO lidar measurements: Spatial properties. J. Geophys. Res. Atmos. 2011, 116. [Google Scholar] [CrossRef]
- Winker, D.M.; Vaughan, M.A.; Omar, A.; Hu, Y.; Powell, K.A.; Liu, Z.; Hunt, W.H.; Young, S.A. Overview of the CALIPSO Mission and CALIOP Data Processing Algorithms. J. Atmos. Ocean. Technol. 2009, 26, 2310–2323. [Google Scholar] [CrossRef]
- Chepfer, H.; Bony, S.; Winker, D.; Chiriaco, M.; Dufresne, J.L.; Sèze, G. Use of CALIPSO lidar observations to evaluate the cloudiness simulated by a climate model. Geophys. Res. Lett. 2008, 35. [Google Scholar] [CrossRef] [Green Version]
- Chepfer, H.; Bony, S.; Winker, D.; Cesana, G.; Dufresne, J.L.; Minnis, P.; Stubenrauch, C.J.; Zeng, S. The GCM-Oriented CALIPSO Cloud Product (CALIPSO-GOCCP). J. Geophys. Res. Atmos. 2010, 115. [Google Scholar] [CrossRef]
- Mace, G.G.; Zhang, Q. The CloudSat radar-lidar geometrical profile product (RL-GeoProf): Updates, improvements, and selected results. J. Geophys. Res. Atmos. 2014, 119, 9441–9462. [Google Scholar] [CrossRef]
- Hong, Y.; Liu, G.; Li, J.L.F. Assessing the Radiative Effects of Global Ice Clouds Based on CloudSat and CALIPSO Measurements. J. Clim. 2016, 29, 7651–7674. [Google Scholar] [CrossRef]
- Zhou, C.; Zelinka, M.D.; Klein, S.A. Impact of decadal cloud variations on the Earth’s energy budget. Nat. Geosci. 2016, 9, 871–874. [Google Scholar] [CrossRef]
- Chand, D.; Wood, R.; Anderson, T.L.; Satheesh, S.K.; Charlson, R.J. Satellite-derived direct radiative effect of aerosols dependent on cloud cover. Nat. Geosci. 2009, 2, 181–184. [Google Scholar] [CrossRef]
- Yu, H.; Chin, M.; Winker, D.M.; Omar, A.H.; Liu, Z.; Kittaka, C.; Diehl, T. Global view of aerosol vertical distributions from CALIPSO lidar measurements and GOCART simulations: Regional and seasonal variations. J. Geophys. Res. Atmos. 2010, 115. [Google Scholar] [CrossRef] [Green Version]
- Huang, L.; Jiang, J.H.; Tackett, J.L.; Su, H.; Fu, R. Seasonal and diurnal variations of aerosol extinction profile and type distribution from CALIPSO 5-year observations. J. Geophys. Res. Atmos. 2013, 118, 4572–4596. [Google Scholar] [CrossRef]
- Nowottnick, E.P.; Colarco, P.R.; Welton, E.J.; da Silva, A. Use of the CALIOP vertical feature mask for evaluating global aerosol models. Atmos. Meas. Tech. 2015, 8, 3647–3669. [Google Scholar] [CrossRef] [Green Version]
- McGill, M.J.; Yorks, J.E.; Scott, V.S.; Kupchock, A.W.; Selmer, P.A. The Cloud-Aerosol Transport System (CATS): A technology demonstration on the International Space Station. In Lidar Remote Sensing for Environmental Monitoring XV; International Society for Optics and Photonics: Bellingham, WA, USA, 2015; Volume 9612, p. 96120A. [Google Scholar] [CrossRef]
- Yorks, J.E.; McGill, M.J.; Palm, S.P.; Hlavka, D.L.; Selmer, P.A.; Nowottnick, E.P.; Vaughan, M.A.; Rodier, S.D.; Hart, W.D. An overview of the CATS level 1 processing algorithms and data products. Geophys. Res. Lett. 2016, 43, 4632–4639. [Google Scholar] [CrossRef] [Green Version]
- Gelaro, R.; McCarty, W.; Suárez, M.J.; Todling, R.; Molod, A.; Takacs, L.; Randles, C.A.; Darmenov, A.; Bosilovich, M.G.; Reichle, R.; et al. The Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). J. Clim. 2017, 30, 5419–5454. [Google Scholar] [CrossRef]
- Noel, V.; Chepfer, H.; Chiriaco, M.; Yorks, J. The diurnal cycle of cloud profiles over land and ocean between 51°S and 51°N, seen by the CATS spaceborne lidar from the International Space Station. Atmos. Chem. Phys. 2018, 18, 9457–9473. [Google Scholar] [CrossRef] [Green Version]
- Lee, L.; Zhang, J.; Reid, J.S.; Yorks, J.E. Investigation of CATS aerosol products and application toward global diurnal variation of aerosols. Atmos. Chem. Phys. 2019, 19, 12687–12707. [Google Scholar] [CrossRef] [Green Version]
- Chepfer, H.; Brogniez, H.; Noel, V. Diurnal variations of cloud and relative humidity profiles across the tropics. Sci. Rep. 2019, 9, 16045. [Google Scholar] [CrossRef] [Green Version]
- Yu, Y.; Kalashnikova, O.V.; Garay, M.J.; Lee, H.; Choi, M.; Okin, G.S.; Yorks, J.E.; Campbell, J.R.; Marquis, J. A global analysis of diurnal variability in dust and dust mixture using CATS observations. Atmos. Chem. Phys. 2021, 21, 1427–1447. [Google Scholar] [CrossRef]
- Rajapakshe, C.; Zhang, Z.; Yorks, J.E.; Yu, H.; Tan, Q.; Meyer, K.; Platnick, S.; Winker, D.M. Seasonally transported aerosol layers over southeast Atlantic are closer to underlying clouds than previously reported. Geophys. Res. Lett. 2017, 44, 5818–5825. [Google Scholar] [CrossRef] [PubMed]
- Christian, K.; Wang, J.; Ge, C.; Peterson, D.; Hyer, E.; Yorks, J.; McGill, M. Radiative Forcing and Stratospheric Warming of Pyrocumulonimbus Smoke Aerosols: First Modeling Results With Multisensor (EPIC, CALIPSO, and CATS) Views from Space. Geophys. Res. Lett. 2019, 46, 10061–10071. [Google Scholar] [CrossRef] [Green Version]
- Hughes, E.J.; Yorks, J.; Krotkov, N.A.; Silva, A.M.d.; McGill, M. Using CATS near-real-time lidar observations to monitor and constrain volcanic sulfur dioxide (SO2) forecasts. Geophys. Res. Lett. 2016, 43, 11,089–11,097. [Google Scholar] [CrossRef]
- McGill, M.J.; Swap, R.J.; Yorks, J.E.; Selmer, P.A.; Piketh, S.J. Observation and quantification of aerosol outflow from southern Africa using spaceborne lidar. S. Afr. J. Sci. 2020, 116, 1–6. [Google Scholar] [CrossRef] [Green Version]
- O’Sullivan, D.; Marenco, F.; Ryder, C.L.; Pradhan, Y.; Kipling, Z.; Johnson, B.; Benedetti, A.; Brooks, M.; McGill, M.; Yorks, J.; et al. Models transport Saharan dust too low in the atmosphere: A comparison of the MetUM and CAMS forecasts with observations. Atmos. Chem. Phys. 2020, 20, 12955–12982. [Google Scholar] [CrossRef]
- Pauly, R.M.; Yorks, J.E.; Hlavka, D.L.; McGill, M.J.; Amiridis, V.; Palm, S.P.; Rodier, S.D.; Vaughan, M.A.; Selmer, P.A.; Kupchock, A.W.; et al. Cloud-Aerosol Transport System (CATS) 1064 nm calibration and validation. Atmos. Meas. Tech. 2019, 12, 6241–6258. [Google Scholar] [CrossRef] [Green Version]
- Vaughan, M.A.; Powell, K.A.; Winker, D.M.; Hostetler, C.A.; Kuehn, R.E.; Hunt, W.H.; Getzewich, B.J.; Young, S.A.; Liu, Z.; McGill, M.J. Fully Automated Detection of Cloud and Aerosol Layers in the CALIPSO Lidar Measurements. J. Atmos. Ocean. Technol. 2009, 26, 2034–2050. [Google Scholar] [CrossRef]
- Yorks, J.E.; Palm, S.P.; McGill, M.J.; Hlavka, D.L.; Hart, W.D.; Selmer, P.A.; Nowottnick, E.P. CATS Algorithm Theoretical Basis Document: Level 1 and Level 2 Data Products; Technical Report; NASA Goddard Space Flight Center: Greenbelt, MD, USA, 2016. [Google Scholar]
- Vaughan, M.A.; Winker, D.M.; Powell, K.A. Part 2: Feature Detection and Layer Properties Algorithms; CALIOP Algorithm Theoretical Basis Document PC-SCI-202.01; Technical Report; NASA Langley Research Center: Hampton, VA, USA, 2005. [Google Scholar]
- Getzewich, B.J.; Vaughan, M.A.; Hunt, W.H.; Avery, M.A.; Powell, K.A.; Tackett, J.L.; Winker, D.M.; Kar, J.; Lee, K.P.; Toth, T.D. CALIPSO lidar calibration at 532 nm: Version 4 daytime algorithm. Atmos. Meas. Tech. 2018, 11, 6309–6326. [Google Scholar] [CrossRef] [Green Version]
- Torres, O.; Ahn, C.; Chen, Z. Improvements to the OMI near-UV aerosol algorithm using A-train CALIOP and AIRS observations. Atmos. Meas. Tech. 2013, 6, 3257–3270. [Google Scholar] [CrossRef] [Green Version]
- Jethva, H.; Torres, O.; Waquet, F.; Chand, D.; Hu, Y. How do A-train sensors intercompare in the retrieval of above-cloud aerosol optical depth? A case study-based assessment. Geophys. Res. Lett. 2014, 41, 186–192. [Google Scholar] [CrossRef] [Green Version]
- Liu, Q.; Ma, X.; Jin, H.; Chen, Y.; Yu, Y.; Zhang, H.; Cai, C.; Wang, Y.; Li, H. Seasonal variation of aerosol vertical distributions in the middle and lower troposphere in Beijing and surrounding area during haze periods based on CALIPSO observation. In Lidar Remote Sensing for Environmental Monitoring XIV; International Society for Optics and Photonics: Bellingham, WA, USA, 2014; Volume 9262, p. 92620J. [Google Scholar] [CrossRef]
- Davis, K.J.; Gamage, N.; Hagelberg, C.R.; Kiemle, C.; Lenschow, D.H.; Sullivan, P.P. An Objective Method for Deriving Atmospheric Structure from Airborne Lidar Observations. J. Atmos. Ocean. Technol. 2000, 17, 1455–1468. [Google Scholar] [CrossRef] [Green Version]
- Liu, Z.; Vaughan, M.; Winker, D.; Kittaka, C.; Getzewich, B.; Kuehn, R.; Omar, A.; Powell, K.; Trepte, C.; Hostetler, C. The CALIPSO Lidar Cloud and Aerosol Discrimination: Version 2 Algorithm and Initial Assessment of Performance. J. Atmos. Ocean. Technol. 2009, 26, 1198–1213. [Google Scholar] [CrossRef]
- Liu, Z.; Kar, J.; Zeng, S.; Tackett, J.; Vaughan, M.; Avery, M.; Pelon, J.; Getzewich, B.; Lee, K.P.; Magill, B.; et al. Discriminating between clouds and aerosols in the CALIOP version 4.1 data products. Atmos. Meas. Tech. 2019, 12, 703–734. [Google Scholar] [CrossRef] [Green Version]
- Yorks, J.E.; Hlavka, D.L.; Hart, W.D.; McGill, M.J. Statistics of Cloud Optical Properties from Airborne Lidar Measurements. J. Atmos. Ocean. Technol. 2011, 28, 869–883. [Google Scholar] [CrossRef]
- Christian, K.; Yorks, J.; Das, S. Differences in the Evolution of Pyrocumulonimbus and Volcanic Stratospheric Plumes as Observed by CATS and CALIOP Space-Based Lidars. Atmosphere 2020, 11, 1035. [Google Scholar] [CrossRef]
- Dolinar, E.K.; Campbell, J.R.; Lolli, S.; Ozog, S.C.; Yorks, J.E.; Camacho, C.; Gu, Y.; Bucholtz, A.; McGill, M.J. Sensitivities in Satellite Lidar-Derived Estimates of Daytime Top-of-the-Atmosphere Optically Thin Cirrus Cloud Radiative Forcing: A Case Study. Geophys. Res. Lett. 2020, 47, e2020GL088871. [Google Scholar] [CrossRef]
- Zavyalov, V.V.; Bingham, G.E.; Wojcik, M.; Johnson, H.; Struthers, M. Application of principal component analysis to lidar data filtering and analysis. In Lidar Technologies, Techniques, and Measurements for Atmospheric Remote Sensing V; International Society for Optics and Photonics: Bellingham, WA, USA, 2009; Volume 7479, p. 747907. [Google Scholar] [CrossRef]
- Chang, S.G.; Yu, B.; Vetterli, M. Adaptive wavelet thresholding for image denoising and compression. IEEE Trans. Image Process. 2000, 9, 1532–1546. [Google Scholar] [CrossRef] [Green Version]
- Walt, S.v.d.; Schönberger, J.L.; Nunez-Iglesias, J.; Boulogne, F.; Warner, J.D.; Yager, N.; Gouillart, E.; Yu, T. Scikit-image: Image processing in Python. PeerJ 2014, 2, e453. [Google Scholar] [CrossRef]
- Samagaio, G.; de Moura, J.; Novo, J.; Ortega, M. Optical Coherence Tomography Denoising by Means of a Fourier Butterworth Filter-Based Approach. In International Conference on Image Analysis and Processing, Proceedings of the Image Analysis and Processing—ICIAP 2017, 19th International Conference, Catania, Italy, 11–15 September 2017; Battiato, S., Gallo, G., Schettini, R., Stanco, F., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2017; pp. 422–432. [Google Scholar] [CrossRef]
- Gidaris, S.; Komodakis, N. Object Detection via a Multi-Region and Semantic Segmentation-Aware CNN Model. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015. [Google Scholar]
- Pradhan, R.; Aygun, R.S.; Maskey, M.; Ramachandran, R.; Cecil, D.J. Tropical Cyclone Intensity Estimation Using a Deep Convolutional Neural Network. IEEE Trans. Image Process. 2018, 27, 692–702. [Google Scholar] [CrossRef]
- Maskey, M.; Ramachandran, R.; Miller, J.J.; Zhang, J.; Gurung, I. Earth Science Deep Learning: Applications and Lessons Learned. In Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 1760–1763. [Google Scholar] [CrossRef] [Green Version]
- Shahin, M.A.; Maier, H.R.; Jaksa, M.B. Data Division for Developing Neural Networks Applied to Geotechnical Engineering. J. Comput. Civ. Eng. 2004, 18, 105–114. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, 18th International Conference, Munich, Germany, 5–9 October 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar] [CrossRef] [Green Version]
- Huang, H.L.; Antonelli, P. Application of Principal Component Analysis to High-Resolution Infrared Measurement Compression and Retrieval. J. Appl. Meteorol. Climatol. 2001, 40, 365–388. [Google Scholar] [CrossRef]
- Virtanen, P.; Gommers, R.; Oliphant, T.E.; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J.; et al. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nat. Methods 2020, 17, 261–272. [Google Scholar] [CrossRef] [Green Version]
Parameter | Interpretation |
---|---|
Feature_Type | 0 = Invalid |
1 = Cloud | |
2 = Undetermined | |
3 = Aerosol | |
Feature_Type_Score | | 10 | = high confidence |
| 1 | = low confidence | |
0 = zero confidence | |
Cloud_Phase | 0 = Invalid |
1 = Water Cloud | |
2 = Unknown Cloud Phase | |
3 = Ice Cloud | |
Cloud_Phase_Score | | 10 | = high confidence |
| 1 | = low confidence | |
0 = zero confidence | |
Aerosol_Type | 0 = Invalid |
1 = Marine | |
2 = Polluted Marine | |
3 = Dust | |
4 = Dust mixture | |
5 = Clean/Background | |
6 = Polluted Continental | |
7 = Smoke | |
8 = UTLS |
Technique | Parameters |
---|---|
PCA | number of components to keep |
Wavelet | wavelet type, number of decomposition levels, noise standard deviation |
Butterworth | cutoff frequency, order |
Gaussian | standard deviation for kernel |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yorks, J.E.; Selmer, P.A.; Kupchock, A.; Nowottnick, E.P.; Christian, K.E.; Rusinek, D.; Dacic, N.; McGill, M.J. Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data. Atmosphere 2021, 12, 606. https://doi.org/10.3390/atmos12050606
Yorks JE, Selmer PA, Kupchock A, Nowottnick EP, Christian KE, Rusinek D, Dacic N, McGill MJ. Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data. Atmosphere. 2021; 12(5):606. https://doi.org/10.3390/atmos12050606
Chicago/Turabian StyleYorks, John E., Patrick A. Selmer, Andrew Kupchock, Edward P. Nowottnick, Kenneth E. Christian, Daniel Rusinek, Natasha Dacic, and Matthew J. McGill. 2021. "Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data" Atmosphere 12, no. 5: 606. https://doi.org/10.3390/atmos12050606
APA StyleYorks, J. E., Selmer, P. A., Kupchock, A., Nowottnick, E. P., Christian, K. E., Rusinek, D., Dacic, N., & McGill, M. J. (2021). Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data. Atmosphere, 12(5), 606. https://doi.org/10.3390/atmos12050606