Novel Deep Level Image State Ensemble Enhancement Method for M87 Imaging

: Standard spatial domain ﬁlters fail to adequately denoise and enhance the contrast of an image. These ﬁlters have drawbacks like oversmoothing, diminished texture, and lack of generative capabilities. This paper proposes a new method of image reconstruction, Image State Ensemble Enhancement (ISEE), based on our previous work, Image State Ensemble Decomposition (ISED). Deep level ISEE and ISED have been developed to produce a class of ﬁlters that can address these issues. Full-reference and no-reference quality metrics are used to assess the image, and the full reference metrics showed a marked improvement, while the no-reference metrics were often better than the test image. The test image was taken from the Spitzer Space Telescope (SST), and ISEE reconstruction yielded improved structural detail over that of ISED and the original test image. Glare and noise were reduced in a narrow bandwidth, which led to the discovery of a vortex-shaped structure and an outburst in M87 (cid:48) s dusty infrared core. The vortex is located over M87 (cid:48) s visible core and black hole. This is veriﬁed with an SST and Hubble Space Telescope (HST) overlay, ISEE processed image. A counter-jet channel was also discovered, and it appears to be the path of the unobservable superluminal counter-jet. methodology, C.-T.C.; investigation,


Introduction
This paper investigates a new method to enhance features and improve color image processing. The Image State Ensemble Enhancement (ISEE) and Image State Ensemble Decomposition (ISED) methods can denoise and enhance contrast. Although the topic we have chosen to analyze is astronomical, these methods can easily be applied to color medical imaging. Denoising and contrast enhancement are commonly applied to medical imaging and have been the subject of intense research [1]. ISED and ISEE reconstructions can denoise, enhance the contrast, and provide feature extraction. ISEE improves the full-reference metrics and narrows the bandwidth of the filter, which allows for the extraction of finer details than ISED. See our unpublished histological image examples of ISED and ISEE reconstructions available online at [2] for more details.
The proposed deep level ISED and ISEE reconstructions differ from traditional types of filters. ISED and ISEE rely on some basic precepts of quantum mechanics to statistically determine and generate filtered color images based on possible image outcomes [3]. ISED and ISEE reconstructions are performed in the spatial domain. Common spatial domain filters often rely on convolution kernels or a window to modify a group of pixels [4]. Mean filtering and median filtering are common methods to smooth the image to reduce types of noise, but at the cost of adding some blurring to the image [5]. ISED and ISEE use a pixel-wise cross-channel relationship that does not suffer from blurring issues. the black hole or perhaps was formed from another outburst from the black hole. The mass is a clump of matter like that of the globular structure that apparently obscures regions of the eastbound visible spectrum superluminal jet, perhaps a globular cluster because so many are prevalent in M87 [17]. As a final point, within M87 s dusty IR core, there appears to be a direct disturbance caused by the proximity of the supermassive black hole. In order to further analyze the findings, a Sobel filter was applied to a tighter crop of the galactic core image to produce images of the Sobel gradient magnitude, directional gradients, and gradient direction. Furthermore, image differencing with intensity scaling was used to build an additional set of images to compare the ISEE images to the original image [18]. Several overlays were also constructed from the SST ISED test image and a Hubble Space Telescope (HST) image [19]. This was done to compare the structural features of the IR chromatic ordered test image and a visible/IR chromatic ordered HST image. The overlay images are in Appendix A and will be discussed briefly. Of note, the dusty IR core, when compared in the ISEE processed HST in the overlay, shows that the IR core is offcenter and roughly aligns with the western edge of dusty IR core (SST) and the smaller visible core (HST). The IR core is much larger and encompasses HST-1. HST-1 appears to be roughly the center of the radio core and the center of the dust ring. Hence, this paper proposes a novel method of image processing while simultaneously revealing astronomical phenomena via applied deep Ln ISED and ISEE reconstruction.
The paper is organized as follows. Section 2 provides the mathematical framework to construct deep Ln ISED and ISEE images and filters. Section 3 provides an overview of the materials and methods used to generate deep Ln ISED and ISEE images. Section 4 presents the experimental results and discussion. Finally, the conclusion in Section 5 explains the impact of deep Ln ISED, ISEE, and the astronomical findings in this study.

Mathematics of Image State Decomposition and Enhancement
In order to produce deep Ln ISEE images, one must first perform deep Ln ISED. The notation of ISED has been modified to create what we term as deep Ln generated images and filters. As Ln increases, the SSIM, PSNR, and IMSE decreases. From our previous study, Theorem 1 is proposed. The ψ-ensemble matrices in (2), designed for an RGB image where ψ m,n ≥ 0, are nonnegative ensemble matrices. For the sake of brevity in the notation, the ij index values will be removed from the following formula. The mathematics is applied pixel-wise and across the color channels to build the ψ-ensemble relations; it is of note for the sake of formalism to indicate the index notation. The m index in the following formula is the level of decomposition (Ln), and the integers 1 to 6 represent the first six levels of ISED decomposition demonstrated in this study. For clarity, the notation L6 is the sixth level of decomposition, and L3 would indicate the third level of decomposition. The n index in the following formula refers to the particular ψ-ensemble matrices ψ m,n (state settings). The matrices in (2) are the possible ensemble relations between three color channels. R, G, and B. There can be between two to n channels for this method, but the ψ-ensemble matrices must be formulated for application-specific purposes [1]. The [0, 1] in our case is a binary decision in our study, 0 = not detected (off) and 1 = detected (on).
To clarify the notation, the example ψ 3,2j (3) is the third decomposition of the ensemble matrix, recall the pixel-wise ij indices are dropped for brevity.

Image States Ensemble Decomposition for Deep Levels (Ln)
The following Equation (4) is the basis of formulating deep Ln ISED equations and plays an integral role in the construction of ISEE image generation. This has been modified for deeper Ln and extends the notation from the previous study [3].
For the example of how the notation in (4) would be written for an L2 ISED image, you would use the above equation to generate the 63 balanced ISED images for the second level of decomposition using Algorithm 1 (L2), where the 63rd state S 2,63 = (RGB)' is used to generate the image in Figure 1c. I hope to avoid as much ambiguity as possible by providing the example equation in (5). These images are different from the previous study [1]. An L6 ISED decomposition would have 36 ψ-ensemble matrices per generated channel. An L1 ISED would have 6 ψ-ensemble matrices per generated channel. In (5) below is an example of an L2 ISED that has 12 ψ-ensemble matrices per generated channel.

Example Formula for a Deep L2 ISED
The second level of decomposition (L2), where the 63rd state S 2,63 = (RGB)'. I chose to reiterate the meaning of L2 because I wish to avoid confusing the reader when involving new notation and implementing a novel image processing technique. The formula in (5) would be one possible method used to generate L2 ISED images and filters. It is used in combination with Algorithm 1 because the decomposition builds on the previous results of the lower level decompositions. So, L1 needs to be processed prior to processing L2, and the L1 results are used to build L2. This process can be used up to Ln. This is the expanded form of (4) for an L2 ISED image.

Filter Recovery for a Deep L2 ISED
The ISED filter for this example can be recovered for L2 by using the following equation derived from (5) or (8), The S − in Equations (6) and (7) is the notation to indicate that it is the Ln ISED filter. So, the deep Ln ISED image is subtracted from the original image for a simple way to determine the filtered information from the image. This is by far the simplest method to recover the image filter information.

General Equation for Filter Recovery
In general, the ISED filter can be generated by, where (RGB)' is the original image and (RGB) m,n is the Ln ISED image. S − m,n is the recovered filter in (7), modified from our previous work in [3] and derived from (5) or (8).

Balanced Image State Ensemble Decomposition for Ln
The following formula in (8) is for balanced state Ln ISED image generation, modified from our previous work in [3].

ISEE Reconstruction Matrices
For levels of decomposition, τ m,n in (9) represents the Ln ISEE image reconstruction. S m,63 is used because the 63rd state Ln ISED image has the most information removed when compared to all other ISED images states. If you refer to Table 1, you can see that all the ψ-ensemble states are activated (on). This indicates that the maximum amount of information is filtered out of the original image, and the resulting ISED image contains the least amount of information when compared to the original image. Incidentally, it would have the poorest full-reference quality metrics as well. The logic behind this is so that there is no overlapping of image data in the reconstruction and oversaturation in the image for a finite range of colors. The restoration will produce an image closer to the original image but with a narrower band of colors removed from the image as opposed to the previous wideband ISED method. This causes the SSIM to increase and improves PSNR and IMMSE.
With an increase in Ln (further levels of decomposition) used in conjunction with the reconstruction for the five selected ISED states (S m,30, S m,44 , S m,45 , S m,46 , and S m,47 ), the image begins to evolve to a convergence condition resulting in closer similarities to the original image. As one increases Ln for ISEE reconstruction, the greater the similarity for the convergent case. Not all ISEE reconstructions converge, some remain divergent or are partly convergent. This is due to either the intrinsic properties of the image and the lack of information contained within the ISED filter.
ISED is not a single filter but a class of filters, each filter with its own unique attribute that selectively removes a finite bandwidth of information from the image. ISED has the characteristics of a wideband filter. However, since we designed the algorithms to generate many possible filter states, it has the advantage of producing the right tool for the right application. When you take the ISED image with the least amount of information (S m,63 ) and combine the image with the lower state filter images (S − m,n , where n = 1~62) it will produce a narrow filter that remains a subset of the original image. This improves the full-reference quality metrics. The equation for ISEE in (9) was derived from a deep understanding of how ISED images and filters operate. You could choose other filters of other ISED images other than S m,63 , but this would cause an oversaturation in regions of the image. The generative image would be outside of the domain of the original image. It was not our desire to have this property, although, in some cases, it may enhance features. Additionally, it would produce a generative image further from the ground base truth. In ISED, S m,63 is like the skeleton of the image, and the combined dissimilar ISED filter restores a great deal of information to produce the ISEE reconstruction. τ m,n = S m,63 + S − m,n where, n = 1 ∼ 62 and m = 1 ∼ Ln (9) For this study, 378 generated Ln ISED images, 378 generated Ln ISED filters, and 372 generated ISEE images were produced from the SST test image.

Materials and Methods
For this study, we used Table 1 and (4) to generate the S m,63 ISED images that are used in part to produce ISEE images from (9). Table 2 is the settings for generating the filters S − m,n , which are recovered by processing Equations (4) and then (7). States 30, 44, 45, 46, and 47 were chosen to provide a sample of the image set that was produced. In total, for L1~6, there were 372 ISEE generated images. This sample of ISEE images are evaluated and will be discussed in further detail in the following section. Algorithm 1 shows the process involved in generating ISED images, filters, and ISEE reconstruction. Table 1 displays the ensemble matrices state settings to generate S m,63 for ISEE generation. The generated images from Table 1 can be seen in Figure 1b-g. Please recall that in order to produce ISEE reconstructions, you must first perform ISED. The S m,n first comes from ISED; then, it is applied to Equation (9).
3. Set the [0, 1] state conditions for ψ m,1 , ψ m,2 , ψ m,3 , ψ m,4 , ψ m,5 , and ψ m,6 . 4. Use the ISED equation and set constant n to obtain Ln levels of decomposition, The equation forms the modified R m,n , G m,n , and B m,n color channels. 5. Recombine the color channels to form a newly generated ISED image, (RGB) m,n . 7. Subtract the ISED image from the original image (RGB)' to obtain the ISED filter image (7) S − m,n for n = 1 ∼ 62 and m = 1 ∼ 6 for six levels of decomposition. This will generate 63 filters and the zero-state filter, which is a black image. 8. Reprocess the ISED output images (RGB) m,n following steps 2~8. When the desired decomposition level is achieved, end. Alternatively, one could code the entire formula; however, L6 would require 36 ensemble matrices for every color channel, so you would be processing 108 matrices to form an image. 9. Combine the 63 rd state ISED image (S m,63 ) to each corresponding decomposition level with the Ln equivalent filter S − m,n such that S m,63 + S − m,n , where n = 1 ∼ 62 and m = 1 ∼ 6 to produce the ISEE image τ m,n . τ m,n = S m,63 + S − m,n .
By using Algorithm 1, there were 372 ISEE images that were generated. Full-reference and no-reference quality metrics were assessed to determine the properties of each of the generated deep Ln ISEE images. These images were then compared to past ISED results to determine if there was an overall improvement in the full-reference quality metrics. This will be discussed in detail in the Results and Discussion section.

Results and Discussion
The research objectives with deep Ln ISED and ISEE were to develop a new method of image processing that can selectively remove artifacts, types of glare, various distortions of an image, extract features, and simultaneously enhance contrast and features of an image. It was not a simple task; however, after extensively studying the most common image processing techniques, we believed that we could fill the image processing gap of knowledge by providing these useful techniques.
Basic quantum mechanics played a key role in formulating Theorem 1 and the resulting ISED and ISEE equations. Deep Ln ISED and ISEE can filter out some of the unwanted effects and, as a byproduct, enhance others. In this section, we shall demonstrate the results of this study.
In the proposed ISEE research, Algorithm 1 was performed on a postprocessed SST image inset. The chromatically ordered image was obtained from NASA/JPL-Caltech/IPAC [18], and a crop of the M87 image was taken from an 11.1MB Tagged Image Format (TIF) file, "ssc2019-05b.tif", available online at [19]. The image file had a resolution of 3580 × 3580 pixels prior to being cropped. The cropped test image was 322KB in size and had a resolution of 730 × 733 pixels. GIMP 2.10.14 was used to crop the image and it was saved as a lossless Portable Network Graphics file (PNG). Another tighter crop was taken to more clearly see the structures in the vicinity of M87 s IR core. The second crop had a resolution of 104 × 104 pixels, and the file size was 7.33 KB. The nominal values of the red, green, and blue channels were mapped from the infrared radiation wavelengths of 8.0, 4.5, and 3.5 µm, respectively [18]. The overlay images in Appendix A were constructed from combinations of different L6 ISEE images from the larger crop and a second L6 ISEE-generated image taken from the HST. The original HST TIFF file, "heic0815f.tif", had a resolution 3854 × 3815, was 20.8 MB, and can be found online at [19]. The nominal values of the red, green, and blue channels were mapped from visible and infrared radiation wavelengths of 475, 606, and 814 nm, respectively [19]. The hardware configuration consisted of a standard gaming laptop with an Intel i7-8750H processor, 32 GB of RAM, and an NVidia GeForce GTX 1060 graphics processing unit. Matlab 2019a and Matlab 2020a were used to implement the algorithms.

Results and Discussion
The research objectives with deep Ln ISED and ISEE were to develop a new method of image processing that can selectively remove artifacts, types of glare, various distortions of an image, extract features, and simultaneously enhance contrast and features of an image. It was not a simple task; however, after extensively studying the most common image processing techniques, we believed that we could fill the image processing gap of knowledge by providing these useful techniques. Basic quantum mechanics played a key role in formulating Theorem 1 and the resulting ISED and ISEE equations. Deep Ln ISED and ISEE can filter out some of the unwanted effects and, as a byproduct, enhance others. In this section, we shall demonstrate the results of this study.
In the proposed ISEE research, Algorithm 1 was performed on a postprocessed SST image inset. The chromatically ordered image was obtained from NASA/JPL-Caltech/IPAC [18], and a crop of the M87 image was taken from an 11.1MB Tagged Image Format (TIF) file, "ssc2019-05b.tif", available online at [19]. The image file had a resolution of 3580 × 3580 pixels prior to being cropped. The cropped test image was 322KB in size and had a resolution of 730 × 733 pixels. GIMP 2.10.14 was used to crop the image and it was saved as a lossless Portable Network Graphics file (PNG). Another tighter crop was taken to more clearly see the structures in the vicinity of M87′s IR core. The second crop had a resolution of 104 × 104 pixels, and the file size was 7.33 KB. The nominal values of the red, green, and blue channels were mapped from the infrared radiation wavelengths of 8.0, 4.5, and 3.5 μm, respectively [18]. The overlay images in Appendix A were constructed from combinations of different L6 ISEE images from the larger crop and a second L6 ISEE-generated image taken from the HST. The original HST TIFF file, "heic0815f.tif", had a resolution 3854 × 3815, was 20.8 MB, and can be found online at [19]. The nominal values of the red, green, and blue channels were mapped from visible and infrared radiation wavelengths of 475, 606, and 814 nm, respectively [19]. The hardware configuration consisted of a standard gaming laptop with an Intel i7-8750H processor, 32 GB of RAM, and an NVidia GeForce GTX 1060 graphics processing unit. Matlab 2019a and Matlab 2020a were used to implement the algorithms.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 10 of 27 (e) (f) (g) In general, the information contained in the ISED image is shifted to the ISED filter as the levels of decomposition increase. Figure 1a is the original cropped image of the postprocessed insert taken from the SST [18]. Figure 1b is the first level of decomposition S1,63, created in the previous study of ISED decomposition [1]. Figure 1c is the L2 ISED image of S2,63. Figure 1d is the L3 ISED image of S3,63 and Figure 1e-g are the deep L4, L5, and L6 ISED images S4,63, S5,63, and S6,63, respectively. The images in Figure 1b-g are used in the creation of the proposed ISEE image reconstruction (Sm,63 images for Ln). I will reiterate that this ensemble state configuration was chosen for reconstruction; the reason is that the 63rd state ISED filter for all Ln has the maximum information removed from the image than all other states to avoid overlapping image data. This happens to be fortuitous because as Ln increases, the ISED filter (Sm,n) information increases proportionally. One must first process deep Ln In general, the information contained in the ISED image is shifted to the ISED filter as the levels of decomposition increase. Figure 1a is the original cropped image of the postprocessed insert taken from the SST [18]. Figure 1b is the first level of decomposition S 1,63 , created in the previous study of ISED decomposition [1]. Figure 1c is the L2 ISED image of S 2,63 . Figure 1d is the L3 ISED image of S 3,63 and Figure 1e-g are the deep L4, L5, and L6 ISED images S 4,63 , S 5,63 , and S 6,63 , respectively. The images in Figure 1b-g are used in the creation of the proposed ISEE image reconstruction (S m,63 images for Ln). I will reiterate that this ensemble state configuration was chosen for reconstruction; the reason is that the 63rd state ISED filter for all Ln has the maximum information removed from the image than all other states to avoid overlapping image data. This happens to be fortuitous because as Ln increases, the ISED filter (S − m,n ) information increases proportionally. One must first process deep Ln ISED images and filters in order to obtain ISEE images.
The cropped overlay image, Figure 2 plays a key role in the physical analysis for select deep Ln ISED and ISEE reconstrctions. This analysis will be covered in greater detail in the Conclusion. For the images in Figure 3a-l, you have both Ln ISED and ISEE reconstructions. The Ln ISED images are on the left side, and the corresponding Ln ISEE reconstructions are on the right. Some interesting features stand out in this set of images. First, the Ln ISED images for S m,30 m = 1~6 remove the blues and the reds as the Ln increases. S 6,30 retains the edges of the outer IR dust ring, the IR core, and the IR shockwave structures that are produced by the superluminal jets. The counter-jet channel to the west of the IR core becomes visible. The vortex and the outburst can also be seen in the dusty core in S 6,30 . Additionally, the counter-jet channel seems to be bent approximately 5 • with respect to the left shockwave. This was verified with the HST/SST image overlay in Figure 2, where the SST ISEE generated image τ 6,30 and the HST generated ISEE image τ 6,19 in Figure A1b. The overlay was built in Gimp, and the top layer is the SST image with an 80% opacity. It was also rotated and cropped and saved in the PNG file format. The overlay selection of SST image τ 6,30 and HST image τ 6,19 was chosen because the SST image has the highest SSIM for τ m,30 and the HST image isolated previously unseen wispy features surrounding the core. This was useful in aligning the image and revealing hidden features in the SST image. In general, the information contained in the ISED image is shifted to the ISED filter as the levels of decomposition increase. Figure 1a is the original cropped image of the postprocessed insert taken from the SST [18]. Figure 1b is the first level of decomposition S1,63, created in the previous study of ISED decomposition [1]. Figure 1c is the L2 ISED image of S2,63. Figure 1d is the L3 ISED image of S3,63 and Figure 1e-g are the deep L4, L5, and L6 ISED images S4,63, S5,63, and S6,63, respectively. The images in Figure 1b-g are used in the creation of the proposed ISEE image reconstruction (Sm,63 images for Ln). I will reiterate that this ensemble state configuration was chosen for reconstruction; the reason is that the 63rd state ISED filter for all Ln has the maximum information removed from the image than all other states to avoid overlapping image data. This happens to be fortuitous because as Ln increases, the ISED filter (Sm,n) information increases proportionally. One must first process deep Ln ISED images and filters in order to obtain ISEE images. The cropped overlay image, Figure 2 plays a key role in the physical analysis for select deep Ln ISED and ISEE reconstrctions. This analysis will be covered in greater detail in the Conclusion. For the images in Figure 3a-l, you have both Ln ISED and ISEE reconstructions. The Ln ISED images are on the left side, and the corresponding Ln ISEE reconstructions are on the right. Some interesting features stand out in this set of images. First, the Ln ISED images for Sm,30 m = 1~6 remove the blues and the reds as the Ln increases. S6,30 retains the edges of the outer IR dust ring, the IR core, and the IR shockwave structures that are produced by the superluminal jets. The counter-jet channel to the west of the IR core becomes visible. The vortex and the outburst can also be seen in the dusty core in generated image τ6,30 and the HST generated ISEE image τ6,19 in Figure A1b. The overlay was built in Gimp, and the top layer is the SST image with an 80% opacity. It was also rotated and cropped and saved in the PNG file format. The overlay selection of SST image τ6,30 and HST image τ6,19 was chosen because the SST image has the highest SSIM for τm,30 and the HST image isolated previously unseen wispy features surrounding the core. This was useful in aligning the image and revealing hidden features in the SST image. The IQA metrics displayed in Table 3 for SSIM, PSNR, and MSE, (Defined in Table 4) NIQE, BRISQUE, and PIQE are well understood and are often employed as a standard in image analysis.  The IQA metrics displayed in Table 3 for SSIM, PSNR, and MSE, (Defined in Table 4) NIQE, BRISQUE, and PIQE are well understood and are often employed as a standard in image analysis.  Table 4. Image quality assessment techniques [20,21].

Quality Assessment Definition
Structural Similarity Index (SSIM) for the images x,y: µ x , µ y , σ x ,σ y , and σ xy are the local means, standard deviations, and cross-covariance. C 1 = (K 1 L) 2 , where k‹‹1 is a small constant and L is the dynamic range for the pixel values. Likewise, C 2 = (K 2 L) 2 , where k‹‹1 is a small constant and L is the dynamic range for the pixel values, when α = β = γ = 1.
Peak Signal to Noise Ratio (PSNR) PSNR = 10log 10 The SSIM produces a maximum score of one based on the images' comparative contrast, local structure, and luminescence [20]. The PSNR indicates the peak error of the image and is used to determine compression loss in images and video [22]. Higher PSNR scores are better when compared to images that have gone through a similar process. Therefore, a rough comparison can be drawn between the same state images for varied Ln. Likewise, the MSE is also a good indicator to see the behavior of ISEE for deeper Ln as a comparative trend as a predictor for Ln > 6. Lower MSE scores are better. The NIQE is trained from a database of pristine images and various images with Gaussian distortions, so it can measure arbitrary distortions in the test images; lower scores are better [23]. Additionally, BRISQUE is trained from a database of known image distortions [24,25], while PIQE evaluates the local variance to detect block-wise distortion in order to calculate the quality of an image, where lower scores are better [26,27]. Table 5 is for the assessment of PIQE scores, as recommended by [27]. From Table 5, it is clear as a trend, the SSIM increases apart from the L2 image. From our previous study, the ISED image seen in Figure 3a had an SSIM of 0.87. This is close to the value of the ISEE L6 image τ 6,30 SSIM of 0.84. The PNSR and MSE have both improved, so it is a probable indicator that the ISEE image has less noise. The NIQE is quite close, with S 1,30 equal to 6.31 and τ 6,30 equal to 6.96. The L6 BRISQUE score for the ISEE reconstruction has not improved over that of the ISED image S 1,30 . The PIQE L6 is fairly close, but for the assessment range, S 1,30 falls into the fair category with a score of 48.14, while τ 6,30 is 54.9 in the poor category. The original photo PIQE was 100 and is in the bad category for the PIQE assessment range. For the higher SSIM ISED images, the ISEE images of L1 have better scores for the no-reference quality metrics and lower scores for the full-reference quality metrics. For deep Ln for ISED images S m,30 , there is a decrease in SSIM, PSNR, and MSE. The L1 ISEE image τ 1,44 in Figure 4b has a well-defined outer IR dust ring, and the IR core is pronounced. The shockwaves and surrounding dust structures have good contrast against the black backdrop, where most of the blue channel information has been eliminated except the blue-biased glare portion of the image. Therefore, the image looks a bit blurred. As Ln increases, this blurring effect is reduced as the S − m,44 information shifts. From the ISED image in Figure 4k, one can discern that there are two prominent features in the IR core of the S 6,44 image. The northwest image is the outburst that originates from the environs of the black hole in M87 and HST-1. The other feature relates to the vortex structure that coincides with the location of the black hole. In Figure 4l, L6 ISEE image τ 6,44 , there is an interesting horizontal band structure that is perpendicular to the C-like shockwave on the western side of the image. This is either an artifact of the image or some mechanism that causes the appearance of planer shockwaves. This C-like structure is produced by a superluminal counter-jet that is traveling toward us and is not directly observable due to relativistic beaming [28].
Appl. Sci. 2020, 10, x FOR PEER REVIEW 13 of 27  The L1 ISEE image τ1,44 in Figure 4b has a well-defined outer IR dust ring, and the IR core is pronounced. The shockwaves and surrounding dust structures have good contrast against the black backdrop, where most of the blue channel information has been eliminated except the blue-biased glare portion of the image. Therefore, the image looks a bit blurred. As Ln increases, this blurring effect is reduced as the Sm,44 information shifts. From the ISED image in Figure 4k, one can discern that there are two prominent features in the IR core of the S6,44 image. The northwest image is the outburst that originates from the environs of the black hole in M87 and HST-1. The other feature relates to the vortex structure that coincides with the location of the black hole. In Figure 4l, L6 ISEE image τ6,44, there is an interesting horizontal band structure that is perpendicular to the C-like shockwave on the western side of the image. This is either an artifact of the image or some mechanism that causes the appearance of planer shockwaves. This C-like structure is produced by a superluminal counter-jet that is traveling toward us and is not directly observable due to relativistic beaming [28].
The ISEE images in Figure 3 are constructed from an ISED filter (Sm,44) with sparse information in L1. Then, as the decomposition progresses for deeper Ln, the information in the filter increases. Conversely, there is a lowering in SSIM for the ISED filter. This attributes to the trend of higher SSIM The ISEE images in Figure 4 are constructed from an ISED filter (S − m,44 ) with sparse information in L1. Then, as the decomposition progresses for deeper Ln, the information in the filter increases. Conversely, there is a lowering in SSIM for the ISED filter. This attributes to the trend of higher SSIM and PSNR for the ISEE reconstructions for τ m,44 , displayed in Table 6. Likewise, the MSE decreases for the most part, where L2 values for this filter seems to oscillate towards convergence to the test images as Ln increases.
By this point, it becomes apparent that the ISEE images vary in detail for Ln and there is an evolution of features one can gather from each generated image as reconstruction deepens. The images begin to converge to the original image while highlighting structures and changing as the process continues. When comparing the Ln ISEE and the ISEE images to the original image, a new world of information emerges from a single image. The implications of these processes for all branches of science that deal with image processing and feature extraction are broad. In Figure 5l, the vortex structure and the outburst structure are apparent in the IR dusty core. By this point, it becomes apparent that the ISEE images vary in detail for Ln and there is an evolution of features one can gather from each generated image as reconstruction deepens. The images begin to converge to the original image while highlighting structures and changing as the process continues. When comparing the Ln ISEE and the ISEE images to the original image, a new world of information emerges from a single image. The implications of these processes for all branches of science that deal with image processing and feature extraction are broad. In Figure 5l, the vortex structure and the outburst structure are apparent in the IR dusty core.  From Table 7, it is apparent that the Ln ISED images quickly lose SSIM in this set of test images. The PNSR is low, and the MSE is high. The no-reference quality metrics also degrade. For the ISEE reconstruction, however, the SSIM is quite high and approaching that of the original image, and the PSNR and MSE improve as Ln increases. ISED and ISEE images are shown below Figure 6.  Table 7, it is apparent that the Ln ISED images quickly lose SSIM in this set of test images. The PNSR is low, and the MSE is high. The no-reference quality metrics also degrade. For the ISEE reconstruction, however, the SSIM is quite high and approaching that of the original image, and the PSNR and MSE improve as Ln increases. ISED and ISEE images are shown below Figure 6.
(i) (j) (k) (l) From Table 7, it is apparent that the Ln ISED images quickly lose SSIM in this set of test images. The PNSR is low, and the MSE is high. The no-reference quality metrics also degrade. For the ISEE reconstruction, however, the SSIM is quite high and approaching that of the original image, and the PSNR and MSE improve as Ln increases. ISED and ISEE images are shown below Figure 6.   In the structure in Figure 7l on the dusty outer ring, there appears to be a globular mass that is in part feeding the black hole, but as it turns out when examining the overlay of the SST/HST image in Figure 2, there are dusty wisp-like structures, probably from a past ejection from the black hole, that appear to be pushing this matter outward like a bulldozer pushes dirt. So, as it is not feeding on the globular mass, it is most likely ejecting matter and energy particles.
According to Table 8, the SSIM and PNSR increase for ISEE τ m,46 as Ln increases. The MSE decreases and NIQE shows improved quality over that of the original image and the ISED images S m,46 . The PIQE scores are, for the most part, better than the ISEE reconstruction because of the decomposition's removal of the blue-biased glare. The glare is reintroduced in the reconstruction; however, the PIQE for the Ln ISEE images still displays improvement over that of the original image. In the structure in Figure 7l on the dusty outer ring, there appears to be a globular mass that is in part feeding the black hole, but as it turns out when examining the overlay of the SST/HST image in Figure 2, there are dusty wisp-like structures, probably from a past ejection from the black hole, that appear to be pushing this matter outward like a bulldozer pushes dirt. So, as it is not feeding on the globular mass, it is most likely ejecting matter and energy particles.
According to Table 8, the SSIM and PNSR increase for ISEE τm,46 as Ln increases. The MSE decreases and NIQE shows improved quality over that of the original image and the ISED images Sm,46. The PIQE scores are, for the most part, better than the ISEE reconstruction because of the decomposition's removal of the blue-biased glare. The glare is reintroduced in the reconstruction; however, the PIQE for the Ln ISEE images still displays improvement over that of the original image.  The ISEE images for τ m,47 have the highest SSIM and PSNR out of all the presented images. Figure 7l has the closest SSIM to that of the original image. The color range that it isolates has the properties to isolate the vortex, outburst, globular structures, and boundary regions for the shockwaves. τ 6,47 is applied to the overlay of the HST image in Figure A1a-c.
The decompositional images of S m,47 displayed in Table 9 also show a steep drop off in SSIM and PSNR as Ln increases and a rapid rise in MSE. The NIQE and PIQE increase, while the BRISQUE exhibits little variation. On the other hand, the Ln ISEE reconstructions have high SSIM, improved PNSR, and low IMSE. τ 6,47 has an MSE of 6.91 with an SSIM approaching 1. The NIQE and PIQE are also improved over that of the original image. Sobel operator uses two 3 × 3 kernels that are convolved with the image (A). They are given by the following equations [3].
The gradient magnitude is defined to be for edge detection. This additional analysis was performed to attempt to determine more details on the physical structure near the dusty IR core and the IR ring. Figure 8a displays a tighter crop of the original inset image in order to more clearly see the region of interest. Figure 8b is the gradient magnitude of the image that presents the edge features of the image. In the center of the image, the dark ring is where HST-1 is located. The actual galactic center and black hole lie on the inner ring to the west of the dark center. It is not apparent in this image, but it can be clearly seen in the overlay image in Appendix A, Figure A2a-c. Figure 8c is the x-gradient image and Figure 8d is the y-gradient image.
The gradient magnitude is defined to be for edge detection. This additional analysis was performed to attempt to determine more details on the physical structure near the dusty IR core and the IR ring.   Figure 8b is the gradient magnitude of the image that presents the edge features of the image. In the center of the image, the dark ring is where HST-1 is located. The actual galactic center and black hole lie on the inner ring to the west of the dark center. It is not apparent in this image, but it can be clearly seen in the overlay image in Appendix A, Figure A2a-c. Figure 8c is the x-gradient image and Figure 8d is the y-gradient image. The image in Figure 9a is the tighter crop of the image seen in Figure 3l. τ6,30 has the feature of the western counter-jet channel that is related to the C-like shockwave structure, which is produced by slowing heated dust from the superluminal jet. The image in Figure 3l is combined with the HST image in Figure A1b to produce the overlays presented in Figure A3a,b. The gradient magnitude image in Figure 9b displays a decent outline of the outer IR dust ring and the IR core. Figure 9c,d are τ6,30′s x and y-gradient images.
The L6 ISEE image in Figure 10a has a higher SSIM than the other displayed images. This image is the tighter crop of Figure 7l. Figure 7l and Figure A1b are combined to produce the overlays in The image in Figure 9a is the tighter crop of the image seen in Figure 3l. τ 6,30 has the feature of the western counter-jet channel that is related to the C-like shockwave structure, which is produced by slowing heated dust from the superluminal jet. The image in Figure 3l is combined with the HST image in Figure A1b to produce the overlays presented in Figure A3a,b. The gradient magnitude image in Figure 9b displays a decent outline of the outer IR dust ring and the IR core. Figure 9c,d are τ 6,30 s x and y-gradient images.
Appl. Sci. 2020, 10, x FOR PEER REVIEW  18 of 27   1 0  1  1  2  1  2 0  2 and G 0 0 The gradient magnitude is defined to be for edge detection. This additional analysis was performed to attempt to determine more details on the physical structure near the dusty IR core and the IR ring.   Figure 8b is the gradient magnitude of the image that presents the edge features of the image. In the center of the image, the dark ring is where HST-1 is located. The actual galactic center and black hole lie on the inner ring to the west of the dark center. It is not apparent in this image, but it can be clearly seen in the overlay image in Appendix A, Figure A2a-c. Figure 8c is the x-gradient image and Figure 8d is the y-gradient image. The image in Figure 9a is the tighter crop of the image seen in Figure 3l. τ6,30 has the feature of the western counter-jet channel that is related to the C-like shockwave structure, which is produced by slowing heated dust from the superluminal jet. The image in Figure 3l is combined with the HST image in Figure A1b to produce the overlays presented in Figure A3a,b. The gradient magnitude image in Figure 9b displays a decent outline of the outer IR dust ring and the IR core. Figure 9c,d are τ6,30′s x and y-gradient images.
The L6 ISEE image in Figure 10a has a higher SSIM than the other displayed images. This image is the tighter crop of Figure 7l. Figure 7l and Figure A1b are combined to produce the overlays in   Figure 11 is the tricolor histogram of the original inset test image so that you can see the response for each channel. For all of the following histogram cases, the order is from L1 to L6 ISEE images and shows that for each deeper Ln for the cases of τn,30, τn,44, τn,45, τn,46, and τn,30, the histograms begin to converge to that of the original image. As stated earlier, some images eventually converge, and some remain divergent. To see full convergence for some cases, you would have to go much deeper than an L6 ISEE image. The histograms for L1~L6 ISEE images τn,30 begin to converge when looking at the trend of the histograms in Figure 12a-f. It is observed that the spiked color channels begin to redistribute, and the pixel count scale becomes closer to the same order of the original image. If this state is computed for deeper levels than L6, this image most likely will converge, with less jaggedness in the histogram as Ln increases. The idea of a fully convergent image means that if Ln is sufficiently large, the algorithm will eventually generate the original image. I suppose this process would be analogous to using Fourier analysis to produce a square wave with amplitude 1 from an infinite sum of sinusoidal waves. Divergent cases do not generate the original image for any Ln.  Figure 11 is the tricolor histogram of the original inset test image so that you can see the response for each channel. For all of the following histogram cases, the order is from L1 to L6 ISEE images and shows that for each deeper Ln for the cases of τ n,30 , τ n,44 , τ n,45 , τ n,46 , and τ n,30 , the histograms begin to converge to that of the original image. As stated earlier, some images eventually converge, and some remain divergent. To see full convergence for some cases, you would have to go much deeper than an L6 ISEE image.
1 is the tricolor histogram of the original inset test image so that you can see nel. For all of the following histogram cases, the order is from L1 to L6 ISEE or each deeper Ln for the cases of τn,30, τn,44, τn,45, τn,46, and τn,30, the histogr that of the original image. As stated earlier, some images eventually conver gent. To see full convergence for some cases, you would have to go much age. tograms for L1~L6 ISEE images τn,30 begin to converge when looking at the n Figure 12a-f. It is observed that the spiked color channels begin to redistri cale becomes closer to the same order of the original image. If this state is c s than L6, this image most likely will converge, with less jaggedness in the . The idea of a fully convergent image means that if Ln is sufficiently large, t lly generate the original image. I suppose this process would be analog sis to produce a square wave with amplitude 1 from an infinite sum of sinus ses do not generate the original image for any Ln. The histograms for L1~L6 ISEE images τ n,30 begin to converge when looking at the trend of the histograms in Figure 12a-f. It is observed that the spiked color channels begin to redistribute, and the pixel count scale becomes closer to the same order of the original image. If this state is computed for deeper levels than L6, this image most likely will converge, with less jaggedness in the histogram as Ln increases. The idea of a fully convergent image means that if Ln is sufficiently large, the algorithm will eventually generate the original image. I suppose this process would be analogous to using Fourier analysis to produce a square wave with amplitude 1 from an infinite sum of sinusoidal waves. Divergent cases do not generate the original image for any Ln.
pixel count scale becomes closer to the same order of the original image. If this state is computed for deeper levels than L6, this image most likely will converge, with less jaggedness in the histogram as Ln increases. The idea of a fully convergent image means that if Ln is sufficiently large, the algorithm will eventually generate the original image. I suppose this process would be analogous to using Fourier analysis to produce a square wave with amplitude 1 from an infinite sum of sinusoidal waves. Divergent cases do not generate the original image for any Ln. The τn,44 images in Figure 13a-f possibly may not converge. The shifting of the peaks indicates that it may be converging to the original image, but this ensemble condition is not conclusive from the above histograms. For the τn,45 cases, the histograms in Figure 14a-f indicate that the image is evolving into what we call a convergent case. This is logical due to the increased SSIM, PSNR, and improved MSE scoring as Ln increases. The ISED deep Ln image has a rapid decomposition; this causes more information to be contained in filter Sm,n. The τn,44 images in Figure 13a-f possibly may not converge. The shifting of the peaks indicates that it may be converging to the original image, but this ensemble condition is not conclusive from the above histograms. For the τn,45 cases, the histograms in Figure 14a-f indicate that the image is evolving into what we call a convergent case. This is logical due to the increased SSIM, PSNR, and improved MSE scoring as Ln increases. The ISED deep Ln image has a rapid decomposition; this causes more information to be contained in filter Sm,n. For the τ n,45 cases, the histograms in Figure 14a-f indicate that the image is evolving into what we call a convergent case. This is logical due to the increased SSIM, PSNR, and improved MSE scoring as Ln increases. The ISED deep Ln image has a rapid decomposition; this causes more information to be contained in filter S − m,n . For the τn,45 cases, the histograms in Figure 14a-f indicate that the image is evolving into what we call a convergent case. This is logical due to the increased SSIM, PSNR, and improved MSE scoring as Ln increases. The ISED deep Ln image has a rapid decomposition; this causes more information to be contained in filter Sm,n. The ISEE reconstruction τ n,46 is a movement towards convergence. If τn,46 is given enough Ln to evolve, it will replicate the original image. In Figure 15a-f, note the rapid evolution in the histograms that begin to evolve to the shape of the original test image's histogram. This is an interesting feature of select deep Ln ensemble states for ISEE reconstructions. The ISEE images produced from τ n,47 are also convergent for deep Ln. The histogram in Figure  15f is associated with ISEE image τ6,47. This image has the highest SSIM (.99), greatest PSNR (39.73), and the lowest MSE (6.91). The reconstruction has a narrow band of pixels that highlight the vortex, outburst, globular structures, and emphasizes regions in M87′s shock waves.The Histograms are shown below in figure 16. The ISEE reconstruction τ n,46 is a movement towards convergence. If τ n,46 is given enough Ln to evolve, it will replicate the original image. In Figure 15a-f, note the rapid evolution in the histograms that begin to evolve to the shape of the original test image's histogram. This is an interesting feature of select deep Ln ensemble states for ISEE reconstructions. The ISEE reconstruction τ n,46 is a movement towards convergence. If τn,46 is given enough Ln to evolve, it will replicate the original image. In Figure 15a-f, note the rapid evolution in the histograms that begin to evolve to the shape of the original test image's histogram. This is an interesting feature of select deep Ln ensemble states for ISEE reconstructions. The ISEE images produced from τ n,47 are also convergent for deep Ln. The histogram in Figure  15f is associated with ISEE image τ6,47. This image has the highest SSIM (.99), greatest PSNR (39.73), and the lowest MSE (6.91). The reconstruction has a narrow band of pixels that highlight the vortex, outburst, globular structures, and emphasizes regions in M87′s shock waves.The Histograms are shown below in figure 16. The ISEE images produced from τ n,47 are also convergent for deep Ln. The histogram in Figure 15f is associated with ISEE image τ 6,47 . This image has the highest SSIM (.99), greatest PSNR (39.73), and the lowest MSE (6.91). The reconstruction has a narrow band of pixels that highlight the vortex, outburst, globular structures, and emphasizes regions in M87 s shock waves.The Histograms are shown below in Figure 16.
The ISEE images produced from τ n,47 are also convergent for deep Ln. The histogram in Figure  15f is associated with ISEE image τ6,47. This image has the highest SSIM (.99), greatest PSNR (39.73), and the lowest MSE (6.91). The reconstruction has a narrow band of pixels that highlight the vortex, outburst, globular structures, and emphasizes regions in M87′s shock waves.The Histograms are shown below in figure 16. The CDF is presented in Figure 17a-f. The test images were separated into red, blue, and green color channels. Then the test images were compared for Ln 1~6 where R', G' and B' is the CDF for the original image. CDF is shown to validate how ISEE evolves for Ln and to test for convergence. τm,30, τm,44, τm,45, τm,46, and τm,47 are represented by their corresponding lines in the legend. Figure 17f shows that τm,45, τm,46, and τm,47 are convergent for L6 and that it appears from the trends in Figure 17a-f that τm,30 will converge. It is also possible that τm,44 will converge as well, but at a deeper level than L6. The CDF is presented in Figure 17a-f. The test images were separated into red, blue, and green color channels. Then the test images were compared for Ln 1~6 where R', G' and B' is the CDF for the original image. CDF is shown to validate how ISEE evolves for Ln and to test for convergence. τ m,30 , τ m,44 , τ m,45 , τ m,46 , and τ m,47 are represented by their corresponding lines in the legend. Figure 17f shows that τ m,45 , τ m,46 , and τ m,47 are convergent for L6 and that it appears from the trends in Figure 17a-f that τ m,30 will converge. It is also possible that τ m,44 will converge as well, but at a deeper level than L6. Table 10 is the average values for the image quality analysis conducted for 63 ISED-generated images from the previous study on the same M87 test image [1].
There were 62 ISEE images generated per Ln. Then, the averages were taken from the full-reference quality metrics and the no-reference quality metrics for each Ln. The results are above in Table 11. It is apparent that overall, the proposed method of deep Ln ISEE is an improvement for the full-reference quality metrics SSIM, PSNR, and MSE. The average NIQE is better than the average ISED L1 values for L1 and L2 ISEE reconstructions. L3 ISEE is on par with L1 ISED reconstructions. L4, L5, and L6 ISED NIQE are close to the average L1 ISED filters (the highest quality metrics of Ln overall). All average ISEE are better than that of the original M87 test image. The BRISQUE scores seem to be best for L1 ISEE, and from L2~L6, it appears to plateau around 51. The first level reconstruction on average scores is better than the original image. The PIQE scores are all better than the original image score of 100, which indicates poor quality. It stands to reason that the PIQE scores would be higher on average than the ISED filters due to the reconstruction process in ISEE. The CDF is presented in Figure 17a-f. The test images were separated into red, blue, and green color channels. Then the test images were compared for Ln 1~6 where R', G' and B' is the CDF for the original image. CDF is shown to validate how ISEE evolves for Ln and to test for convergence. τm,30, τm,44, τm,45, τm,46, and τm,47 are represented by their corresponding lines in the legend. Figure 17f shows that τm,45, τm,46, and τm,47 are convergent for L6 and that it appears from the trends in Figure 17a-f that τm,30 will converge. It is also possible that τm,44 will converge as well, but at a deeper level than L6.   There were 62 ISEE images generated per Ln. Then, the averages were taken from the fullreference quality metrics and the no-reference quality metrics for each Ln. The results are above in Table 11. It is apparent that overall, the proposed method of deep Ln ISEE is an improvement for the

Conclusions
Deep Ln ISEE is a promising tool for image processing. It can selectively filter via deep Ln ISED and generate many possible filters that can uncover features in a postprocessed image that was previously obscured due to biased glare, oversaturation, and color saturation. Then, with Ln ISEE, you can reconstruct and generate additional images that enhance image features of the generated images and improve full-reference quality metrics without substantial degradation of no-reference quality metrics. In many cases, the no-reference quality metrics were better than that of the original image for deep Ln ISEE images. The ISEE method opens many possibilities for future applications in image processing across the spectrum of scientific imaging. The ISEE reconstructed image is a type of narrowband filter, and the information is contained within the domain of the original image. ISEE filters, if given enough time to evolve to a very deep Ln, can result in the original image (converge) or diverge. ISED and ISEE both perform well, and for certain applications, ISED would be more suited then ISEE for contrast enhancement. In other instances, ISEE may be more desirable to extract finer detailed structures. It depends on the image you are trying to denoise or enhance and the types of features that you are trying to extract.
The specific application to observational astronomical imaging has revealed some profound insights by applying the novel ISEE method. Hidden within the postprocessed images from the HST and SST, we discovered an outburst that was located within the dusty IR core of M87 displayed in the overlay of Figure 2, which shows it directly relates to a dusty wisp-like structure ("purple right") that seems to be emanating from HST-1. Hence, it is a localized distortion, most likely caused by the wisps. This structure appears to be slingshot around HST-1 and probably originates from the jet traveling through this structure. It stands to reason that the mass of HST-1 is strong enough to produce this slingshot effect by beam-splitting some of the superluminal jet particles to cause it to change its trajectory. Perhaps this and the jets' interaction with matter in this region in part explains the outburst observed in HST-1. A second wisp-like structure ("purple left") appears to follow a parallel path to that of HST-1. I suppose if HST-1 is massive enough, it would act like a beam collimator for the jets. This structure seems to emanate from the galactic core center of mass, which is offset from the black hole [29]. The path of the right jet is "right yellow". The "orange circle" is the globular structure that is interacting with the jet. The "green U" is another structure that is being ejected from the galactic core from some type of explosive outburst. It appears to be pushing and heating the dust that forms the second globular structure, the "orange oval". The channel for the counter-jet is displayed with the "left yellow line". The suspected HST-2 is shown by the blue circle. HST-2 appears linear to the right counter-jet channel. The fact that the two jets are not aligned may be due to the angle in which this image is viewed. The counter-jet is firing in our direction, and the other jet is shooting away. This bending effect seems akin to a straw in a drinking glass half-filled with water. If you look at the glass, the straw appears bent. The viewing angle of the jets and the local effects of gravity in galaxy M87 may be warping space-time just enough to cause such an effect, or the gravitational effects of HST-2 are changing the angle of the counter-jet's apparent trajectory by culminating the jet. The "blue lines" appear to be planar waves that seem to be parallel to the C-like shockwaves and perpendicular to the counter-jet channel. However, this effect could be a sampling error in the image. From the image, the apparent rotation of the black hole and the galaxies core is counterclockwise with respect to our viewing angle. Around the local vicinity core in Figure A3b, you can observe the detail of a swirling whirlpool structure, most likely caused by the rotation of the matter around the galactic core. Acknowledgments: Timothy Ryan Taylor, one of the authors, would like to thank his Mother and Father, William F. Taylor and JoAnn Taylor. I would also like to thank my home of Taiwan for giving me a haven and opportunities to grow as a researcher. I would also like to thank Gracie, for giving me the support I need so that I can be a better version of myself. I hope the world can become a better version of itself with this tragic plague we face. I would like to thank all the health and service workers risking their lives daily. I hope and pray all our loved ones are safe and prosperous. Thank you and may we together engineer a better future.

Conflicts of Interest:
The authors declare no conflict of interest.