Novel Image State Ensemble Decomposition Method for M87 Imaging

Abstract

This paper proposes a new method of image decomposition with a filtering capability. The image state ensemble decomposition (ISED) method has generative capabilities that work by removing a discrete ensemble of quanta from an image to provide a range of filters and images for a single red, green, and blue (RGB) input image. This method provides an image enhancement because ISED is a spatial domain filter that transforms or eliminates image regions that may have detrimental effects, such as noise, glare, and image artifacts, and it also improves the aesthetics of the image. ISED was used to generate 126 images from two tagged image file (TIF) images of M87 taken by the Spitzer Space Telescope. Analysis of the images used various full and no-reference quality metrics as well as histograms and color clouds. In most instances, the no-reference quality metrics of the generated images were shown to be superior to those of the two original images. Select ISED images yielded previously unknown galactic structures, reduced glare, and enhanced contrast, with good overall performance.

1. Introduction

There is need to provide a highly tunable fundamental image processing technique that can remove unwanted color, biased glare, and noise; reduce image artifacts; and improve the contrast and aesthetics in a post-processed RGB image. Generally, one has two main paths to follow when applying image decomposition or enhancement. One typically chooses to be in either the spatial domain or the frequency domain. The image state ensemble decomposition method (ISED) uses sets of spatial domain filters that decompose an image by selectively removing discrete state ensembles from the original image in a red, green, and blue (RGB) color space. This removed portion of the image contains a range of color information that encompasses regions of the image with noise that may biased to a certain domain of the image. These regions may also contain unwanted artifacts and glare. ISED generates images to help discover these biased regions and reduces the unwanted characteristics from the image. ISED generates possible image outcomes from the information contained within a post-processed RGB image. Additionally, ISED is a novel approach that is applicable in many fields; astronomical imaging was chosen out of personal interest.

In the spatial domain, direct manipulation of the pixels is normally modified by averaging, median filtering, contrast stretching, Gaussian blurring, many types of histogram equalizations (HE), and the Retinex algorithm, just to name a few [1,2,3,4]. These methods suffer from major drawbacks where the pixel information is either blurred, redistributed, or the intensity is scaled in an unrealistic way [1,3]. ISED is an image decomposition method that has the ability to remove unwanted color and biased glare, reduce noise, improve contrast, reduce image artifacts, and improve the aesthetics in a post-processed RGB image. ISED retains all of the photonic information from the original information if one simply recombines the ISED image and the ISED filter; provided that the image is saved in a lossless format such as tagged image file (TIF) or PNG, one would have the original image with a structural similarity index (SSIM) equal to 1. Therefore, the process is fully reversible, and the information can be conserved. This is unlike the aforementioned conventional image processing methods, which in essence have a loss or distortion of pixel information. However, the standard image processing techniques are very useful in a case-by-case basis, such as using the median filter to clean salt and pepper noise, using an averaging filter to reduce random statistical noise, and using some form of histogram equalizations (HE) or Retinex to redistribute pixel information to enhance contrast [1,2]. With ISED, you can be more selective about the informative structures that you wish to study and can extract features in a way that does not corrupt the information in the image. It creates classes of possible filters and possible images, while preserving information, which is dissimilar to existing methods. ISED provides details of structures previously not seen in the images processed by NASA (The National Aeronautics and Space Administration), JPL-Caltech (Jet Propulsion Laboratory, California Institute of Technology), and IPAC (Infrared Processing and Analysis Center).

ISED is able to maintain the information of the selected image and move portions of the image to the filter. Thus, there is no loss of pixel information if both the image and filter are known. Therefore, there is no loss of entropy when the ISED filter is recombined with the filtered ISED image: you obtain the original image. With ISED, one can simply generate many possible images from the original image, and as a result of this study, ISED has the ability to show features of galactic structures that were previously hidden within the initial image. Therefore, the need and the purpose of the specific example studied in this paper is to reveal new visual information about M87′s galactic structure. The ISED method is not limited to astronomical applications but can be applied to any color image or video. This novel method is akin to Schrödinger’s cat [5,6] but with a deterministic twist. There is a cat in a closed box, but with ISED, I can simply change the color of the cat, decide if the cat has no hair, or make part of the cat invisible. Maybe at a certain instance in time the cat is purring, sitting, or sleeping. The cat can move around freely and be relatively comfortable inside its box. We know the cat is inside, but we do not know its current state until we decide to peek inside the box. An image is similar to Schrödinger’s cat in that the light collected from an image is a record of various flavors of photons that are counted, stored, and allocated to a specific location in order to form a record of the photons. This record is the image, which I reiterate is the representation of an occurrence of photonic information observed over some time interval. Since the image is a record of photonic information, the use of statistics should be employed to remove a finite collection of photonic information from the image or in other words remove discrete quanta. Additionally, all the ISED images analyzed in this paper along with Supplementary ISED-generated images and ISED filter images can be accessed online at [7]. Currently, the Supplemental images are related to the topics of art, astronomy, general photography, histology, and mineralogy [7].

General relativity was introduced by Einstein in 1917 [8]. One result of the Einstein field equations (EFEs) was the theoretical concept of black holes. Schwarzschild noticed this bizarre effect in the EFEs [8]. From this, Schwarzschild calculated that if an immense nonrotating mass is present, an object can exist with gravitational forces so strong that even light, the fastest known object in the universe, could not escape the object’s grasp [9]. Prior to the validation of the existence of black holes, Wheeler coined the term “black hole” in a lecture [10]. This theoretical construct was later confirmed by Webster et al. [11] and Bolton [12] to be a physical reality when they observed Cygnus X-1. Black holes are fascinating, mysterious, and have captivated many scientists. In a recent study, the Event Horizon Telescope collaboration produced the first image of a black hole [13]. It was presented alongside photographs taken by the Spitzer Space Telescope (SST) as an inset [14]. These SST images were used as the input data for this study.

In order to improve the visualization of post-processed SST images and thus reveal more informative visual details and improve our understanding of the nature and structure of the universe, we propose ISED. ISED uses a series of cross-channel relations made between the red (R), green (G), and blue (B) color channels. A pixel in a modern monitor uses additive mixing color theory, wherein colors can be added and mixed from three primary colors: red, green, and blue (RGB). Additive mixing theory was introduced by Maxwell in “On the Theory of Compound Colours, and the Relations of the Colours of the Spectrum” [15]. The standard monitor operates with the pixels being a certain ratio/mix of varying intensities within the RGB color channels. This system is scaled from 0 to 255, in which 0 is black and 255 is the highest intensity in the color channel. Therefore, a standard pixel can represent 256³ different colors. These pixel intensities combine in a matrix to form an image. The cross-channel relationship in ISED is designed to be a possible set of states. The ensemble of states can either exist (as “on”) or not (as “off”) for the most basic case. These ensemble states can overlap regions of an image that contain unwanted pixel information, such as glare. If a region contains glare biased to a particular color channel, one can reduce the effects by selecting the correct ISED state. This switch choice depends on the desired image outcome and the input image. One may output various image states and simply compare the characteristics of the ISED-generated image for the desired outcome. For example, the ISED filter can remove glare that is biased to a certain bandwidth of color while maintaining the structure of the image of interest. Once the correct ISED state is determined for this particular image solution, you can apply the same state selection to a similar image to yield the desired results.

The paper is organized as follows. Section 2 provides an overview of the materials and methods used to generate ISED images. Section 3 provides the mathematical framework to construct ISED images and filters. Section 4 presents the experimental results and discussion of this study. Finally, the conclusion in Section 5 gives the ramifications of ISED.

2. Materials and Methods

In the proposed ISED experiments, MATLAB R2018a and R2019a were used to implement our proposed algorithm on the post-processed SST image sets. The images were sourced from NASA/JPL-Caltech/IPAC. The implemented hardware configuration was comprised of an Intel i7-8750H processor, 32 GB of RAM, and an NVidia GeForce GTX 1060 graphics processing unit. The experimental input data were sampled images of M87. The first file, “ssc2019-05c.tif”, is 11.2 MB in size with a pixel resolution of 3580 × 3580. The second image of M87, file name “ssc2019-05b.tif”, is 11.1 MB in size with a pixel resolution of 3580 × 3580. The inset of this second image contains an enlarged region of M87′s core, in which the region of M87′s black hole and its jets are shown. This inset was cropped by using GIMP 2.10.14 and saved as a lossless tagged image file (TIF) format for processing. The remainder of the image in file “ssc2019-05b.tif” was discarded, because “ssc2019-05c.tif” is the same image without the inset. The size of the cropped image was 1.54 MB with a pixel resolution of 730 × 733. The two images were taken by the SST using an Infrared Array Camera (IRAC). The nominal values of the red, green, and blue channels were mapped to the IRAC infrared radiation wavelengths of 8.0, 4.5, and 3.5 μm, respectively [14]. Both images were generated with chromatic ordering taken from the infrared portion of spectrum. Therefore, the images were an approximation of what we see if the sampled photons were in the visible range of the spectrum. The physical filter selection can cause part of the continuum sources (e.g., a star) to not exhibit true color, such as the case of a narrow-band filter not properly sampling the stellar blackbody [16]. The ISED filter can operate in a similar manner by reducing a narrow band of the image matrix information.

The two images were separated into their constitute components. The intensities of the individual trichromatic components of red, green, and blue shall be referred to as R’, G, and B’ respectively. The R’, G, and B intensity values range from 0 to 255. The “zero” value represents black, and as the value increases on the scale, the pixel becomes lighter [1]. To generalize the problem, assume that the three color channels of the image do not have the same corresponding color channel pixel values for a discrete range between respective R’, G, and B’ image matrices. Taking this into account, difference relationships were formulated between the color channels to modify the post-processed images. This process is implemented in the spatial domain; the ISED image is matrix-wise transformed or generated at the pixel level.

For the sake of analogy, let us say that the image is analogous to the Gibbs microcanonical ensemble wherein a statistical ensemble is used to know all the possible states of a system [17]. The photons are converted to electrons via the photoelectric effect [18], counted discretely, and then categorized into three discretized groups, or color channels, to represent a single RGB pixel. Multiple pixels combine to form an image matrix. This matrix contains the information of many possible states, as ISED will later demonstrate. ISED can transfer photonic information in varying amounts from the ISED image to the ISED filter without losing any information. The sum of the image and filter equals the input image. Use of this method is possible because quantum mechanical objects (e.g., photons) can behave as waves or particles [19,20], and waves can be observed as single quanta [21]. The proposed method decomposes images without loss, provided that the information of both the image and filter is stored.

Theorem 1.

ϕ_n ≡ The scalar probability of an ensemble of states to either succumb to wave collapse and be measured or to not be detected. The regions of the wave collapse are bounded the probability of [0,1].

The variable ϕ_n is similar to a fuzzy set unit interval [0,1] to help deal with the statistical nature of light. Theorem 1 is also analogous to the infinite square well model in quantum mechanics, where the particle is bound between 0 and L and has a probability of having a certain positive energy level somewhere inside the well and at a certain moment in time. The ϕ indices, m and n, are index notations indicating that this function is applied pixel-wise in the ψ ensemble matrices.

Mathematically for this method, this set is convenient to use to determine whether a state ensemble is activated and to what degree. This image processing method takes advantage of the particle-like behavior of light and uses elements of statistical ensembles to define subsets or portions of the image. These subsets are a discrete ranges of mixed color values. Furthermore, the ISED method does not need to be a binary decision of “on” and “off” states; however, in the most elementary of applications, binary switching yields useful results. Additionally, in the most basic case, the state ensemble exists or it does not. Then, the binary conditions of, “0” or “1,” yield up to 2¹⁸ different possible states of decomposition and can hence generate 2¹⁸ possible images and 2¹⁸ possible filters. We now turn to formulating the state ensembles and remove some “fuzziness” from an image that contains a super massive black hole.

3. Mathematics of ψ Image State Ensembles

The following novel mathematical formulas govern the behavior of the proposed ISED method. The pixel-wise function ϕ_n where, n = [1,2,3,…,6].

3.1. ψ Color Channel Relations

The following equations are the color channel relations for the development of ISED.

$$\begin{array}{c}{\psi }_1={\varphi }_1({\alpha }_{1}B\prime -{\beta }_{1}G\prime )\\ {\psi }_2={\varphi }_2({\alpha }_{2}R\prime -{\beta }_{2}G\prime )\\ {\psi }_3={\varphi }_3({\alpha }_{3}R\prime -{\beta }_{3}B\prime )\\ {\psi }_4={\varphi }_4({\alpha }_{4}G\prime -{\beta }_{4}B\prime )\\ {\psi }_5={\varphi }_5({\alpha }_{5}G\prime -{\beta }_{5}R\prime )\\ {\psi }_6={\varphi }_6({\alpha }_{6}B\prime -{\beta }_{6}R\prime )\end{array}$$

(1)

where matrix ψ is constrained by the values, ψ_n ≥ 0, and n = [1,2,3,…,6]. In order to simplify the experimental amplitudes of the variables, α and β are set to 1. Variables α and β can be used to change the intensity scaling of the ensemble in the image matrix. The aforementioned ψ-state ensembles are intended for an RGB image. However, the concept can be generalized for a higher dimensionality state comparison that would allow for the further mixing of colors and more state ensembles; hence, additional ISED images can be generated.

3.2. Generalized ψ Image State Ensemble Constitutive Relationships for N-wavelength λ

The following equations are the generalized ψ image state constitutive relationships, and they can be used to formulate n-color channels.

$$\begin{array}{l}{\psi }_n={\varphi }_n({\alpha }_n{\lambda }_o-{\beta }_n{\lambda }_p)\\ {\psi }_n={\varphi }_n({\beta }_n{\lambda }_p-{\alpha }_n{\lambda }_o)\end{array}$$

(2)

with the constraint that matrix difference values, ψ_n ≥ 0, o ≠ p.

3.3. ψ Ensemble Matrices for ϕ_n = [0,1], α_n = β_n = 1

The ensemble matix relationships are designed to produce a binary decision to construct the matricies that build the ISED filters.

$$\begin{array}{c}{\psi }_1=\left[0,1\right](B\prime -G\prime )\\ {\psi }_2=\left[0,1\right](R\prime -G\prime )\\ {\psi }_3=\left[0,1\right](R\prime -B\prime )\\ {\psi }_4=\left[0,1\right](G\prime -B\prime )\\ {\psi }_5=\left[0,1\right](G\prime -R\prime )\\ {\psi }_6=\left[0,1\right](B\prime -R\prime )\end{array}$$

(3)

with the constraint that matrix difference values, ψ_n ≥ 0.

For Equation (3), ϕ_n = [0,1] indicates the on and off states for the ψ matrix state ensemble. These are the ψ₁, ψ₂, ψ₃, ψ₄, ψ₅, ψ₆ conditions that were applied in the following systems of equations to obtain the results in this paper.

3.4. Image State Ensemble Decomposition

Below are the ISED equations with a scaling factor included to modify the intensity of the the origional images R’, G’ and B’ color channels.

$$\begin{array}{l}{R}_n=A_{1}R\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\\ G_n=A_{2}G\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\\ B_n=A_{3}B\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\end{array}$$

(4)

where A_n = 1, A_n is an intensity scaling factor and can be used to change the intensity of the original color channel. In this experiment, A_n is set to 1.

3.5. Simplified Image States Ensemble Decomposition

Beneath are the novel simplified ISED equations which can be used to genrate ISED image and ISED filters. The filter component is the difference portion of the equation.

$$\begin{array}{l}{R}_n=R\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\\ G_n=G\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\\ B_n=B\prime -\frac{1}{\sqrt{2}}({\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4+{\psi }_5+{\psi }_6)\end{array}$$

(5)

The equations in (5) can be used to generate 2¹⁸ possible filters and 2¹⁸ possible images based on the above formula. The matrix variables R_n, G_n, and B_n are the resultant image color channels after the desired image states have been assigned ϕ_n values. The values are 0 or 1 in our bivariate case. Afterwards, the resultant image color channels are recombined to form the generated image (RGB)_n.

3.6. Generalized Image States Ensemble for N-Dimensions

This generalized equation can be used to set up an ISED relation between 2 to n color channels.

$${\lambda }_n=A_n\lambda \prime -\frac{1}{\sqrt{2}}\left({\psi }_1+\dots +{\psi }_n\right)$$

(6)

3.7. Balanced Image State Ensemble Decomposition for RGB

The following equation is the summation notation for the balanced ISED states studied in this paper.

$${(RGB)}_n=(RGB)\prime -\frac{1}{\sqrt{2}}{\displaystyle \sum _{n=1}^6\left({\psi }_n\right)}$$

(7)

This paper demonstrates and analyzes the “balanced state” condition, which is defined by the image state ensemble relationship in Equation (7) wherein the R’, G’ and B’ ψ image states are set to “on” (1) or “off” (0). For clarification, if ψ₆ is set to the “on” state for the red channel, that means it will also be set to “on” for the green and blue channels. This is called a “balanced state”. In all, there are 64 balanced states possible for this configuration; however, for the “zero” image state, all ψ are set to “off” (0). Therefore, the output image is equal to the input image, and the filter is a zero image or black.

4. Results and Discussion

Image quality assessments (IQAs) were used to evaluate the experimental results; both full-reference and no-reference quality metrics were used. The no-reference metrics are able to analyze both the test image and those of the ISED images. The full-reference metrics compare the original image to the modified image for quality assessment. Full-reference quality metrics are used with the original NASA reference image, such that the modified image is compared with the original image. The full-reference quality metrics that were used are the peak signal-to-noise ratio (PSNR), image mean squared error (IMSE), and structural similarity (SSIM) index [22]. The SSIM generates a maximum score of one based on the images comparative contrast, local structure, and luminescence [22]. The IMSE is the squared error between the original image and a compressed image, whereas the PSNR is the peak error in the image. The IMSE and PSNR are used to show the amount of compression for an image, but for the case of decomposition, it can be used to gauge the relative change between the original image and the ISED image. A table of values was output for the SSIM, IMSE, and PNSR, and the results are detailed in Appendix B. For the experiment, the no-reference quality metrics statistically compare the features of the original image to those of the modified images. Furthermore, the image results shown in this paper and on the Supplemental website clearly indicate the potential worth of this filter method. In Fact, the no-reference IQAs in some instances have improved the perceptual quality over the original NASA image and show more details. The no-reference quality metrics implemented were the perception-based image quality evaluator (PIQE) [23,24], natural image quality evaluator (NIQE) [25], and blind/reference-less image spatial quality evaluator (BRISQUE) [26,27]. PIQE analyzes the local variance to see if the image has a block-wise distortion to calculate the quality of an image; lower scores are better [23]. NIQE is trained on a database of pristine images that uses nature scene statistics, in concert with Gaussian distributions, so that it can measure arbitrary distortions in the test image [25]. BRISQUE is also trained from a database of known distortions and pristine images; this method is limited to evaluating only the types of distortions that have been trained for the database [26,27]. A table of values was output for PIQE, NIQE, and BRISQUE, as shown in Appendix C. The IQAs are discussed in the results and discussion sections. Furthermore, histograms and color clouds of the images were used to show the pixel distribution over the RGB color space.

The following pseudocode flow chart in Figure 1 illustrates the design of ISED images and filters, and Algorithm 1 is for ISED generation.

Algorithm 1 ISED Generation

1: Load the input image.

2: Separate the image into its color channels R’, G’, and B’ in RGB color space.

3: Use Equation (3) is used to build the filters for ψ R, ψ G, and ψ B. ${\mathsf{\psi }}_1=\left[0,1\right]{(\mathrm{B}\prime -\mathrm{G}\prime ),\mathsf{\psi }}_2=\left[0,1\right]{(\mathrm{R}\prime -\mathrm{G}\prime ),\mathsf{\psi }}_3=\left[0,1\right]{(\mathrm{R}\prime -\mathrm{B}\prime ),\mathsf{\psi }}_4=\left[0,1\right]{(\mathrm{G}\prime -\mathrm{B}\prime ),\mathsf{\psi }}_5=\left[0,1\right]{(\mathrm{G}\prime -\mathrm{R}\prime ),\mathrm{and}\mathsf{\psi }}_6=\left[0,1\right](\mathrm{B}\prime -\mathrm{R}\prime )$.

4: Set the [0,1] state conditions from Appendix A, for ψ1, ψ2, ψ3, ψ4, ψ5, ψ6. This defines the state ensemble conditions. 5: The ISED color channel filter is $\frac{1}{\sqrt{2}}{(\mathsf{\psi }}_1{+\mathsf{\psi }}_2{+\mathsf{\psi }}_3{+\mathsf{\psi }}_4{+\mathsf{\psi }}_5{+\mathsf{\psi }}_6)$ for ψ R, ψ G, and ψ B. The combination of ψ R, ψ G, and ψ B would produce the ISED filter image. An example of an ISED filter image is shown in Figure 2c.

6: Applying Equation (5)$\begin{array}{l}{\mathrm{R}}_{\mathrm{n}}=\mathrm{R}\prime -\frac{1}{\sqrt{2}}{(\mathsf{\psi }}_1{+\mathsf{\psi }}_2{+\mathsf{\psi }}_3{+\mathsf{\psi }}_4{+\mathsf{\psi }}_5{+\mathsf{\psi }}_6)\\ {\mathrm{G}}_{\mathrm{n}}=\mathrm{G}\prime -\frac{1}{\sqrt{2}}{(\mathsf{\psi }}_1{+\mathsf{\psi }}_2{+\mathsf{\psi }}_3{+\mathsf{\psi }}_4{+\mathsf{\psi }}_5{+\mathsf{\psi }}_6)\\ {\mathrm{B}}_{\mathrm{n}}=\mathrm{B}\prime -\frac{1}{\sqrt{2}}{(\mathsf{\psi }}_1{+\mathsf{\psi }}_2{+\mathsf{\psi }}_3{+\mathsf{\psi }}_4{+\mathsf{\psi }}_5{+\mathsf{\psi }}_6)\end{array}$ forms the modified Rn, Gn, and Bn color channels. These channels are used to build the ISED Image.

7: Recombine the color channels to form a newly generated ISED image in (RGB)n. 8: Output the ISED image 9: Optional Output the ISED filter image. Original image – ISED image = ISED filter image or equivalently combine recombine the color channels from ψ R, ψ G, and ψ B to build ψ(RGB).

Figure 1. Algorithm flow chart for image state ensemble decomposition (ISED).

Figure 2. Image results for a full-sized tagged image file (TIF)-format image of M87 (a) original high resolution, credited to NASA/JPL-Caltech/IPAC, (b) ISED-generated image for the sixth ensemble state condition, and (c) image of the sixth ISED filter.

According to Figure 2b, the ISED image produces well-defined regions in M87, and previously obscured structures become more pronounced. Some of the more distant galaxies have been reduced or removed; however, the larger structure is observed in greater detail with the additional advantage of reduced glare, making the detailed structure of the core of the galaxy more prominent. This unique process makes it possible to see somewhat into the interior structure of the galactic structures, similar to how an X-ray can see the bones in the human body. There appears to be a great deal of information that can be learned for the chromatic ordered images by using ISED: much more detail than the original image (Figure 2a), the ISED image (Figure 2b) has SSIM (0.79), IMSE (128.8), PSNR (26 dB), NIQE (5.32), PIQE (58.1), and BRISQUE (44.1) values. See Appendix B and Appendix C.

The states in Table 1 correspond to Equation (5), where the simplified state ensemble of the ISED image generated (in Figure 2b) is produced from Equation (7).

$$\begin{array}{l}{R}_n=R\prime =\frac{1}{\sqrt{2}}({\psi }_4+{\psi }_5)\\ G_n=G\prime =\frac{1}{\sqrt{2}}({\psi }_4+{\psi }_5)\\ B_n=B\prime =\frac{1}{\sqrt{2}}({\psi }_4+{\psi }_5)\end{array}$$

(8)

Table 1. Balanced state ensemble of the sixth ISED generated image (seen in Figure 2b). RGB: red, green, and blue.

Ensemble State	Balanced			ϕn
Image		ψ1	ψ2	ψ3	ψ4	ψ5	ψ6
Original image	(RGB)n	0	0	0	0	0	0
The 6th ISED	(RGB)n	0	0	0	1	1	0

Finally, the ISED image is made after the three color channels R_n, G_n, and B_n are combined to form the image (RGB)_n.

In Figure 3b, the branch on the left side of the color cloud has been removed, and Figure 3c presents the information remaining from the original image: the ISED filter.

Figure 3. Pixel distributions over an RGB color cube as color clouds of the (a) original image (in Figure 2a), (b) ISED image of state 6 (in Figure 2b), and (c) corresponds to the ISED filter image (in Figure 2c).

The third histogram is state 6 ψ and that of the ISED filter. A standard horizontal axis of a histogram ranges from 0 to 255; however, in this instance, nothing substantial occurs in the image for the intensity value above 100, and the scales for the histograms have been adjusted accordingly. The image is rather dark, so most of the information and interesting features are under 80 for the RGB value displayed in Figure 4. With this filter, some information is shifted to increase the dark pixel count, yet it maintains a majority of the SSIM (0.79). Appendix B indicates that the other IQAs are on par with or superior to the original in regards to the no-reference quality metrics.

Figure 4. Individual histograms of red (R), green (G), and blue (B) components of the (a) original image, (b) ISED image of state 6, and (c) the ISED filter image of state 6.

In Figure 5b, the structure of the jets that were expelled from the black hole in the center of M87 improved. The galactic core with its supermassive black hole is also markedly pronounced, because the biased blue glares in the background of Figure 5c were partially removed by the ISED filter. The remaining structure of the region of interest was maintained, with improved contrast enhancement over the low deterioration SSIM. The bright knots of HST-1 and a second knot are much more apparent. Additionally, the flare ejected from the supermassive black hole is more prominent. Similarly, the jets are visible in greater detail, and the core is also more pronounced and its boundary is well defined when compared to the original image. Furthermore, the ISED-generated image is also less blurred than that of the original. In Figure 5d, SSIM is higher than in the image in Figure 5c. This is useful in certain instances to be discussed in more detail in a follow-up paper. Most of the ISED generated images in this study provide the clearest view and most of the detail of M87s core, knots, and its jets currently in publication. Note that the results seen in Figure 5 do not reflect necessarily the best perceptual quality results, but rather only a sample. Additionally, the majority of the generated images have better no reference IQAs than those of the original image. So, this method is useful for analysis, and the images look beautiful. The full set of released generated images can be seen online at [7].

Figure 5. Cropped images of (a) original image, credited to NASA/IPL-Caltech/IPAC, (b) 22^nd ISED-generated image, (c) 23^rd ISED-generated image, and (d) the 24^th ISED-generated image.

Table 2 presents the switch states for the ISED images in Figure 5.

Table 2. Balanced state ensembles of ISED generated images in Figure 5a–c.

Ensemble State	Balanced			ϕn
Image	Switch	ψ1	ψ2	ψ3	ψ4	ψ5	ψ6
Original image	(RGB)n	0	0	0	0	0	0
The 22nd ISED	(RGB)n	0	1	0	1	1	0
The 23rd ISED	(RGB)n	0	1	0	1	1	1
The 24th ISED	(RGB)n	0	1	1	0	0	0

The experimental results are presented in Figure 6b–d. The red-portioned point clouds are shifted down and observed as a reduction of jet saturation in the ISED-generated images. The intensities of the images in Figure 5b,c have remarkably diminished, but the informative structural features of the core, jets, and knots are maintained.

Figure 6. Color clouds of pixel color distribution over the RGB color cube of the (a) original image of M87 in Figure 5a, (b) 22^nd ISED-generated image in Figure 5b, (c) 23^rd ISED-generated image in Figure 5c, and (d) 24^th ISED-generated image in Figure 5d.

In Figure 7b, the recognizable values exhibit a left-shifting of the red component to approach “zero” value (i.e., black), reducing the “reddishness” (R) of the image. Both the green and blue pixel counts are shifted as well, reducing the oversaturation of the bright jets in the original image and making them appear clearer. Furthermore, the structural detail is visually enhanced by the contrast improvement. The histogram in Figure 7c indicates that the blue and green values are less shifted than in Figure 7b, allowing for greater contrast and structural detail. The histogram in Figure 7d reveals that all the red, blue, and green pixel counts are shifted closer to the “zero” value, which is known as the “black” state. This produces a darker image, one that has the highest SSIM among the three ISED-generated images, in Figure 5b–d.

Figure 7. Histograms of images shown in Figure 5 as the (a) original image of M87 in Figure 5a, (b) 22^nd ISED-generated image in Figure 5b, (c) 23^rd ISED-generated image in Figure 5c, and (d) 24^th ISED-generated image in Figure 5d.

This zoomed-in cropped image is taken to show greater details of the galactic core in M87. Figure 8b shows that the structural details of the galactic core are improved and glare blurring is reduced. The core and the knots are much more apparent. In Figure 8c, the details of the galactic core have the most improvement, and the contrast is enhanced. One can easily see that the core and the knots have the greatest detail. This image also exhibits further reduction in the strong blue biased glaring that is seen in the original image. For Figure 8d, the details are slightly deteriorated over a somewhat darker background. It has a reduction in some of the reds, which have shifted to the darker values shown in Figure 8d.

Figure 8. Magnification of M87′s galactic core structure: (a) original image of M87 in Figure 5a, (b) 22^nd ISED-generated image in Figure 5b, (c) 23^rd ISED-generated image in Figure 5c, and (d) 24^th ISED-generated image in Figure 5d.

The results indicate that ISED makes it possible to remove select quanta from a stellar image, and other informative features such as the core, knots, and jets became more pronounced. Figure 9 is a collage of 63 ISED-generated images and most images reveals details that were previously obscured. This method has tremendous potential to peer into the heart of a galaxy. Structures of the galaxy, seen in Figure 2, lose some of their “fuzziness” after ISED and have clearly defined boundaries. These boundary regions are in relation to a finite wavelength of the IR radiation or, in this case, color filtered out of the ISED image. This feature will also make ISED useful for studying the morphology of galaxies and gaseous diffuse nebula. Further investigation into this is warranted because of the interest in the field on the topic [28]. This idea will be expanded upon in greater detail in a follow-up paper.

Figure 9. Collage of M87′s galactic core for 63 ISED-generated images. The original image is subfigure 1, top left and 63 ISED-generated images subfigures 2–64 of M87. The ISED ensemble states of the images are displayed in Appendix A. Full-sized images are available online at [7].

Theoretically, no image information is lost when using this method if the information of the selected ISED filter and ISED-generated image is stored. The entropy of the original image is equal to the entropy of the ISED image plus the ISED filter. It follows the superposition principal in that the sum of the filter and the image generated form the original image. As Richard Feynman stated, “No one has ever been able to define the difference between interference and diffraction satisfactorily. It is just a question of usage” [29]. The same can be said for the ensemble interpretation of quantum mechanics. The table below shows the average values for the image quality analysis of the 63 ISED generated images. The averages in Table 3 are only given as a holistic ballpark reference, as all the filters have varied performance results.

Table 3. Average values for image quality assessments (IQAs). SSIM: structural similarity index, PSNR: peak signal-to-noise ratio, IMSE: image mean squared error, NIQE: natural image quality evaluator, BRISQUE: natural image quality evaluator, PIQE: perception-based image quality evaluator.

Compared Methods	Image Quality Assessment Metrics
	SSIM	PSNR	IMSE	NIQE	BRISQUE	PIQE
M87 Full Original	NA	NA	NA	5.54	43.56	57.37
Average	0.69	26.0	298.2	5.66	46.46	63.36
M87 cropped	NA	NA	NA	8.22	43.38	100.0
Average	0.70	22.8	473.1	7.22	42.90	46.23

SSIM indicates the similarity of luminance, contrast, and structure between the original and processed images; an SSIM value of 1 is the highest score [22]. According to the quality assessment of 63 ISED-generated images, the average SSIM of the full-sized image was 0.69 and that of the cropped image was 0.70. As shown in Figure 10, some ISED states are similar to those of the original image. However, this does not indicate the desired effect for an image. The SSIM value of the 23^rd state in Figure 5c was 0.46, but it revealed more detail than the other two images in Figure 5b,d. The removal of the blue-biased glare from the image contributed to structural, luminal, and contrastive enhancement. In Figure 7b, the ISED filter contains more informative features than it does in Figure 7a,c. In fact, the SSIM value of the 23^rd state ISED filter was 0.61, which was higher than that of the 23^rd ISED-generated image. A low SSIM score does not mean that ISED generation was not performed favorably. In the 23^rd state ISED, the information sent to the filter contained substantial unwanted blue-biased glare. The SSIM scores for the 22^nd and 24^th state ISED filters were 0.24 and 0.23, respectively.

Figure 10. SSIM values of 63 ISED-generated images. The cropped images of M87 can be seen in Figure 9.

According to Figure 11 the PSNR appears to be in line with the SSIM. A higher PSNR corresponds in this case to a higher SSIM. PSNR was measured to determine how the states compare in relation to the original image. PSNR is often used to analyze image compression, and the decomposition in ISED can be comparable to a compression-like loss. The fourth state ISED had the highest PSNR for the full-sized image at 25.89 dB; the cropped image had a PSNR of 26.97 dB. Parenthetically, this corresponds to the worst PIQE value of 100 in the fourth ISED state. As a result of the nature of the algorithm’s switching scheme, the PSNR trends downward, which is a result of filtering more information from the generated images. The lowest PSNR was that in the 63^rd state ISED image. For this state, all the state ensembles are activated, and the maximum amount of information is filtered. The PSNR of the full-sized image was 1.65 dB, and that of the cropped image was 2.85 dB.

Figure 11. PSNR values of 63 ISED-generated images. The cropped images of M87 can be seen in Figure 9.

In Figure 12 the IMSE is also in line with the ISED states. A lower IMSE corresponds to a higher SSIM. The fourth ISED state had the lowest IMSE. The score of the full-sized image was 2.28 and that for the cropped image was 4.75. The IMSE trends upward as more information is removed from the original image and transferred to the ISED filter. The highest IMSE is in the 63^rd state. This is again a result of all of the ISED ensemble states being activated.

Figure 12. IMSE values of 63 ISED-generated images. The cropped images of M87 can be seen in Figure 9.

Different informative features of the original image of M87 have different technical requirements. Some informative features should be kept and enhanced, whereas one should attempt to minimize or eliminate glare and noise. In Figure 13 the NIQE value of the full-sized original image was 5.54, and the average value of the ISED-generated images was 5.66. By contrast, the NIQE score of the galactic core cropped from the original was 8.22, and the average score of its ISED-generated images was 7.22. The lower scores indicate improved image quality. Therefore, the global perceptual image quality was improved. NIQE uses a quality-aware natural scene statistic feature model and compares it to a multivariate Gaussian fit model [25].

Figure 13. NIQE values of 63 ISED-generated images. The cropped images of M87 can be seen in Figure 9.

The BRISQUE shown in Figure 14 is another no-reference quality metric. Its value for the full-sized original image was 43.56; the average value of its ISED-generated images was 46.46. Lower BRISQUE scores indicate superior perceptual image quality. The average score for the full-sized original image is close to the average score of the generated images. The BRISQUE of the cropped image was 43.38 and the average score for its generated images was 42.92, indicating improvement in the overall perceptual image quality.

Figure 14. BRISQUE values of 63 ISED-generated images. The cropped images of M87 can be seen in Figure 9.

Scores for the PIQE no-reference quality metric in Figure 15, were 57.37 for the full-sized original image and 63.36 for the averaged ISED-generated images. Table 4 presents the PIQE assessment scale ranges from excellent to bad image quality [23]. A lower PIQE score indicates higher image quality. The original image, depicted in Figure 4a, is in the poor quality range, and the average score for the generated images is similar. However, some individual ISED-generated images have PIQE values higher than that of the original. The PIQE value of the cropped original image was 100, which indicates bad quality. The average score of its ISED-generated images was 46.23, which is fair quality. Some of the ISED-generated images fall within the excellent and good ranges. The 24^th state ISED-generated image had the best PIQE value (16.41), which was a marked improvement.

Figure 15. PIQE values of 63 ISED-generated images. The cropped images of M87 shown in Figure 9.

Table 4. PIQE assessment range [23].

Quality Scale	Excellent	Good	Fair	Poor	Bad
Score range	[0,20]	[21,35]	[36,50]	[51,80]	[81,100]

5. Conclusions

The results suggest that ISED is a novel and effective astrophysical image processing method. New revealing ISED-generated images expose previously unknown structures that were imbedded in the information and were presented. M87s core, knots, and jets are much clearer that the original image in many cases. Therefore, we have provided clear evidence of the merit of the proposed method. A mathematical framework for the application of ISED was provided, and two sets of 63 balanced state ISED image and filters were analyzed and compared to those of the original image. Only the balanced ISED state conditions were covered in this paper. Additionally, the majority of the generated images produced improved IQAs over that of NASA’s post-processed image. Particularly, the no-reference quality metrics used indicated that in most cases, the image perceptional quality improved dramatically. The SSIM results did not necessarily reflect the perceived quality of the images, because of the decompositional nature of ISED images and filter, the change in SSIM for the ensemble states was an expected result. Further development of the method will be reported in a follow-up paper.