Coded Aperture Hyperspectral Image Reconstruction
Abstract
:1. Introduction
- Hyperspectral imagers are quite expensive even to this day;
- There is a lack of accessible hyperspectral image repositories;
- Processing algorithms designed as extensions of the ones used for processing RGB images could potentially miss certain special characteristics inherent to the spectral properties of the data.
- Whiskbroom: A point of information is captured over the whole spectral information at once [8]. Scanning point-by-point both spatial dimensions and mapping each capture to a 2D detector would yield a complete datacube;
- Pushbroom: A slit image of the scene containing spatial information along one axis and spectral information along the other is obtained at once [9]. Each slit image is mapped onto a 2D detector. Scanning the spatial dimension perpendicular to the one of the slit images provides a complete datacube;
- Staring: A single spectral band is captured at once containing the full spatial information over a certain spectral slice. Changing the output wavelength of the filter over time leads to a complete datacube;
- A comprehensive overview of the block CASSI model is presented with all the details required for the implementation of a practical sensing and reconstruction system based on this technology;
- A comparison between several traditional reconstruction sparse algorithms is made for the considered scenario;
- A detailed analysis of the different parameters that influence the reconstruction of hyperspectral images is carried out. It is supported by a large variety of computer experiments in which the obtained results are included and discussed.
2. Hyperspectral Images Sensing
- Image data: In hyperspectral imaging, this information is represented by a datacube (various spectral slices over two spatial dimensions). These data hence correspond to the discrete representation of the scene captured with the CASSI camera across the considered spectral bands. Let us denote as the input information from the scene, where are the spatial coordinates and represents the wavelength for a particular spectral band. The corresponding discrete datacube is modeled by a 3D matrix, , with dimensions , where N determines the spatial resolution and L the number of spectral bands;
- Coded aperture: Grids are used to block or unblock the wavelengths of electromagnetic radiation at each spatial coordinate by following a known coded pattern in order to cast a “shadow” upon a plane, which in this case would be the radiation detected by the camera sensor. Their main goal is to obtain data samples with a structure that allows compression while undersampling the data in a way in which measurements obtained from them are highly compressed despite retaining sufficient information to obtain an accurate reconstruction. Let us denote as the binary matrix that represents the coded aperture pattern applied to the spectral information of the scene;
- Prism: This acts as a dispersive element shifting each of the spectral slices along the columns of the datacube. In practice, this means that each 2D matrix representing a fixed wavelength will be moved one column to the right from the previous;
- FPA detector: This corresponds to the image sensing element consisting of an array (typically rectangular) of light-sensing pixels at the focal plane of a lens. In this sensing step, the dispersed information corresponding to the different spectral bands is hence integrated into a single 2D matrix. This resulting matrix will contain the compressed representation of the hyperspectral images and will be called a measurement shot or, directly, a shot.
- An optimized coding was first applied to the spectral information of the scene, i.e., . It is worth highlighting that the same aperture pattern given by was applied to all the L spectral bands obtained with the CASSI camera to generate a measurement shot. Thus, the same spatial coordinates were blocked or unblocked across the spatial dimension. However, note that different aperture patterns would be used for different measurement shots in order to obtain different compressive representations;
- The result from applying these coded apertures is , which was subsequently modified by the dispersive element (prism) as described above, thus producing the dispersed data . It is important to note that the coded aperture pattern and the dispersive effect would decisively determine the construction of the measurement matrix in the general formulation of the CS problem;
- Finally, these dispersed data were sensed by the FPA detector and integrated into a two-dimensional FPA measurement shot. In this way, the dispersed information from the different spectral bands at the same spatial coordinate collapsed into the same area of the FPA detector, and such information was hence combined to obtain a single value into the corresponding entry of the matrix that defines the measurement shot. Let us denote as the matrix that contains the compressive measurements of the hyperspectral image. Note that the number of columns of the matrix Y is given by due to the aforementioned dispersive effect of the prism.
3. CS Formulation
3.1. Sparse Estimation from Compressive Measurements
- Large sparsity levels for the solution by using the norm;
- Low difference between and by using the norm.
- OMP: This is an effective approach and one of the simplest algorithms that forms a part of the greedy pursuit approaches. OMP defines a refinement iterative algorithm where the best candidate from a dictionary is chosen at each step. The dictionary comprises the column vectors from the measurement matrix , and therefore, the best candidate corresponds to the vector of the dictionary that most contributes to decreasing the difference between the current solution and the actual one (see the left term in (8)). After selecting the best candidate, the solution vector is updated, and the iterative algorithm continues the refinement procedure to minimize the residual term .Greedy pursuit approaches generally provide near-optimal sparse approximations for incoherent dictionaries [27]. This premise can be extended for our scenario as long as the dictionary elements and the basis of the sparse domain also satisfy mutual incoherence. The main issue of the basic version of OMP is related to its computational complexity because the number of required iterations is given by the number of nonzero components of the vector to be reconstructed. Hence, its computational cost is acceptable for very sparse solutions, but it could be large otherwise. Related to this issue, it can lead to inaccurate estimations for input data with a large amount of small values in the sparse domain.To solve these drawbacks, more sophisticated methods have been developed based on selecting multiple columns per iteration or pruning the set of active columns at each step (see [23] and the references therein). In this work, we focused on the basic version of OMP because the hyperspectral images were expected to be sufficiently sparse in the frequency domain:
- LASSO: Another fundamental approach for sparse reconstruction consists of employing a convex relaxation of the natural sparse estimation problem. This is performed by replacing the nonconvex norm in (8) by the norm, which is the closest convex function to . Therefore, the optimization problem following this strategy reads as:It is also worth noting the difficulty in selecting an appropriate value for in advance. Therefore, we would actually need to solve the problem above repeatedly for different choices of this parameter and track the obtained solutions. An equivalent problem to (9) is given by the LASSO formulation, which is stated as:There exists several works that have analyzed the convergence of LASSO for the estimation of sparse signals [31,41,42]. Such works confirmed the suitability of LASSO-formulation-based algorithms as they provide accurate approximations for incoherent dictionaries even for noisy CS measurements [41,43].Although OMP and LASSO solve different minimization problems, they are based on the same principles. Both approaches start from an all-zero solution and, then, iteratively construct a sparse solution by considering the correlation between columns of the measurement matrix and the current residual vector. However, OMP usually requires fewer iterations than LASSO to converge to the final solution due to the strategy of handling the active set of candidates. On the contrary, OMP produces less accurate solutions when there is a certain correlation between the dictionary elements (columns of );
- GPSR: This is one of the gradient projection methods. These iterative algorithms solve the relaxed convex problem in (9) by updating the solutions at each step with the gradient of the cost function. These kinds of algorithms have been demonstrated to perform well in a wide range of applications, although they tend to degrade as the regularization term is de-emphasized [24]. Indeed, gradient-based methods are especially suitable for the estimation of very sparse signals where the weight of the regularization terms is significant. Their main advantage is that they are considerably faster than other methods in spite of the fact that they also require solving a set of problems for different values of the regularization parameter. GPSR is able to efficiently solve a sequence of problems such as (9) for a sequence of values of .There exist two main approaches for the implementation of GPSR-based algorithms, although both share the same underlying formulation. The first approach is GPSR-Basic [24], which is similar to a step descent approach. The optimization problem is hence reformulated according to a BCQP formulation such that the resulting cost function is minimized following the negative gradient direction. The iterative solutions are then projected onto the feasible set of solutions determined by the problem constraints. A backtracking line search is also performed to optimize the step parameter.The second approach is the Gradient Projection Barzilai-Borwein (GPSR-BB) algorithm [44]. The main difference with respect to the previous approach is the choice of the step size, which is based on an approximation for the Hessian of the cost function. As a consequence of this choice, the objective function does not necessarily decrease at every iteration, unlike the basic version of GPSR. However, this particular choice for the gradient algorithm step has been proven analytically to produce accurate solutions for simple problems [44];
- IST: Iterative thresholding algorithms [45] are some of the simplest known techniques for sparse reconstruction, along with greedy approaches. Typical iterative thresholding algorithms update the solution at each iteration as follows:Iterative thresholding algorithms are mainly classified into two categories depending on the type of considered thresholding operator: IHT and Iterative Soft Thresholding (IST). In this work, we focused on IST as it generally provides better results for sparse reconstruction. The algorithms based on IST are especially suitable for well-conditioned problems since they have been shown to present slow convergence when the sensing matrix is ill-conditioned. An alternative to solve this issue is the use of the Tw-IST approach, which was proposed in [26] to accelerate the convergence rate of IST algorithms with this class of problems. The main difference of the Tw-IST algorithm with respect to the basic versions of IST is to consider the two previous estimates at each iterate to update the current estimation, rather than considering only the previous one. This modification in the update step has been shown to provide significant gains in terms of execution time for ill-conditioned problems and moderate ones otherwise.
3.2. CS Formulation for the CASSI Setup
- We first place the entries of at the main diagonal;
- We next move positions in the horizontal dimension and N positions in the vertical one and place the entries of again;
- We repeat this procedure L times to complete the construction of the matrix ;
- The rest of elements are set to zero.
4. Practical Images Reconstruction Based on a Block Model
- Dividing the shot into nonoverlapping blocks of a smaller size ();
- Solving the sparse estimation problem individually for each measurement block in order to obtain an estimate of the datacube chunk, which produces the considered block of measurements;
- Assembling the entire datacube from the partial reconstructions of the different blocks.
- For each block, we transform the estimate sparse vector back to the original domain, obtaining ;
- We reconstruct the corresponding oblique parallelepiped from ;
- We place the parallelepiped at the appropriate coordinates within the 3D datacube.
5. Experimental Results
- For each input hyperspectral image, measurements shots were generated by using different random aperture patterns;
- The block CASSI model is defined for a particular value of the block size B. The partial measurement matrices were constructed for a given sparsifying basis and considering the aperture patterns to generate the shots;
- The nonoverlapping blocks of the obtained shots together with the corresponding partial measurement matrices were employed to reconstruct each oblique parallelepiped in the datacube. Different algorithms proposed for the sparse estimation problem were considered and assessed in this scenario;
- The datacube partial reconstructions were assembled to complete the recovering of the hyperspectral image from its compressive measurements;
- This procedure was repeated for all the items of an image database and the parameters of interest were averaged to obtain the numerical results presented in this section.
5.1. Sparsity Level
- A clear conclusion is that DCT was the basis that obtained the sparsest representation of the data when it was transformed to the sparse domain. This is reasonable because wavelet-based transforms are known to work better when they are applied to the whole image (i.e., considerably large block sizes) to efficiently exploit the properties of the multilevel decomposition [54];
- For a similar reason, the simple Haar transform (single-level) provided poor sparsity values as it was only applied once to the different datacube blocks. Recall that the capacity of the energy compaction of wavelet transforms increases with the number of applied levels. This also explains that the sparsity level is practically the same for this basis regardless of the block size;
- DCT worked well even for small blocks as it approached the optimal Karhunen–Loève transform when the spatial information was highly correlated [48], as it was in this case;
- Increasing the block size in general increased the sparsity level. This conclusion matches the initial intuition. On the one hand, larger blocks usually led to larger data correlation levels for hyperspectral images. This fact benefits DCT-based schemes, which are able to represent more spatial information with a small increase in the amount of relevant frequency coefficients. On the other hand, larger blocks allow the utilization of multilevel wavelets with higher levels of decomposition;
- The increase in the sparsity level was especially remarkable for the Daubechies wavelet with four coefficients (DB4). Although DB4 provided lower levels of sparsity for small block sizes, its levels improve significantly when the block size increased, thus achieving a performance better than the multilevel Haar basis for blocks of size . Note that the basis matrix for DB4 had larger dimensions than for multilevel Haar because the number of wavelet coefficients was larger. This fact determines the maximum number of decomposition levels that can be applied to each datacube for a given size. As an example, for a block size , each datacube can be transformed using a three-level decomposition with multilevel Haar, whereas a two-level decomposition should be employed for DB4.
5.2. Objective Quality of the Reconstructed Images
5.2.1. Impact of the Reconstruction Algorithm
- As expected, the quality of the reconstructed images improved with the number of shots, i.e., when increasing the number of compressive measurements generated from the input image. The obtained PSNR ranged from low values (about 20 dB) for shots to excellent values (almost 40 dB) for a large number of measurement shots. As observed in Figure 6, the PSNR curves present a logarithmic behavior regardless of the sparse estimation algorithm. This means that the cost of incorporating more measurements is not worth it beyond a certain point;
- LASSO provided the highest average PSNR values regardless of the number of measurement shots. This is probably related to the fact that this algorithm is more robust than other alternatives such as OMP or IST when the columns of the measurement matrix are not completely orthogonal to each other. In our setup, such columns were not totally random because of the particular structure of the dictionary matrices used for the projection operations (see Figure 5);
- OMP and IST recovered the hyperspectral images with similar quality. This makes sense as they intrinsically performed in a similar way even though the implementation of the operations at each iteration significantly differed;
- GPSR-based approaches provided the worst results when using a small number of shots. However, GPSR (basic implementation) was able to slightly outperform OMP and IST algorithms for shots. This is an interesting observation as this class of algorithms stands out because of their lower computational cost, and hence, it was postulated as a low-complexity alternative to obtain accurate reconstructions, especially when a large number of measurements are available;
- The variant of IST based on using the Tw-IST algorithm converged to the performance obtained with the basic IST as the number of shots increased, but the quality of the reconstructions was slightly worse for a small number of measurements;
- Finally, we observed that GPSR-BB was clearly the worst algorithm in our setup. In this case, the alternative step initialization proposed by this method and the fact of allowing the cost function to increase along the iterative procedure did not apparently improve the performance for the CASSI reconstruction problem.
5.2.2. Impact of the Sparsifying Basis
- The DCT basis provided the highest PSNR values for all the range of values. Next, the multilevel Haar curve closely approached the DCT one, although the performance was slightly worse. Finally, the other two transforms (single-level Haar and DB4) achieved lower PSNR values, especially for a small number of shots. Notice that this behavior is coherent with the sparsity levels achieved by the set of bases for blocks of size (see Table 1). In this setup, the DCT achieved the highest sparsity levels followed closely by the multilevel Haar, and further away, the other two transforms. A higher sparsity level directly implies a smaller number of nonzero coefficients of the input images in the sparse domain, and consequently, a better reconstruction with the available measurements;
- The PSNR curves for the four considered bases converged to the same point for a large number of measurement shots. This is also reasonable because the impact of the sparsity level vanished when the number of available measurement was enough to ensure reconstruction with minimum error, i.e., when that number was larger than the threshold provided by (19). As discussed, this situation only holds for values around 18 or 20. In such a scenario, the number of available measurements allowed accurate reconstructions for the four transform bases, even for those that produced vectors with a larger number of nonzero elements. Indeed, the improvement of obtaining sparser vectors in this situation was negligible in terms of the objective quality of the reconstructed images since the extra measurements hardly improved the accuracy in the estimation procedure.
5.2.3. Impact of the Block Size
- As expected, the PSNR values grew with the block size. Note that increasing the block size led to sparser vectors in the transformed domain, which could be estimated accurately. For each block, we can define the following quantities:
- The PSNR curves followed again a logarithmic tendency. Indeed, the gain in terms of PSNR was almost negligible from to for = 2 shots, and especially for the DCT. In the same way, the gain was imperceptible from to for shots. These results can be explained directly by the statement developed in the previous point and considering the increase of the sparsity levels with the block size presented in Table 1. As observed in the table, the sparsity level grew with the block size, but this increase slowed down as B became larger. This effect was even more pronounced when having more shots as we had a larger amount of measurements available for the reconstruction phase;
- The gains with the DB4 basis were slightly higher than with the DCT. Indeed, the curves tended to approach each other as B grew. This is reasonable as the Daubechies wavelet was shown to work better with larger data blocks, and it also agreed with the sparsity levels observed for this basis in Table 1.
5.3. Execution Time
5.3.1. Impact of the Reconstruction Algorithm
- The slowest method was, with a significant gap, LASSO. This is because of the number of iterations required for this method when the final goal is to achieve good quality results for the reconstructions. As already discussed in Section 3.1, this result was expected as LASSO usually requires a number of iterations larger than OMP or IST to converge to the final solution due to its strategy of handling the active set of candidates;
- OMP and IST had comparable times for both block sizes. This was due to the fact that, in both cases, the number of required iterations was closely related to the sparsity level of the solutions;
- The use of Tw-IST reduced the time required to reconstruct the hyperspectral images with respect to the basic version of IST, especially for blocks. As observed, the obtained reconstruction times were of the same order as those of the GPSR-based methods;
- Both GPSR approaches were fast algorithms, with GPSR-Basic being the quickest one in both cases. They were clearly the fastest approaches for blocks. Recall that the computational complexity of gradient-based algorithms is considerable lower than other iterative approaches based on pursuit methods or homotopy-based techniques [23];
- The execution time increased with the block size for OMP, IST, and LASSO. These three algorithms are iterative procedures where the number of iterations grows with the number of nonzero elements in the input vector to be estimated. It is clear that such a number would be larger for bigger input blocks as the total number of input elements quadratically grew with B;
- Conversely, the execution time for GPSR-based approaches was shorter for than for . These results may seem counterintuitive, but the explanation of this behavior is closely related to the values of the regularization parameter obtained for each block size. In the case of , these values were in general lower than for , which provided less sparse solutions in the estimation procedure. In this case, the convergence of the gradient-based approaches was faster and the number of required iterations to produce the sparse solutions was smaller.
5.3.2. Impact of the Block Size
- The execution time exponentially grew with the block size. Indeed, this time becomes prohibitive for most algorithms with blocks. The only exception to this behavior is for GPSR-based approaches and the Tw-IST algorithm with and blocks, which was mentioned in the previous section;
- Execution times increased for the GPSR-Basic, GPSR-BB, and Tw-IST algorithms for sizes larger than . In this case, the dimension of the variables involved in the reconstruction problem significantly increased with , while the observed values were of the same order, which led to similar convergence properties. In any case, these approaches were still the fastest algorithms to reconstruct the hyperspectral images regardless of the block size;
- LASSO was clearly the slowest algorithm, and its execution time increase with the block size was more noticeable compared to the rest of algorithms. Indeed, the time for blocks was already extremely high considering that it was even superior to the time of most algorithms for blocks;
- OMP and IST required intermediate times to reconstruct the hyperspectral images. Nevertheless, the time growth was more pronounced in the case of the OMP algorithm. This indicates that IST could be more appropriate for large dimensions of the input vectors;
- Finally, these results confirmed that the use of Tw-IST significantly reduced the reconstruction times (by approximately half) with respect to the basic version of IST, especially as the block size was larger. As in the previous experiments, the execution times obtained with Tw-IST were slightly higher than for GPSR-based methods, but in the same order. These results suggest that the estimation problem with CASSI was not well-conditioned, probably due to the fact that the structure of the measurements matrix specifically depends on the considered aperture patterns, and hence, the resulting dictionary was not utterly incoherent.
5.4. Impact of the Number of Shots
- All algorithms increased the time taken to reconstruct the images as more shots were considered. This fact meets our expectations since adding more shots implies increasing the variable dimensions and, therefore, increasing the execution time. This growth was more perceptible for a small number of shots, and it apparently lessened as became larger;
- LASSO was the slowest algorithm by far regardless of the number of shots. This agrees with the results obtained in the rest of the experiments;
- Focusing on the LASSO algorithm, the impact of increasing the number of shots was significantly larger for blocks (see Figure 11). This was expected since the dimension of the involved variables also depended on the block size B;
- The GPSR-based algorithms generally provided the fastest times and showed a slight increase in time with the number of shots. This is an interesting result from the point of view of saving computational resources since GPSR-Basic was the best alternative for scenarios where a large number of shots was available.
5.5. Perceptual Quality
6. Discussion
- LASSO was the algorithm that provided the image reconstructions with the highest objective quality. It was able to provide excellent performance in spite of the particular structure of the columns in the CASSI measurement matrix. However, these better results were achieved at the expense of increasing the execution time required for the reconstruction step, especially when employing a larger number of shots or larger block sizes. Hence, LASSO is the best alternative for applications where it is essential to work with high-quality reconstructions, for applications without delay constraints, and for devices with enough computational capabilities;
- GPSR-Basic is an appealing alternative as it properly balanced image quality and time consumption. It was especially interesting for the case of many shots since, in this situation, it was able to reconstruct the hyperspectral images with excellent quality and negligible impact on the time consumption;
- The best option to improve the quality of the image reconstructions is to increase the number of shots generated from the input scene. If possible, this alternative is preferable to increasing the block size in the block CASSI model, both in terms of quality improvement and required reconstruction time. From the reconstruction perspective, the increase of the generated shots allows producing hyperspectral images with excellent quality at an affordable computational cost and delay. The use of larger block sizes for a given number of shots led to a slight improvement of the reconstruction quality at the expense of increasing the computational cost and reconstruction delay. However, generating more shots implies more hardware complexity, more measurements to be stored/transmitted, and lower compression efficiency. Again, the final decision about the system configuration will depend on the application requirements, the hardware features of the CASSI prototype, and the computational resources of the devices where the hyperspectral images will be reconstructed;
- In those scenarios where it is not possible to generate a large number of shots, we can opt to increase the block size. In any case, the decision of using LASSO or GPSR will depend on the available computational resources;
- The benefits of generating more shots for the hyperspectral image vanished beyond a certain point, which can be determined from the CS theory. This threshold value will mainly depend on the sparsity level of the input image;
- The impact of the sparsifying bases on the system performance is less relevant than that of other configuration parameters. In any way, the DCT basis is apparently the best choice for the considered block sizes as it is able to produce sparser vectors in the transformed domain.
- The use of more sophisticated and modern reconstruction algorithms, including those based on deep learning networks;
- The impact of the transmission of the measurements over a certain communication channel on the reconstruction procedure and, correspondingly, the impact of noisy measurements on the reconstruction of hyperspectral images;
- The optimization of the coded aperture patterns for the generation of the measurement shots;
- A comparison of the considered block model with respect to using overlapped blocks in the reconstruction problem.
7. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
AMP | Approximate Message-Passing |
BCQP | Bound-Constrained Quadratic Programming |
CASSI | Coded Aperture Snapshot Spectral Imagers |
CS | Compressive Sensing |
DCT | Discrete Cosine Transform |
DWT | Discrete Wavelet Transform |
FPA | Focal Plane Array |
GPSR | Gradient Projection Sparse Reconstruction |
IHT | Iterative Hard Thresholding |
IoT | Internet of Things |
IST | Iterative Shrinkage Thresholding |
LASSO | Least Absolute Shrinkage and Selection Operator |
OMP | Orthogonal Matching Pursuit |
PSNR | Peak Signal-to-Noise Ratio |
Tw-IST | Two-Step Iterative Shrinkage/Thresholding |
References
- Fei, B. Hyperspectral imaging in medical applications. In Data Handling in Science and Technology; Elsevier: Amsterdam, The Netherlands, 2020; Volume 32, pp. 523–565. [Google Scholar]
- Halicek, M.; Dormer, J.D.; Little, J.V.; Chen, A.Y.; Fei, B. Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning. Biomed. Opt. Express 2020, 11, 1383–1400. [Google Scholar] [CrossRef]
- Guilloteau, C.; Oberlin, T.; Berné, O.; Dobigeon, N. Hyperspectral and multispectral image fusion under spectrally varying spatial blurs–Application to high dimensional infrared astronomical imaging. IEEE Trans. Comput. Imaging 2020, 6, 1362–1374. [Google Scholar] [CrossRef]
- Lu, B.; Dao, P.D.; Liu, J.; He, Y.; Shang, J. Recent advances of hyperspectral imaging technology and applications in agriculture. Remote Sens. 2020, 12, 2659. [Google Scholar] [CrossRef]
- Caballero, D.; Calvini, R.; Amigo, J.M. Hyperspectral imaging in crop fields: Precision agriculture. In Data Handling in Science and Technology; Elsevier: Amsterdam, The Netherlands, 2020; Volume 32, pp. 453–473. [Google Scholar]
- Gonzalez, S.A.R.; Shimoni, M.; Plaza, J.; Plaza, A.; Renhorn, I.; Ahlberg, J. The Detection of Concealed Targets in Woodland Areas using Hyperspectral Imagery. In Proceedings of the 2020 IEEE Latin American GRSS ISPRS Remote Sensing Conference (LAGIRS), Santiago, Chile, 22–26 March 2020; pp. 451–455. [Google Scholar] [CrossRef]
- Willett, R.M.; Duarte, M.F.; Davenport, M.A.; Baraniuk, R.G. Sparsity and Structure in Hyperspectral Imaging: Sensing, Reconstruction, and Target Detection. IEEE Signal Process. Mag. 2014, 31, 116–126. [Google Scholar] [CrossRef] [Green Version]
- Jia, J.; Wang, Y.; Chen, J.; Guo, R.; Shu, R.; Wang, J. Status and application of advanced airborne hyperspectral imaging technology: A review. Infrared Phys. Technol. 2020, 104, 103115. [Google Scholar] [CrossRef]
- Jurado, J.M.; Pádua, L.; Hruška, J.; Feito, F.R.; Sousa, J.J. An Efficient Method for Generating UAV-Based Hyperspectral Mosaics Using Push-Broom Sensors. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 6515–6531. [Google Scholar]
- Hagen, N.A.; Kudenov, M.W. Review of snapshot spectral imaging technologies. Opt. Eng. 2013, 52, 1–23. [Google Scholar] [CrossRef] [Green Version]
- West, M.; Grossman, J.; Galvan, C. Commercial Snapshot Spectral Imaging: The Art of the Possible; Technical Report; The MITRE Corporation: McLean, VA, USA, 2018. [Google Scholar]
- Cheng, N.; Huang, H.; Zhang, L.; Wang, L. Snapshot Hyperspectral Imaging Based on Weighted High-order Singular Value Regularization. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021; pp. 1267–1274. [Google Scholar]
- Wagadarikar, A.; John, R.; Willett, R.; Brady, D. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 2008, 47, B44–B51. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Arce, G.R.; Brady, D.J.; Carin, L.; Arguello, H.; Kittle, D.S. Compressive Coded Aperture Spectral Imaging: An Introduction. IEEE Signal Process. Mag. 2014, 31, 105–115. [Google Scholar] [CrossRef]
- Hlubuček, J.; Lukeš, J.; Václavík, J.; Žídek, K. Enhancement of CASSI by a zero-order image employing a single detector. Appl. Opt. 2021, 60, 1463–1469. [Google Scholar] [CrossRef] [PubMed]
- Donoho, D.L. Compressed sensing. IEEE Trans. Inf. Theory 2006, 52, 1289–1306. [Google Scholar] [CrossRef]
- Donoho, D.L.; Huo, X. Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 2001, 47, 2845–2862. [Google Scholar] [CrossRef] [Green Version]
- Candes, E.J.; Tao, T. Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? IEEE Trans. Inf. Theory 2006, 52, 5406–5425. [Google Scholar] [CrossRef] [Green Version]
- Atta, R.E.; Kasem, H.M.; Attia, M. A comparison study for image compression based on compressive sensing. In Proceedings of the Eleventh International Conference on Graphics and Image Processing (ICGIP 2019), Hangzhou, China, 12–14 October 2019; Volume 11373, p. 1137315. [Google Scholar]
- Mousavi, A.; Rezaee, M.; Ayanzadeh, R. A survey on compressive sensing: Classical results and recent advancements. arXiv 2019, arXiv:1908.01014. [Google Scholar]
- Manchanda, R.; Sharma, K. A Review of Reconstruction Algorithms in Compressive Sensing. In Proceedings of the 2020 International Conference on Advances in Computing, Communication & Materials (ICACCM), Dehradun, India, 21–22 August 2020; pp. 322–325. [Google Scholar]
- Chatterjee, A.; Yuen, P.W. Rapid Estimation of Orthogonal Matching Pursuit Representation. In Proceedings of the IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA, 26 September–2 October 2020; pp. 1315–1318. [Google Scholar]
- Tropp, J.A.; Wright, S.J. Computational Methods for Sparse Solution of Linear Inverse Problems. Proc. IEEE 2010, 98, 948–958. [Google Scholar] [CrossRef] [Green Version]
- Figueiredo, M.A.T.; Nowak, R.D.; Wright, S.J. Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems. IEEE J. Sel. Top. Signal Process. 2007, 1, 586–597. [Google Scholar] [CrossRef] [Green Version]
- Herrity, K.K.; Gilbert, A.C.; Tropp, J.A. Sparse Approximation Via Iterative Thresholding. In Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, France, 14–19 May 2006; Volume 3, p. III. [Google Scholar] [CrossRef] [Green Version]
- Bioucas-Dias, J.M.; Figueiredo, M.A.T. A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration. IEEE Trans. Image Process. 2007, 16, 2992–3004. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Donoho, D.L.; Maleki, A.; Montanari, A. Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 2009, 106, 18914–18919. [Google Scholar] [CrossRef] [Green Version]
- Borgerding, M.; Schniter, P.; Rangan, S. AMP-inspired deep networks for sparse linear inverse problems. IEEE Trans. Signal Process. 2017, 65, 4293–4308. [Google Scholar] [CrossRef]
- Arguello, H.; Correa, C.V.; Arce, G.R. Fast lapped block reconstructions in compressive spectral imaging. Appl. Opt. 2013, 52, D32–D45. [Google Scholar] [CrossRef]
- Shi, Y.Q.; Sun, H. Image and Video Compression for Multimedia Engineering; CRC Press: Boca Raton, FL, USA, 1999. [Google Scholar]
- Donoho, D.; Elad, M. Optimally sparse representation in general (non- orthogonal) dictionaries via l1 minimization. Proc. Natl. Acad. Sci. USA 2003, 100, 2197–2202. [Google Scholar] [CrossRef] [Green Version]
- Duarte, M.F.; Eldar, Y.C. Structured Compressed Sensing: From Theory to Applications. IEEE Trans. Signal Process. 2011, 59, 4053–4085. [Google Scholar] [CrossRef] [Green Version]
- Tropp, J.A.; Wakin, M.B.; Duarte, M.F.; Baron, D.; Baraniuk, R.G. Random Filters for Compressive Sampling and Reconstruction. In Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, France, 14–19 May 2006; Volume 3. [Google Scholar] [CrossRef] [Green Version]
- Zhang, J.; Zhao, C.; Gao, W. Optimization-Inspired Compact Deep Compressive Sensing. IEEE J. Sel. Top. Signal Process. 2020, 14, 765–774. [Google Scholar] [CrossRef] [Green Version]
- Elad, M. Optimized Projections for Compressed Sensing. IEEE Trans. Signal Process. 2007, 55, 5695–5702. [Google Scholar] [CrossRef]
- Wang, Z.; Arce, G.R. Variable Density Compressed Image Sampling. IEEE Trans. Image Process. 2010, 19, 264–270. [Google Scholar] [CrossRef] [Green Version]
- Candès, E.; Romberg, J. Sparsity and incoherence in compressive sampling. Inverse Probl. 2007, 23, 969–985. [Google Scholar] [CrossRef] [Green Version]
- Li, B.; Zhang, L.; Kirubarajan, T.; Rajan, S. Projection matrix design using prior information in compressive sensing. Signal Process. 2017, 135, 36–47. [Google Scholar] [CrossRef]
- Hong, T.; Li, X.; Zhu, Z.; Li, Q. Optimized structured sparse sensing matrices for compressive sensing. Signal Process. 2019, 159, 119–129. [Google Scholar] [CrossRef] [Green Version]
- Baraniuk, R.G. Compressive Sensing [Lecture Notes]. IEEE Signal Process. Mag. 2007, 24, 118–121. [Google Scholar] [CrossRef]
- Donoho, D.L.; Elad, M.; Temlyakov, V.N. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inf. Theory 2005, 52, 6–18. [Google Scholar] [CrossRef]
- Zhang, T. On the consistency of feature selection using greedy least squares regression. J. Mach. Learn. Res. 2009, 10. [Google Scholar]
- Tropp, J.A. Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 2006, 52, 1030–1051. [Google Scholar] [CrossRef] [Green Version]
- Barzilai, J.; Borwein, J.M. Two-Point Step Size Gradient Methods. IMA J. Numer. Anal. 1988, 8, 141–148. [Google Scholar] [CrossRef]
- Blumensath, T.; Davies, M.E. Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 2009, 27, 265–274. [Google Scholar] [CrossRef] [Green Version]
- Duarte, M.F.; Baraniuk, R.G. Kronecker Compressive Sensing. IEEE Trans. Image Process. 2012, 21, 494–504. [Google Scholar] [CrossRef]
- Antonini, M.; Barlaud, M.; Mathieu, P.; Daubechies, I. Image coding using wavelet transform. IEEE Trans. Image Process. 1992, 1, 205–220. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Gonzalez, R.C.; Woods, R.E. Digital Image Processing, 3rd ed.; Pearson: London, UK, 2008. [Google Scholar]
- Wei, Z.; Zhang, J.; Xu, Z.; Liu, Y. Optimization methods of compressively sensed image reconstruction based on single-pixel imaging. Appl. Sci. 2020, 10, 3288. [Google Scholar] [CrossRef]
- Tung, T.; Gündüz, D. SparseCast: Hybrid Digital-Analog Wireless Image Transmission Exploiting Frequency-Domain Sparsity. IEEE Commun. Lett. 2018, 22, 2451–2454. [Google Scholar] [CrossRef] [Green Version]
- Hyperspectral Color Imaging Repository. Available online: https://sites.google.com/site/hyperspectralcolorimaging/dataset/general-scenes (accessed on 6 March 2021).
- TokyoTech Dataset. Available online: http://www.ok.sc.e.titech.ac.jp/res/MSI/MSIdata31.html (accessed on 15 March 2021).
- Real-World Hyperspectral Images Database. Available online: http://vision.seas.harvard.edu/hyperspec/download.html (accessed on 3 March 2021).
- Akansu, A.; Medley, M. Wavelet, Subband and Block Transforms in Communications and Multimedia; Springer: New York, NY, USA, 1999. [Google Scholar]
Basis/Block Size | 4 × 4 | 8 × 8 | 16 × 16 | 32 × 32 |
---|---|---|---|---|
DCT | 0.8717 | 0.9030 | 0.9180 | 0.9296 |
Haar (single-level) | 0.8052 | 0.8072 | 0.8097 | 0.8105 |
Haar (multilevel) | 0.8564 | 0.8752 | 0.8852 | 0.8905 |
Daubechies (DB4) | 0.8034 | 0.8582 | 0.8835 | 0.9057 |
Algorithm/Size | 4 × 4 | 8 × 8 | 16 × 16 | 32 × 32 |
---|---|---|---|---|
OMP | 2.8510 | 7.7574 | 165.3975 | 1784.93 |
LASSO | 13.1868 | 36.5139 | 524.9444 | 5940.48 |
GPSR | 3.4153 | 2.0287 | 24.3466 | 188.75 |
GPSR-BB | 5.6395 | 2.5284 | 29.8273 | 229.26 |
IST | 3.4501 | 6.7514 | 70.8613 | 512.07 |
Tw-IST | 3.2421 | 2.3771 | 31.0239 | 243.58 |
Algorithm/#Shots | 2 | 8 | 14 | 20 |
---|---|---|---|---|
OMP | 2.8510 | 25.0447 | 55.8019 | 70.2041 |
LASSO | 13.1868 | 87.1824 | 143.0577 | 169.5150 |
GPSR | 3.4153 | 5.9478 | 7.3869 | 7.9944 |
GPSR-BB | 5.6395 | 6.9739 | 8.1231 | 8.6928 |
IST | 3.4421 | 17.1846 | 30.2728 | 38.8165 |
Tw-IST | 3.2421 | 6.1562 | 8.1849 | 9.3319 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
García-Sánchez, I.; Fresnedo, Ó.; González-Coma, J.P.; Castedo, L. Coded Aperture Hyperspectral Image Reconstruction. Sensors 2021, 21, 6551. https://doi.org/10.3390/s21196551
García-Sánchez I, Fresnedo Ó, González-Coma JP, Castedo L. Coded Aperture Hyperspectral Image Reconstruction. Sensors. 2021; 21(19):6551. https://doi.org/10.3390/s21196551
Chicago/Turabian StyleGarcía-Sánchez, Ignacio, Óscar Fresnedo, José P. González-Coma, and Luis Castedo. 2021. "Coded Aperture Hyperspectral Image Reconstruction" Sensors 21, no. 19: 6551. https://doi.org/10.3390/s21196551
APA StyleGarcía-Sánchez, I., Fresnedo, Ó., González-Coma, J. P., & Castedo, L. (2021). Coded Aperture Hyperspectral Image Reconstruction. Sensors, 21(19), 6551. https://doi.org/10.3390/s21196551