Adaptive Memory-Augmented Unfolding Network for Compressed Sensing

Abstract

Deep unfolding networks (DUNs) have attracted growing attention in compressed sensing (CS) due to their good interpretability and high performance. However, many DUNs often improve the reconstruction effect at the price of a large number of parameters and have the problem of feature information loss during iteration. This paper proposes a novel adaptive memory-augmented unfolding network for compressed sensing (AMAUN-CS). Concretely, without loss of interpretability, we integrate an adaptive content-aware strategy into the gradient descent step of the proximal gradient descent (PGD) algorithm, driving it to adaptively capture the adequate features. In addition, we extended AMAUN-CS based on the memory storage mechanism of the human brain and propose AMAUN-CS⁺ to develop the dependency of deep information across cascading stages. The experimental results show that the AMAUN-CS model surpasses other advanced methods on various public benchmark datasets while having lower complexity in training.

1. Introduction

Compressed sensing theory shows that when a signal is sparse or can be sparsely represented in a certain transform domain, the original signal can be recovered from the sampled signal at a much lower frequency than the Nyquist sampling frequency. CS is capable of sampling and compressing signals at the same time, which greatly alleviates the problem of wasted resources at the sampling end of the signal [1,2]. Since its proposal, this theory has been widely used in many fields, such as single-pixel imaging [3], cognitive radio networks [4], video compression [5], information security [6], speech and sound processing [7], radar imaging [8], low-dose CT [9], optical tomography [10], fluorescence microscopy [11], and MRI [12].

Mathematically, the purpose of CS reconstruction is to reconstruct the original signal $\mathrm{x}\in {ℝ}^{\mathrm{N}}$ from a random measurement $\mathrm{y}\in {ℝ}^{\mathrm{M}}$, which can be expressed as:

$$\mathrm{y}=\mathrm{\Phi }\mathrm{x},$$

(1)

where $\mathrm{\Phi }\in {ℝ}^{\mathrm{M}\times \mathrm{N}}$ represents the random measurement matrix, and $\mathrm{r}=\mathrm{M}/\mathrm{N}$ is defined as the sampling rate of CS. Since in CS $\mathrm{M}\ll \mathrm{N}$, the reconstruction of X by $\mathrm{y}$ and $\mathrm{\Phi }$ is an ill-posed inverse problem, this paper focuses on image reconstruction based on CS in natural scenes. Signal reconstruction is a difficult problem in CS theory, and traditional methods generally use some structured sparsity as an image prior, and then solve the sparsity regularization optimization problem in an iterative manner [13,14,15,16], which has the advantage of utilizing the a priori knowledge of the signal to model the signal, and thus possesses a certain degree of mathematical interpretability. However, its disadvantage is that the iterative algorithm leads to excessive computational complexity, while the need to manually select the optimal parameters limits the reconstruction performance of the CS algorithm.

In recent years, thanks to the powerful data processing capabilities of deep learning in computer vision [17], numerous CS methods have been proposed, which are broadly classified into two categories: (1) deep neural network-based methods; (2) optimization-inspired deep unfolding networks. Most existing deep neural network-based methods [18,19,20,21,22,23] achieve reconstruction by directly learning the mapping function between an image and its CS measurements, and this way of learning a priori from an external dataset ignores the internal imaging model, leading to its lack of flexibility in different imaging modalities. To address this problem, deep unfolding networks unfold traditional optimization algorithms into deep neural networks in order to improve the fidelity of the imaging model through a data-driven prior, achieving a balance between speed and interpretability.

However, most existing deep-learning reconstruction methods still have some shortcomings. For example, due to the weight-sharing nature of convolutional neural networks, the extracted features are mainly affected by the spatial distribution of pixels, thus making it difficult for the network to flexibly and adaptively adjust and learn according to the image content. In addition, many network frameworks are purely oriented to the reconstruction effect, and many stacked network layers are used to obtain better reconstruction performance, which also leads to a huge number of parameters and high computational costs. Finally, in order to enhance the information transfer capability between adjacent iterations, many deep unfolding networks tend to introduce residual feature information during the optimization process; however, in this way, the information transfer effect is limited, and the fault tolerance is poor. All these problems limit the further improvement of image reconstruction quality.

To cope with the above deficiencies, we propose an adaptive memory-augmented unfolding network for compressed sensing, in which we design a content-aware adaptive gradient descent module. The module adaptively adjusts the output result of algorithmic gradient descent by dynamic weights according to different scene elements, so as to obtain a feature gradient representation with stronger generalization. Meanwhile, we construct a lightweight dual-scale gated denoising module, which flexibly extracts useful information through the gating mechanism and denoises the image at different scales to improve the denoising performance. In addition, in order to be applicable to different application scenarios, this paper extends AMAUN-CS, based on which we utilize the memory storage mechanism of the human brain to design the deep memory transfer module (DMTM) for long-term memory of the information between neighboring iterations and name this network structure as AMAUN-CS⁺. Extensive experiments show that our proposed AMAUN-CS and AMAUN-CS⁺ have superior reconstruction performance compared with other CS methods on the Set11, Urban100, and BSD68 benchmark datasets.

Briefly, the main contributions of this paper can be summarized as follows:

• We propose a novel adaptive memory-augmented unfolding network for compressed sensing that can adaptively capture more features and recover more details and textures.
• We designed a content-aware adaptive gradient descent module (CAAGDM), which can flexibly extract information and adaptively adjust the output results. Meanwhile, we design a lightweight dual-scale gated denoising module (LDGDM), which effectively reduces the impact of different noises in network learning. In addition, we propose a DMTM, which aims to enhance the transfer and interaction of information in the optimization iteration phase.
• Extensive experimental results on several benchmark datasets show that the proposed AMAUN-CS and AMAUN-CS⁺ have superior reconstruction performance on the compressed sensing reconstruction task compared to most methods.

2. Related Work

CS Reconstruction Approaches

Traditionally, the inverse problem of compressed imaging is usually solved by solving the following optimization method:

$$\mathrm{x}=\underset{\mathrm{x}}{\mathrm{argmin}}{\displaystyle \frac{1}{2}}{‖\mathrm{\Phi }\mathrm{x}-\mathrm{y}‖}_2^2+\mathrm{\lambda }\mathrm{R}\left(\mathrm{x}\right),$$

(2)

where $\mathrm{\lambda }$ represents the regularization parameter, and R(x) denotes the regularization term associated with the prior knowledge of the image. This approach usually adopts an iterative optimization framework, e.g., approximate message passing (AMP) [13], iterative shrinkage thresholding algorithm (ISTA) [14], half-quadratic splitting (HQS) [15] algorithms, and the alternating direction multiplier method (ADMM) [16], which usually have good convergence and theoretical-analytical advantages but are often constrained by high computational complexity and low adaptivity.

In recent years, compressed sensing methods based on deep learning have been proposed one after another, demonstrating good reconstruction results and fast inference speeds. Shi et al. [18] proposed a method using convolutional layers of neural networks for the first time instead of the traditional hand-designed measurement matrices, which jointly optimizes the sampling and reconstruction networks. Fan et al. [19] proposed an improved multi-scale convolutional layer neural network model for input images for sampling. Regarding reconstruction, Kulkarni et al. [20] proposed an improved non-iterative reconstruction network (ReconNet) method based on convolutional neural networks, which achieves non-iterative reconstruction through weight sharing, thus reducing the computational complexity. Yao et al. [21] proposed a deep residual reconstruction network (DR²-Net) method, where DR2-Net uses a residual network to learn the inverse mapping between measurements and reconstructed images. However, all the above methods inevitably produce block artifacts in the reconstructed image. Thereafter, Zhou et al. [22] proposed a multi-channel block compression-based perceptual deep network (DeepBCS) method, which reduces the artifactual noise by using multi-channel block-by-block sampling and whole-image-based denoising. Tian et al. [23] proposed the use of generative adversarial networks combined with pixel loss to optimize the reconstruction results, which addresses the problem of over-smoothing and lack of structural information, such as texture after image reconstruction. This class of deep neural network methods lacks theoretical interpretability due to the black-box principle of its modeling, thus falling into a data-driven dependency, where the extent of its resource requirements is proportional to the data size.

Deep unfolding network maps the reconstruction iteration process of mathematical algorithms into a deep network model; Zhang et al. [24] proposed an optimization-inspired iterative shrinkage thresholding algorithm (ISTA). ISTA-Net combines the traditional iterative shrinkage thresholding algorithm with a convolutional neural network, and uses a nonlinear transform to solve the proximal mapping associated with the sparse-induced regularization factor to further improve the quality of reconstructed images. Thereafter, Zhang et al. [25] proposed an optimization-inspired interpretable deep network (OPINE-Net⁺) based on ISTA-Net. OPINE-Net+ performs deep reconstruction through the local similarity between image blocks, which effectively mitigates the block effect. Yang et al. [26] proposed a compression based on the alternating direction multiplier method for the compression sensing network (ADMM-CSNet), and ADMM-CSNet achieved remarkable results in MRI. Zhang et al. [27] developed a multilayer network from the denoising point of view by iteratively unfolding the approximate message-passing algorithm into a multilayer network. Recently, Shen et al. [28] proposed a deep unfolding network based on a hybrid of convolutional neural network (CNN) and Transformer, which combines the features of CNN and Transformer that can acquire both local and non-local information of the image and apply them to the steps of gradient descent and soft thresholding iterations of ISTA.

Although existing deep unfolding networks inherit good structure from optimization algorithms and have good interpretability, their prevalent design of single-channel images as inputs and outputs for each stage leads to the loss of many image details and greatly hinders information transfer in the network. Moreover, existing solutions lack flexibility in processing information and face high modeling complexity. In this paper, we propose an efficient solution.

3. Proposed Method

3.1. Overall Architecture

The proximal gradient descent (PGD) algorithm is well suited for solving many large-scale linear inverse problems [29], e.g., general CS and CS-MRI. Conventional PGD solves the compressed sensing reconstruction problem in Equation (2) by iteratively updating between the following two steps, and thus, the kth optimization process of the algorithm is denoted as:

$${\mathrm{r}}^{\left(\mathrm{k}\right)}={\mathrm{x}}^{(\mathrm{k}-1)}-{\mathrm{\rho }}^{\left(\mathrm{k}\right)}{\mathrm{\Phi }}^{\top }(\mathrm{\Phi }{\mathrm{x}}^{(\mathrm{k}-1)}-\mathrm{y}),$$

(3)

$${\mathrm{x}}^{\left(\mathrm{k}\right)}=\underset{\mathrm{x}}{\mathrm{argmin}}{\displaystyle \frac{1}{2}}{‖\mathrm{x}-{\mathrm{r}}^{\left(\mathrm{k}\right)}‖}_2^2+\mathrm{\lambda }\mathrm{R}\left(\mathrm{x}\right),$$

(4)

where k denotes the number of iterations, ${\mathrm{x}}^{\left(\mathrm{k}\right)}$ denotes the output image of the kth iteration, y denotes the sampled image, ${\mathrm{\Phi }}^{\mathrm{T}}$ denotes the transpose of the measurement matrix $\mathrm{\Phi }$, Equation (3) denotes the gradient descent step, ${\mathrm{\rho }}^{\left(\mathrm{k}\right)}$ is a learnable descent step size, and Equation (4) denotes the proximal mapping step.

As mentioned earlier, this traditional implementation lacks adaptivity and has information loss due to image-level inter-stage transmission. To rectify these weaknesses, we propose AMAUN-CS, which possesses a powerful data processing speed based on the deep neural network approach while fully retaining the core idea of the traditional iterative optimization algorithm to further enhance the theoretical interpretability of the network. The overall architecture of the network is shown in Figure 1a, and AMAUN-CS consists of a sampling subnetwork, an initial reconfiguration subnetwork, and a deep iterative reconfiguration subnetwork. Among them, the sampling sub-network replaces the traditional hand-designed measurement matrix with a convolutional layer, and the initial reconstruction sub-network converts the measurements in the low-dimensional space into the high-dimensional space to complete the initial image reconstruction. The reconstructed feature image is gradually optimized through multiple iterations in the deep iterative reconstruction subnetwork, and in each optimization stage, our designed CAAGDM adaptively adjusts the output result of the gradient descent step by using dynamic weights according to the different features of the inputs, and meanwhile, our proposed LDGDM is capable of denoising the image at two scales separately, thus improving the denoising performance. In order to effectively utilize the information of adjacent optimization stages, we further construct the DMTM, which enhances the information transfer capability between adjacent iterations through multiple gating mechanisms, and finally obtains the final reconstruction result using a 1 × 1 convolution after n optimization stages. Thus, as shown in Figure 1b, the kth iteration of AMAUN-CS can be expressed as ($k\in \left\{1,2,\dots ,n\right\}$):

$${\mathrm{r}}^{\left(\mathrm{k}\right)}={\mathcal{H}}_{\mathrm{CAAGDM}}^{\left(\mathrm{k}\right)}({\mathrm{x}}^{(\mathrm{k}-1)},{\mathrm{z}}^{(\mathrm{k}-2)}),$$

(5)

$${\mathrm{x}}^{\left(\mathrm{k}\right)}={\mathcal{H}}_{\mathrm{LDGDM}}^{(\mathrm{k})}\left({\mathrm{r}}^{\left(\mathrm{k}\right)}\right),$$

(6)

where ${\mathrm{r}}^{\left(\mathrm{k}\right)}$ and ${\mathrm{x}}^{\left(\mathrm{k}\right)}$ denote the kth feature domain output result, and Equation (5) represents the gradient descent process in the proximal gradient descent algorithm via our proposed CAAGDM. For the solution of Equation (4), it is well known that it actually corresponds to a denoising problem from a Bayesian perspective [30]. However, the hand-designed regularization term has some limitations because it relies on fixed threshold rules. Therefore, we implement Equation (6) by introducing the LDGDM, respectively, and ${\mathrm{z}}^{(\mathrm{k}-2)}\in {ℝ}^{\mathrm{H}\times \mathrm{W}\times \mathrm{C}-1}$ is obtained by clipping latter C-1 channels from ${\mathrm{x}}^{(\mathrm{k}-2)}$. For the first iteration, the input ${\mathrm{x}}^{\left(0\right)}$ is the feature result obtained by the 3 × 3 convolution of the initial reconstruction result.

3.2. Sampling and Initial Reconfiguration of Subnetworks

Instead of the traditional hand-designed measurement matrix for non-overlapping chunk sampling [18], we utilize a convolutional layer to first divide the input original image x into non-overlapping image chunks of size $\mathrm{B}\times \mathrm{B}$, and then construct $\mathrm{r}\times {\mathrm{B}}^2$ unbiased term convolutional kernels ${\mathrm{\Phi }}_{\mathrm{B}}$ of size $\mathrm{B}\times \mathrm{B}$ and step size $\mathrm{B}\times \mathrm{B}$. Thus, the sampling network can be represented as:

$$\mathrm{y}={\mathrm{\Phi }}_{\mathrm{B}}\ast \mathrm{x},$$

(7)

where $\ast$ denotes the convolution operation and y denotes the measured value.

In order to make full use of the a priori information of the image, in the initial reconstruction stage, we use the transpose matrix ${{\mathrm{\Phi }}_{\mathrm{B}}}^{\mathrm{T}}$ of ${\mathrm{\Phi }}_{\mathrm{B}}$ to back-convolve y to obtain the initial reconstructed image ${\mathrm{x}}_{\mathrm{init}}$; ${{\mathrm{\Phi }}_{\mathrm{B}}}^{\mathrm{T}}$ is actually the ${\mathrm{B}}^2$ unbiased term convolution kernels of size $1\times 1\times \mathrm{r}{\mathrm{B}}^2$. Through the end-to-end training approach, the measurement matrix ${\mathrm{\Phi }}_{\mathrm{B}}$ and the reconstruction network parameters are able to be continuously learned in a data-driven manner, and therefore, the initial reconstruction can be expressed as:

$${\mathrm{x}}_{\mathrm{init}}={{\mathrm{\Phi }}_{\mathrm{B}}}^{\mathrm{T}}\ast \mathrm{y},$$

(8)

3.3. Deep Iterative Reconfiguration Subnetwork

3.3.1. Content-Aware Adaptive Gradient Descent Module

Before performing the gradient descent step, considering the existence of different scene elements in natural images, the variability of the optimal gradient backpropagation for each pixel, and the lack of flexible, adaptive adjustment of the existing network structure with respect to the image content [31], as shown in Figure 2, our proposed CAAGDM contains an attention branch and a convolutional branch. In the attention branch, the module utilizes the non-local self-similarity of the input features to generate model parameters with adaptive weights, which enables the effective extraction of key features in the image. The output results of the two branches are adaptively balanced by dynamic weights to enhance the model’s perceptual ability.

We first use a 3 × 3 convolution and activation function on the input image to obtain its dynamic weights, so the process can be represented as follows:

$$\mathrm{dw}=\mathrm{\delta }({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{x}}^{(\mathrm{k}-1)}\left)\right),$$

(9)

where $\mathrm{\delta }$ denotes the Softmax activation function, ${\mathrm{W}}_{\mathrm{p}}^3$ denotes the convolution kernel size of 3 × 3, and $\mathrm{dw}$ denotes the dynamic weights.

Next, the shallow features are extracted using 3 × 3 convolution in the convolution branch, and the result is denoted as ${\mathrm{x}}_{\mathrm{conv}}\in {ℝ}^{\mathrm{H}\times \mathrm{W}\times \mathrm{C}}$, then the product of ${\mathrm{x}}^{(\mathrm{k}-1)}$ and the dynamic weight $dw$ are used as the input to the attention branch, assuming that the input data are shaped as ${\mathrm{x}}^{(\mathrm{k}-1)}\in {ℝ}^{\mathrm{H}\times \mathrm{W}\times \mathrm{C}}$, denoting that the channel of the input data are C, the height is H, and the width is W. The query (Q), key (K), and value (V) matrices are obtained by using the convolution on them, respectively, and then matrix dimensioning is performed on them to obtain $\mathrm{Q}\in {ℝ}^{\mathrm{HW}\times \mathrm{C}}$ and $\mathrm{K}\in {ℝ}^{\mathrm{C}\times \mathrm{HW}}$, respectively. After that, K is multiplied by Q and then by the activation function to obtain the attention matrix $\mathrm{s}\in {ℝ}^{\mathrm{C}\times \mathrm{C}}$. Multiplying the attention matrix with V to capture the non-local self-similarity result of each feature, this output feature is denoted as ${\mathrm{x}}_{\mathrm{atten}}\in {ℝ}^{\mathrm{H}\times \mathrm{W}\times \mathrm{C}}$, and the process can be represented as:

$$\mathrm{S}=\mathrm{\delta }\left(\mathrm{R}\right({\mathrm{W}}_{\mathrm{p}}^1({\mathrm{k}}^{\mathrm{T}}\left)\right)\otimes \mathrm{R}({\mathrm{W}}_{\mathrm{p}}^1\left(\mathrm{Q}\right)\left)\right),$$

(10)

$${\mathrm{x}}_{\mathrm{atten}}=\mathrm{R}({\mathrm{W}}_{\mathrm{p}}^1\left(\mathrm{k}\right))\otimes \mathrm{S},$$

(11)

where $\mathrm{R}$ denotes Reshape, ${\mathrm{W}}_{\mathrm{p}}^1$ denotes convolutional kernel size of 1 × 1, and $\otimes$ denotes elemental multiplication.

Finally, the output result ${\mathrm{x}}_{\mathrm{DB}}^{(\mathrm{k}-1)}$ is adaptively balanced by dynamic weights for the two-branch output result, a process that is able to discard redundant features and focus on retaining important features. These optimized features are integrated into the gradient descent step in Equation (13) to obtain the output result ${\mathrm{r}}^{\left(\mathrm{k}\right)\prime }$. In addition, in order to enhance the ability of information interaction between adjacent iterations, CAAGDM fuses ${\mathrm{z}}^{(\mathrm{k}-2)}$ in the output stage to obtain the final result ${\mathrm{r}}^{\left(\mathrm{k}\right)}$. Therefore, the process can be expressed as:

$${\mathrm{x}}_{\mathrm{DB}}^{(\mathrm{k}-1)}={\mathrm{x}}_{\mathrm{conv}}\otimes \mathrm{dw}+\mathrm{dw}\otimes {\mathrm{x}}_{\mathrm{atten}},$$

(12)

$${\mathrm{r}}^{\left(\mathrm{k}\right)\prime }={\mathrm{x}}_{\mathrm{DB}}^{(\mathrm{k}-1)}-{\mathrm{\rho }}^{\left(\mathrm{k}\right)}{\mathrm{\Phi }}^{\top }(\mathrm{\Phi }{\mathrm{x}}_{\mathrm{DB}}^{(\mathrm{k}-1)}-\mathrm{y}),$$

(13)

$${\mathrm{r}}^{\left(\mathrm{k}\right)}={\mathrm{W}}_{\mathrm{p}}^3({\mathrm{z}}^{(\mathrm{k}-2)}+{\mathrm{r}}^{\left(\mathrm{k}\right)\prime }),$$

(14)

3.3.2. Lightweight Dual-Scale Gated Denoising Module

The process of solving the proximal mapping can be regarded as denoising the image signal. However, traditional denoising methods tend to remove high-frequency texture information from the input image [32]. CNN-based methods have been shown to be very effective in denoising, but due to the weight-sharing inductive bias of convolutional neural networks, they tend to treat all input features as valid pixel features, which limits their flexibility. Considering the denoising effect and computational complexity, we designed a lightweight dual-scale gated denoising module that processes noise reduction on two scales, separately, to enhance overall performance. Meanwhile, a gating mechanism [33] is introduced in each scale to effectively suppress less informative features and allow only useful information to be passed further in the network optimization structure.

LDGDM is shown in Figure 3. First, the input data are dimensionally expanded using 1 × 1 convolution, after which depth convolution is introduced to effectively capture the information of spatially neighboring pixels. Next, the features are divided into two branches, one of which uses the GELU activation function to realize the gating mechanism. Then, the features are multiplied element by element with the features of the other branch. Finally, the output features at each scale are fused using 1 × 1 convolution, thus filtering low-information features at different scales and retaining only useful information. In addition, the head section of the LDGDM uses hopping connections to map features to the residual domain, which facilitates information flow while further improving the denoising performance of the module, and thus can be expressed as follows:

$${\mathrm{x}}^{\mathrm{k}}={\mathrm{W}}_{\mathrm{p}}^3(\mathrm{Gating} ({\mathrm{r}}^{\mathrm{k}}\left)\right)+{\mathrm{r}}^{\mathrm{k}},$$

(15)

$$\mathrm{Gating}({\mathrm{r}}^{\mathrm{k}})=\mathrm{\sigma }({\mathrm{W}}_{\mathrm{d}}^3({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{r}}^{\mathrm{k}}\left)\right))\otimes {\mathrm{W}}_{\mathrm{d}}^3({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{r}}^{\mathrm{k}}\left)\right)+\mathrm{\sigma }({\mathrm{W}}_{\mathrm{d}}^5({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{r}}^{\mathrm{k}}\left)\right))\otimes {\mathrm{W}}_{\mathrm{d}}^5({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{r}}^{\mathrm{k}}\left)\right),$$

(16)

where $\mathrm{\sigma }$ denotes the GELU activation function, i.e., the gating mechanism, and ${\mathrm{W}}_{\mathrm{d}}^3$ and ${\mathrm{W}}_{\mathrm{d}}^5$ denote the depth convolution of 3 × 3 and 5 × 5 size, respectively.

3.4. Enhanced Version: AMAUN-CS⁺

The human brain can realize the memory mechanism by continuously encoding and storing new information, which contributes to the formation and long-term storage of memories by enhancing the synaptic strength and connections between neurons in the brain [34]. Therefore, an optimization stage of a deep unfolding network can be regarded as a neuron in the human brain, and the information flow between adjacent iterations can be regarded as a memory transfer mechanism. Conventional optimization processes usually introduce residual feature information to enhance the information transfer capability between adjacent iterations. However, this approach is regarded as a kind of short-term memory, which is prone to losing information during multiple iterations. Thanks to the fact that a long short-term memory network (LSTM) can effectively capture long-term dependencies in sequential data [35], we designed a deep memory transfer module based on AMAUN-CS for long-term memorization of information across iterations, and the specific structure is shown in Figure 4. Therefore, the AMAUN-CS+ of the kth iteration can be expressed as:

$${\mathrm{r}}^{\left(\mathrm{k}\right)}={\mathcal{H}}_{\mathrm{CAAGDM}}^{\left(\mathrm{k}\right)}({\mathrm{x}}^{(\mathrm{k}-1)},{\mathrm{z}}^{(\mathrm{k}-2)}),$$

(17)

$${\mathrm{x}}^{\left(\mathrm{k}\right)\prime }={\mathcal{H}}_{\mathrm{LDGDM}}^{\left(\mathrm{k}\right)}\left({\mathrm{r}}^{\left(\mathrm{k}\right)}\right),$$

(18)

$${\mathrm{x}}^{\left(\mathrm{k}\right)}={\mathcal{H}}_{\mathrm{DMTM}}^{\left(\mathrm{k}\right)}\left({\mathrm{x}}^{\left(\mathrm{k}\right)\prime }\right),$$

(19)

where Equations (18) and (19) contain modules consistent with those described in Section 3.2 and Section 3.3, and this section is followed by a description of the manner in which the DMTM is implemented.

The core of DMTM is composed of a memory unit ${\mathrm{c}}^{\left(\mathrm{k}\right)}$ and three gating mechanisms, which are the forgetting gate, input gate, and output gate, and the input features can be memorized by the memory unit in the network for a long time, as shown in Figure 4, where the forgetting gate decides whether to keep the memory of the previous moment to the current moment, the input gate decides the effect of the input of the current moment on the memory, and the output gate decides the memory’s effect on the output ${\mathrm{h}}^{\left(\mathrm{k}\right)}$, which is the hidden state of the DMTM for storing previous memory information with an initial value of zero; ${\mathrm{x}}^{\left(\mathrm{k}\right)\prime }$ denotes the input image.

First, the DMTM extracts the information with convolution for the input image ${\mathrm{x}}^{\left(\mathrm{k}\right)\prime }$ and the hidden state ${\mathrm{h}}^{\left(\mathrm{k}\right)}$, respectively. It then obtains the information of the input gate through the sigmoid activation function. ${\mathrm{f}}^{\left(\mathrm{k}\right)}$ denotes the output of the forgetting gate, which discards the useless information by controlling the information retained by the previous memory unit ${\mathrm{c}}^{(\mathrm{k}-1)}$. The above process can be represented as follows:

$${\mathrm{i}}^{\left(\mathrm{k}\right)}=\sigma ({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{x}}^{\left(\mathrm{k}\right)\prime })+{\mathrm{W}}_{\mathrm{p}}^3\left({\mathrm{h}}^{(\mathrm{k}-1)}\right)+{\mathrm{b}}_{\mathrm{i}}),$$

(20)

$${\mathrm{f}}^{\left(\mathrm{k}\right)}=\mathrm{\sigma }({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{x}}^{\left(\mathrm{k}\right)\prime })+{\mathrm{W}}_{\mathrm{p}}^3\left({\mathrm{h}}^{(\mathrm{k}-1)})+{\mathrm{b}}_{\mathrm{f}}\right),$$

(21)

where $\mathrm{\sigma }$ denotes the activation function and ${\mathrm{b}}_{\mathrm{i}}$, ${\mathrm{b}}_{\mathrm{f}}$ denote the bias terms of the input and forgetting gates, respectively.

Secondly, update the memory unit ${\mathrm{c}}^{\left(\mathrm{k}\right)}$; this process is mainly divided into two steps. The first step is to discard the useless information in the previous stage through the output ${\mathrm{f}}^{\left(\mathrm{k}\right)}$ of the forgetting gate, and the second step is to multiply the information ${\mathrm{i}}^{\left(\mathrm{k}\right)}$ of the input gate and the result obtained from the tanh function to selectively add useful information, this process can be expressed as follows:

$${\mathrm{c}}^{\left(\mathrm{k}\right)}={\mathrm{i}}^{\left(\mathrm{k}\right)}\otimes \tan \mathrm{h}({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{x}}^{\left(\mathrm{k}\right)\prime })+{\mathrm{W}}_{\mathrm{p}}^3({\mathrm{h}}^{(\mathrm{k}-1)})+{\mathrm{b}}_{\mathrm{c}})+{\mathrm{f}}^{\left(\mathrm{k}\right)}\otimes {\mathrm{c}}^{(\mathrm{k}-1)},$$

(22)

where $b_c$ denotes the bias term of the memory cell.

Finally, the value of the output gate ${\mathrm{o}}^{\left(\mathrm{k}\right)}$ and the hidden state ${\mathrm{h}}^{\left(\mathrm{k}\right)}$ are calculated, and the memory cell ${\mathrm{c}}^{\left(\mathrm{k}\right)}$ is further extracted with information using the tanh function. Then, the extracted result is multiplied by ${\mathrm{o}}^{\left(\mathrm{k}\right)}$ to obtain the result of ${\mathrm{h}}^{\left(\mathrm{k}\right)}$, and finally, the information of ${\mathrm{h}}^{\left(\mathrm{k}\right)}$ and ${\mathrm{c}}^{\left(\mathrm{k}\right)}$ is combined to obtain as ${\mathrm{x}}^{\left(\mathrm{k}\right)}$. This process can be expressed as follows:

$${\mathrm{o}}^{\left(\mathrm{k}\right)}=\mathrm{\sigma }({\mathrm{W}}_{\mathrm{p}}^3({\mathrm{x}}^{\left(\mathrm{k}\right)\prime })+{\mathrm{W}}_{\mathrm{p}}^3\left({\mathrm{h}}^{(\mathrm{k}-1)})+{\mathrm{b}}_{\mathrm{o}}\right),$$

(23)

$${\mathrm{h}}^{\left(\mathrm{k}\right)}={\mathrm{o}}^{\left(\mathrm{k}\right)}\otimes \tan \mathrm{h}({\mathrm{c}}^{\left(\mathrm{k}\right)}),$$

(24)

$${\mathrm{x}}^{\left(\mathrm{k}\right)}={\mathrm{h}}^{\left(\mathrm{k}\right)}+{\mathrm{c}}^{\left(\mathrm{k}\right)},$$

(25)

where ${\mathrm{b}}_{\mathrm{o}}$ denotes the bias term of the output gate.

3.5. Loss Function

Given a set of fully sampled images ${\left\{{\mathrm{x}}_{\mathrm{i}}\right\}}_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{a}}}$ and some sampling patterns with the specific CS ratio $\mathrm{r}$, the training data ${\left\{{\mathrm{y}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{i}}\right\}}_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{a}}}$ are generated by ${\mathrm{y}}_{\mathrm{i}}=\mathrm{\Phi }{\mathrm{x}}_{\mathrm{i}}$, which will denote the input features ${\mathrm{x}}_{\mathrm{i}}$ and ${ \hat{\mathrm{x}}}_{\mathrm{i}}$ as well as denote the output reconstruction results. We adopt the L₁ loss function to compute the difference between ${\mathrm{x}}_{\mathrm{i}}$ and ${ \hat{\mathrm{x}}}_{\mathrm{i}}$, as shown below:

$$\mathcal{L}(\mathrm{\Theta })={\displaystyle \frac{1}{{\mathrm{N}}_{\mathrm{b}}{\mathrm{N}}_{\mathrm{a}}}}{\displaystyle \sum _{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{a}}}{‖{\mathrm{x}}_{\mathrm{i}}-{ \hat{\mathrm{x}}}_{\mathrm{i}}‖}_1},$$

(26)

where N_a and N_b denote the number of training images and the size of each image, respectively, and $\mathrm{\Theta }$ denotes the set of all learnable network parameters, $\mathrm{\Theta }={\left\{{\mathrm{\Phi }}_B,{\mathcal{H}}_{\mathrm{CAAGDM} }^{\left(\mathrm{k}\right)}(\cdot ),{\mathcal{H}}_{\mathrm{LDGDM}}^{\left(\mathrm{k}\right)}(\cdot ),{\mathcal{H}}_{\mathrm{DMTM}}^{\left(\mathrm{k}\right)}(\cdot )\right\}}_{\mathrm{K}=1}^{\mathrm{n}}$.

4. Experimental Results and Analysis

4.1. Implementaion Details

We use the BSD400 [36] and DIV2K [37] datasets as training datasets. To achieve data enhancement, we obtained better generalization by randomly cropping, panning, and rotating the training set, setting the default number of training batches to 1500 and the batch size to 8. The network was trained using the Adam [38] optimizer with the initial learning rate set to 5 × 10⁻⁴ and the gradient descent step size ${\rho }^{(k)}$ as a learnable linear scaling factor, which is initialized to 0.5 in this paper. Our test set uses three public benchmark datasets: Set11 [20], BSD68 [39], and Urban100 [40]. Color images are processed in the YCbCr space and evaluated on the luminance channel. The reconstructed image quality evaluation criteria use objective metrics: peak signal-to-noise ratio (PSNR) [41], structural similarity (SSIM) [42], and subjective visual judgments of the human eye, and the experimental environment is an Intel Core i7-13700KF CPU and an RTX4070Ti GPU.

4.2. Evaluation Metrics

PSNR is a classical objective evaluation metric, the higher the PSNR value, the smaller the difference between the reconstructed image and the original image, i.e., the higher the image quality. For images X and Y, both of size $\mathrm{m}\times \mathrm{n}$, the PSNR is computed as shown in Equation (27):

$$\mathrm{PSNR}=10\cdot {\log }_{10}({\displaystyle \frac{{\mathrm{MAX}}_{\mathrm{X}}^2}{\mathrm{MSE}}}),$$

(27)

where ${\mathrm{MAX}}_{\mathrm{X}}^2$ is the maximum possible pixel value of image X, and MSE denotes the mean square error between images X and Y.

SSIM is an objective evaluation metric used to measure the similarity between two images. The SSIM value ranges from 0 to 1, where the larger value indicates a higher similarity between the images. The SSIM between images X and Y is calculated according to Equation (28):

$$\mathrm{SSIM}(\mathrm{X},\mathrm{Y})={\displaystyle \frac{(2{\mu }_{\mathrm{X}}{\mu }_{\mathrm{Y}}+C_1)(2{\sigma }_{\mathrm{X}\mathrm{Y}}+C_2)}{({{\mu }_{\mathrm{X}}}^2+{{\mu }_{\mathrm{Y}}}^2+C_1)({{\sigma }_{\mathrm{X}}}^2+{{\sigma }_{\mathrm{Y}}}^2+C_2)}},$$

(28)

where ${\mu }_{\mathrm{X}}$ and ${\mu }_{\mathrm{Y}}$ represent the mean values of images X and $\mathrm{Y}$, while ${{\sigma }_{\mathrm{X}}}^2$ and ${{\sigma }_{\mathrm{Y}}}^2$ represent their variances. ${\sigma }_{\mathrm{X}\mathrm{Y}}$ denotes their covariance. Additionally, $C_1$ and $C_2$ are constant terms.

4.3. Comparison with State-of-the-Arts

To validate the performance of AMAUN-CS, we compare it with recent representative CS reconstruction methods including TVAL3 [16], ReconNet [20], DPA-Net [43], ISTA-Net⁺ [24], AMP-Net [27], OPINE-Net⁺ [25], COAST [44], TransCS [28], SODAS-Net [45], and DPC-DUN [46]. Among them, TVAL3 is based on traditional optimization methods, ReconNet and DPA-Net are deep neural network methods, and all other methods are optimization-inspired deep unfolding networks.

Table 1. Average PSNR (dB)/SSIM performance comparisons of recent deep network-based CS methods on Set11 dataset with different CS ratios. The best and second-best results are highlighted in bold and underlined, respectively.

Methods			CS Ratio
	4%	10%	25%	30%	40%	50%	Average
TVAL3	18.75/0.3997	22.99/0.4758	27.92/0.7238	29.00/0.7764	31.46/0.8531	33.55/0.8957	27.28/0.6874
ReconNet	20.93/0.5897	23.96/0.7172	26.38/0.7883	28.20/0.8424	30.02/0.8837	30.62/0.8983	26.68/0.7857
ISTA-Net+	21.32/0.6037	26.58/0.8066	32.48/0.9242	33.81/0.9393	36.04/0.9581	38.06/0.9706	31.37/0.8670
DPA-Net	23.50/0.7205	27.66/0.8530	32.38/0.9311	33.35/0.9425	35.21/0.9580	36.80/0.9685	31.48/0.8955
AMP-Net	25.26/0.7722	29.40/0.8779	34.63/0.9481	36.03/0.9586	38.28/0.9715	40.34/0.9804	33.99/0.9181
OPINE-Net+	25.69/0.7920	29.79/0.8905	34.81/0.9503	36.04/0.9600	37.96/0.9633	40.17/0.9797	34.07/0.9228
COAST	-/-	28.69/0.8618	32.54/0.9251	35.04/0.9501	37.13/0.9648	38.94/0.9744	-/-
TransCS	25.41/0.7883	29.54/0.8877	35.06/0.9548	35.62/0.9588	38.46/0.9737	40.49/0.9815	34.09/0.9241
SODAS-Net	24.69/0.7838	28.84/0.8665	34.24/0.9443	35.54/0.9545	37.72/0.9680	39.59/0.9769	33.43/0.9156
DPC-DUN	25.20/0.7710	29.40/0.8798	34.69/0.9482	35.88/0.9570	37.98/0.9694	39.84/0.9778	33.83/0.9172
AMAUN-CS	26.34/0.8089	30.50/0.9018	35.84/0.9596	37.08/0.9668	39.19/0.9767	41.00/0.9830	34.99/0.9328
AMAUN-CS+	26.52/0.8122	30.72/0.9045	36.02/0.9614	37.26/0.9696	39.30/0.9792	41.15/0.9859	35.16/0.9354

summarizes the PSNR/SSIM reconstruction metrics of different methods on the Set11 dataset, and the boldfaced font shows the optimal value of each row. From Table 1, it can be observed that the objective metrics of AMAUN-CS and AMAUN-CS⁺ proposed in this paper are overall better than the other methods on all datasets.

As shown in Table 1, the average PSNR of AMAUN-CS and AMAUN-CS⁺ is improved by 0.9 dB and 1.07 dB, respectively, and the average SSIM is improved by 0.0087 and 0.0113 compared with the best performance of the other methods, TransCS. In addition, in order to further validate the generalization of the method of this paper, this paper compares the other methods with AMAUN-CS on the Urban100 and BSD68 for comparison, as shown in

Table 2. Average PSNR (dB)/SSIM performance comparisons of recent deep network-based CS methods on Urban100 dataset with different CS ratios. The best and second-best results are highlighted in bold and underlined, respectively.

Methods			CS ratio
	4%	10%	25%	30%	40%	50%	Average
TVAL3	18.15/0.4291	22.31/0.4655	27.75/0.7628	28.84/0.8214	31.74/0.8846	34.64/0.9315	27.24/0.7158
ReconNet	20.37/0.5514	23.74/0.7044	26.25/0.8402	28.07/0.8929	30.62/0.9131	32.98/0.9394	27.01/0.8069
ISTA-Net+	19.66/0.5370	19.66/0.7238	28.93/0.8840	30.21/0.9079	32.43/0.9377	34.43/0.9571	27.55/0.8245
DPA-Net	21.64/0.6498	24.55/0.7841	28.80/0.8944	29.47/0.9034	31.09/0.9311	32.08/0.9447	27.93/0.8513
AMP-Net	21.89/0.6340	26.04/0.8151	30.89/0.9202	32.19/0.9365	34.37/0.9578	36.33/0.9712	30.28/0.8724
OPINE-Net+	23.00/0.7020	26.93/0.8397	31.86/0.9308	32.58/0.9414	33.88/0.9445	37.23/0.9741	30.91/0.8882
COAST	-/-	25.94/0.8035	31.10/0.9168	32.23/0.9321	34.22/0.9530	35.99/0.9665	-/-
TransCS	23.23/0.7018	26.72/0.8413	31.72/0.9330	31.95/0.9381	35.22/0.9648	37.20/0.9734	31.00/0.8920
SODAS-Net	22.69/0.6991	25.65/0.7739	31.49/0.9190	32.90/0.9376	35.09/0.9583	37.23/0.9736	30.84/0.8769
DPC-DUN	23.02/0.7023	26.99/0.8345	32.36/0.9323	33.53/0.9449	35.61/0.9624	37.52/0.9737	31.51/0.8916
AMAUN-CS	23.89/0.7354	27.98/0.8684	32.54/0.9450	33.30/0.9435	36.76/0.9729	37.42/0.9724	31.98/0.9062
AMAUN-CS+	24.04/0.7411	28.07/0.8710	32.75/0.9468	33.61/0.9457	37.01/0.9755	38.52/0.9784	32.33/0.9098

and

Table 3. Average PSNR (dB)/SSIM performance comparisons of recent deep network-based CS methods on BSD68 dataset with different CS ratios. The best and second-best results are highlighted in bold and underlined, respectively.

Methods			CS Ratio
	4%	10%	25%	30%	40%	50%	Average
TVAL3	17.86/0.4261	19.26/0.4758	21.27/0.7204	22.34/0.7764	25.39/0.8531	29.59/0.8957	22.62/0.6912
ReconNet	21.66/0.4994	23.88/0.6400	25.75/0.7317	26.72/0.7870	28.96/0.8499	30.13/0.8499	25.87/0.8499
ISTA-Net+	22.17/0.5486	25.32/0.7022	29.36/0.8525	30.20/0.8771	32.21/0.9321	34.04/0.9424	28.88/0.8095
DPA-Net	23.27/0.6096	25.02/0.6958	28.73/0.8814	29.93/0.8722	31.85/0.9128	33.60/0.9401	28.73/0.8186
AMP-Net	25.40/0.6985	27.86/0.7926	31.74/0.9048	32.84/0.9240	33.10/0.9383	35.02/0.9510	30.99/0.8682
OPINE-Net+	25.00/0.6825	27.82/0.8045	31.51/0.9061	32.35/0.9215	33.45/0.9412	36.47/0.9669	31.10/0.8704
COAST	-/-	26.28/0.7422	29.00/0.8413	31.06/0.8934	32.93/0.9267	34.74/0.9497	-/-
TransCS	25.28/0.6881	27.93/0.8141	31.84/0.9154	32.66/0.9303	34.94/0.9559	36.94/0.9711	31.59/0.8780
SODAS-Net	24.69/0.7838	26.59/0.7528	30.62/0.8818	31.69/0.9052	33.67/0.9360	35.54/0.9568	30.46/0.8694
DPC-DUN	24.90/0.7856	26.79/0.7611	30.71/0.8828	31.76/0.9051	33.70/0.9364	35.62/0.9573	30.58/0.8713
AMAUN-CS	25.67/0.7921	27.96/0.8132	31.92/0.9161	33.62/0.9399	35.68/0.9611	37.71/0.9748	32.09/0.8995
AMAUN-CS+	25.72/0.7945	28.10/0.8210	32.10/0.9213	34.20/0.9469	36.01/0.9630	37.85/0.9802	32.33/0.9044

, the proposed method in this paper still significantly outperforms the other methods in PSNR/SSIM. In the Urban100 dataset, AMAUN-CS achieves an average improvement of 0.47 dB/0.0146 over the best-performing method, DPC-DUN, while in the BSD68 dataset, AMAUN-CS demonstrates an average improvement of 0.5 dB/0.0215 over the top-performing TransCS. By incorporating long- and short-term memory operations, AMAUN-CS⁺ further enhances performance based on AMAUN-CS, with an additional improvement of 0.35 dB/0.0036 and 0.24 dB/0.0049, respectively.

In order to compare the differences in reconstruction results between AMAUN-CS and other methods based on subjective visualization, we present the following analysis. Figure 5 and Figure 6 show the reconstruction results of different methods at different observation rates. As shown in Figure 5, both ReconNet and ISTA-Net⁺ fail to reconstruct the image with complete details, resulting in a significant block effect. At the same time, TransCS and OPINE-Net⁺ make the reconstructed image blurrier and lose part of the texture information due to the lack of further utilization of the information between adjacent iterations.

In addition, Figure 6 gives a comparison of the reconstructed images of different methods on the color image Urban100. It can be seen that the other methods suffer from different degrees of block artifacts and blurring. Whereas the method proposed in this paper is more full in terms of texture details. It also recovers the structural information of the image better and makes the texture details more visible. In summary, the experimental results above demonstrate that the AMAUN-CS and AMAUN-CS+ methods proposed in this paper achieve advanced performance in both objective quality evaluation and subjective visual comparison. To assess the scalability of AMAUN-CS, a set of medical brain images was used to evaluate its performance on the CS-MRI reconstruction task. As shown in Figure 7, the experimental results demonstrate that our method can clearly reconstruct texture structures.

4.4. Complexity Analysis

The computation cost and model size are important in many practical applications.

Table 4. Comparison of the parameters, running time, and FLOPs for reconstructing the Set11 dataset in the case of CS ratio = 10%. The best performance is labeled in bold.

Method	Running Time(s)	Params(M)	FLOPs(G)
ISTA-Net+	0.0188	1.112	405.12
AMP-Net	0.0564	1.529	434.84
OPINE-Net+	0.0156	1.095	384.34
TransCS	0.0286	1.916	564.86
AMAUN-CS	0.0143	0.554	264.20
AMAUN-CS+	0.0172	1.103	396.70

compares the number of parameters and average running time of different methods in the 0.1 sampling rate and Set11 dataset. For the fairness of the comparison, only the algorithm complexity is compared with the open-source model on the same platform. As shown in Table 4, TransCS has a larger number of parameters and a relatively complex structure due to the dual-path network structure that combines CNN and Transformer. In contrast, the number of parameters of the network in this paper is less than that of other methods, and at the same time, it can maintain better reconstruction performance. It shows that the parameters of the model can be more fully utilized under the framework of this paper.

4.5. Ablation Analysis

4.5.1. Number of Optimization Phases of the Network

To explore the effect of different iteration numbers on the image reconstruction results, Figure 8 shows the reconstruction performance of AMAUN-CS obtained by training with different numbers of iterations under the Set11 dataset and a 10% sampling rate. The horizontal axis is the number of training iterations of the network, and the vertical axis is the average PSNR value of the dataset reconstruction. From the curve, we observe that the PSNR value increases with the number of iterations, but it tends to plateau when n ≥ 9. While larger iteration numbers might further improve performance slightly, they also lead to increased computational complexity and parameter overhead. Therefore, we selected nine as the default iteration number, considering the trade-off between reconstruction performance and network complexity.

4.5.2. Effectiveness of Different Modules

In order to validate the effectiveness of our proposed different network modules in AMAUN-CS, this section performs ablation experiments on three modules at a 10% sampling rate. As shown in

Table 5. Ablation experiments of different modules on the Set11 dataset under a 50% sampling rate. The best PSNR (dB) is labeled in bold.

Cases	CAAGDM	LDGDM	DMTM	PSNR/SSIM
(a)	×	×	×	38.06/0.9706
(b)	√	×	×	40.12/0.9781
(c)	×	√	×	39.54/0.9742
(d)	√	√	×	41.00/0.9830
(e)	√	√	√	41.15/0.9859

, case (a) serves as our baseline model, with a simple CNN replacing our proposed module. In cases (b) and (c), CAAGDM and LDGDM are introduced alone, respectively, and gains of 2.06 dB/0.0075 and 1.48 dB/0.0036 are obtained compared to the baseline model, indicating the limited effect of LDGDM alone. Case (d) shows that when both sub-modules are used simultaneously, a significant improvement of 2.94 dB/0.0124 is achieved compared to the baseline model. Case (e) shows that the introduction of DMTM brings a further improvement of 3.09 dB/0.0153, which is good proof of its transfer of information across iterations.

In summary, the application of CAAGDM, LDGDM, and DMTM has a significant positive effect on improving the reconstruction of the baseline model.

4.6. Sensitivity to Noise

To evaluate the robustness of the reconstructed images under different noise levels, we first added different levels of Gaussian noise to the original images of the Set11 dataset. AMAUN-CS and other methods were then sampled and reconstructed with the noisy images as inputs at 10% and 25% observation rates. Figure 9 illustrates the PSNR relationship of the various methods for different noise standard deviations. It can be seen that the image reconstruction quality of ISTA-Net is significantly affected and only slightly improved even at higher observation rates. The other methods are also affected by different degrees of noise interference at low observation rates. In contrast, the proposed method shows strong robustness under different noise levels.

5. Conclusions

In this paper, an adaptive memory-augmented unfolding network for compressed sensing was proposed. Specifically, the model effectively combines a proximal gradient descent algorithm with a deep neural network and designs a deep reconstruction sub-network with multiple phases to further refine the initial reconstructed image, with each optimization stage corresponding to one iteration of the traditional optimization algorithm. Meanwhile, oriented to lightweight, we then construct two learning modules, CAAGDM and LDGDM, to enhance the flexibility of the network, which effectively reduces the influence of different noises in network learning. In addition, we propose a deep memory transfer module to enhance the transfer and interaction of information in the optimization iteration phase. In extensive experiments on several benchmark datasets, we demonstrate its significant advantages over other state-of-the-art networks. In the future, we plan to extend AMAUN-CS to video transmission applications.