Non-Iterative Recovery Information Procedure with Database Inspired in Hopfield Neural Networks

Abstract

This work establishes a simple algorithm to recover an information vector from a predefined database available every time. It is considered that the information analyzed may be incomplete, damaged, or corrupted. This algorithm is inspired by Hopfield Neural Networks (HNN), which allows the recursive reconstruction of an information vector through an energy-minimizing optimal process, but this paper presents a procedure that generates results in a single iteration. Images have been chosen for the information recovery application to build the vector information. In addition, a filter is added to the algorithm to focus on the most important information when reconstructing data, allowing it to work with damaged or incomplete vectors, even without losing the ability to be a non-iterative process. A brief theoretical introduction and a numerical validation for recovery information are shown with an example of a database containing 40 images.

1. Introduction

Hopfield neural networks (HNN) are a tool with a simple recurrent structure and with an efficient associative memory in which a search and recognition of patterns are performed [1], where the search for patterns is obtained by minimizing the energy, which can be obtained as long as there is a change in the input [2]. In addition, these networks can be trained in continuous time in a deterministic way representing Amari’s gradient [3], and it is known that it can solve combinatorial optimization problems. However, it is only sure that convergence to a local minimum is obtained if the problem can be written in terms of an energy function [4]. Applications of this type of data-driven neural networks are found in various sectors, such as medicine [5], in robotics solving the traveling salesman problem (TSP) [6], for mathematical modeling based on pattern recognition and system knowledge [7]. Reference [8] shows an approach to performing image segmentation using Hopfield neural networks applying constraints on the edges of the figure, where a probability-based pixel difference criterion is performed, and the neural network automatically adapts this probability to find optimal or suboptimal segmentation results. For the Hopfield neural network, in the computational sense, ideas for solving weighted constraint problems continue to appear [9]. Some of the applications for HNN also have comparisons with other types of tools, such as artificial immune systems (AIS), from which ideas are derived to solve computational problems that generate an optimal result with constraints such as obtaining Boolean satisfiability using the MAX-kSAT problem [10,11]. Also, it can be possible to determine the stability of HNN networks and perform the segmentation of objects in images [12]. However, when the image has multi-modal information, it is more complicated to get the results, so they apply a neural network to measure the threshold levels in the objects in gray scales in an adaptive way [13].

For the training of Hopfield neural networks, alternatives have been developed for the minimization of the energy or generate structure additions to have improvements, such as the change of the search for an optimum using the Moore-Penrose pseudo inverse [14], or even appearing alternative neural networks such as oscillatory neural networks (ONN) that can work as self-associative memory networks that have comparative with Hopfield neural networks due to their operation [15]. In the case of an application for images, it is also possible to change the function to minimize Hopfield neural networks to obtain the limits of the images while minimizing this function [16]. Knowing which neurons have much or little noise, it is possible to reinforce the neurons with little noise to improve the tolerance to this [17]. Also, ways to train neural networks can be found in [18], where the training idea is presented for feed-forward networks based on energy minimization with two steps: unsupervised learning in the first step to obtain the weights that relate the inputs with the neurons and network parameters, and the second step to perform the backpropagation of the internal and external energy to the network structure. However, all this is iterative to achieve the goal.

With the evolution of the Hopfield neural networks, it has been attempted to eliminate the information that is not part of the data, such as noise in the transmission of messages [19]. They are also being used to recover a set of correct data from a database when having corrupted information, with an approximate or compressed associative memory matrix [20]. In addition, implementing these neural networks using a Field Programmable Gate Array (FPGA) has been achieved without losing the advantages that the neural network provides, such as eliminating noise in the information [21]. There are even applications of this type of neural network using memristors, with good robustness against memristive variations and noises in the input, for restoring images [22].

In this work, we seek to implement algorithms for data recovery, specifically images, using non-iterative processes and compare it with information retrieval using the HNN network: (1) propose the theoretical background for the implementation of a non-iterative process for information recovery where a metric is established between actual data and data in an image database to determine similarity; (2) establish a methodology that achieves information recovery from a database using an unsupervised HNN; (3) create an algorithm that generates information recovery in a single iteration; and (4) present an extension of the algorithm to weighted information vectors.

The structure of this document is as follows: Section 2 shows a mathematical background and basic vector operations as well as the mathematical ideas of Hopfield neural networks. Section 3 contains the ideas for applying the HNN and the author’s algorithms presented for this work. Section 4 contains an image data-driven example for information retrieval using the algorithms proposed by the authors. Section 5 and Section 6 present the discussion and conclusions for this work, respectively.

2. Materials and Methods

2.1. Theoretical Background

In the computational field, vectors can be called database elements because they represent a data set as an array. These vectors have the following definitions and properties:

Definition 1. 

Consider a non-empty finite set of column vectors ${\mathcal{B}}_{n,m}:=\left\{X_i\in {\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m\right.$$\left.:i\in ⟦1,n⟧\right\}$, with $n\ge 2$ and same dimension, number of positive data as input, and form of non-proportional even vectors can form a database for recovery information.

This must not represent a basis of a vector subspace; moreover, ${\mathbb{R}}^{{+}^m}$ is not a vector space. It is only a set in which constant information can be stored. In addition, for this case, $0\notin {\mathcal{B}}_{n,m}$ with which it is possible to construct the database matrix as follows:

$$X=\left[\begin{array}{ccccc}{X}_1& X_2& X_3& \dots & X_n\end{array}\right]$$

(1)

Considering the inner product of two vectors $U,V\in \mathcal{V}\subseteq {\mathbb{R}}^m$ as $〈U,V〉:=U^{⊺}V$ and to each vector $V\in \mathcal{V}\subseteq {\mathbb{R}}^m$ can be applied the Euclidean norm with the form $∥V∥:=\sqrt{〈V,V〉}$. The Euclidean norm represents an unbounded function, so applying information theory would be better if a bounded norm were used to induce a finite metric to compare the similarity of two vectors.

Lemma 1. 

Consider $x,y\in {\mathbb{R}}^{+}$ and $q\in \mathbb{N}$ with $q>2$, then

$${\left|x+y\right|}^{{\displaystyle \frac{2}{q}}}\le {\left|x\right|}^{{\displaystyle \frac{2}{q}}}+{\left|y\right|}^{{\displaystyle \frac{2}{q}}}$$

(2)

Proof. 

Consider the math operations:

$$\begin{array}{cc}\hfill {\left|x+y\right|}^{{\displaystyle \frac{2}{q}}}& =\left(x+y\right){\displaystyle \frac{{\left|x+y\right|}^{\frac{2}{q}}}{x+y}}\hfill \\ \hfill & =x{\displaystyle \frac{1}{{\left|x+y\right|}^{1-\frac{2}{q}}}}+y{\displaystyle \frac{1}{{\left|x+y\right|}^{1-\frac{2}{q}}}}\hfill \\ \hfill & \le x{\displaystyle \frac{1}{{\left|x\right|}^{1-\frac{2}{q}}}}+y{\displaystyle \frac{1}{{\left|y\right|}^{1-\frac{2}{q}}}}\hfill \\ \hfill & ={\left|x\right|}^{{\displaystyle \frac{2}{q}}}+{\left|y\right|}^{{\displaystyle \frac{2}{q}}}\hfill \end{array}$$

(3)

   □

Proposition 1. 

For two vectors $U,V\in \mathcal{V}\subseteq {\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m$ ($\mathcal{V}$ is not a vector subspace) the function $d\left(·,·\right):\mathcal{V}\times \mathcal{V}\to \left[0,1\right]$ given by

$$d\left(U,V\right):=\left\{\begin{array}{cc}1\hfill & \mathit{only}\mathit{one}\mathit{vector}\mathit{is}\mathit{zero}\hfill \\ 0\hfill & U=V=0\hfill \\ {\left|1-{\displaystyle \frac{〈U,V〉}{∥U∥∥V∥}}\right|}^{{\displaystyle \frac{1}{q}}}\hfill & \mathit{otherwise}\hfill \end{array}\right.$$

(4)

This represents an induced bounded metric.

Proof. 

By construction, the following is known $d\left(0,0\right)=0$, adding the inner product properties it follows that $d\left(U,V\right)=d(V,U)$ and due to the Cauchy–Schwarz inequality if $U\ne V$ implies $d\left(U,V\right)>0$, then note that

$$\begin{array}{cc}\hfill {\left|1-{\displaystyle \frac{〈U,V〉}{∥U∥∥V∥}}\right|}^{{\displaystyle \frac{1}{q}}}& ={\displaystyle \frac{1}{\sqrt[q]{2}}}{\left|2-2{\displaystyle \frac{〈U,V〉}{∥U∥∥V∥}}\right|}^{{\displaystyle \frac{1}{q}}}\hfill \\ \hfill & ={\displaystyle \frac{1}{\sqrt[q]{2}}}{\left|{\displaystyle \frac{〈U,U〉}{{∥U∥}^2}}+{\displaystyle \frac{〈V,V〉}{{∥V∥}^2}}-2{\displaystyle \frac{〈U,V〉}{∥U∥∥V∥}}\right|}^{{\displaystyle \frac{1}{q}}}\hfill \\ \hfill & ={\displaystyle \frac{1}{\sqrt[q]{2}}}{\left|〈{\displaystyle \frac{U}{∥U∥}}-{\displaystyle \frac{V}{∥V∥}},{\displaystyle \frac{U}{∥U∥}}-{\displaystyle \frac{V}{∥V∥}}〉\right|}^{{\displaystyle \frac{1}{q}}}\hfill \\ \hfill & ={\displaystyle \frac{1}{\sqrt[q]{2}}}{\left|∥{\displaystyle \frac{U}{∥U∥}}-{\displaystyle \frac{V}{∥V∥}}∥\right|}^{{\displaystyle \frac{2}{q}}}\hfill \\ \hfill & ={\displaystyle \frac{1}{\sqrt[q]{2}}}{\left|d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{V}{∥V∥}}\right)\right|}^{{\displaystyle \frac{2}{q}}}\hfill \end{array}$$

(5)

where $d_E\left(·,·\right)$ is the Euclidean metric. Consider that the function $f\left(x\right):=2^{-{\displaystyle \frac{1}{q}}}{\left|x\right|}^{{\displaystyle \frac{2}{q}}}$ with $x\ge 0$ and $q\in \mathbb{N}$ with $q>2$ satisfies $f\left(0\right)=0$, if $0<x<y$ clearly $f\left(x\right)<f\left(y\right)$ and by Lemma 1 $f\left(x+y\right)\le f\left(x\right)+f\left(y\right)$ then by properties of metric $d_E\left(·,·\right)$ is:

$$d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{V}{∥V∥}}\right)\le d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{Z}{∥Z∥}}\right)+d_E\left({\displaystyle \frac{Z}{∥Z∥}},{\displaystyle \frac{V}{∥V∥}}\right)$$

(6)

and applying $f\left(·\right)$ yields:

$$\begin{array}{cc}\hfill d\left(U,V\right)& =f\left(d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{V}{∥V∥}}\right)\right)\hfill \\ \hfill & \le f\left(d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{Z}{∥Z∥}}\right)+d_E\left({\displaystyle \frac{Z}{∥Z∥}},{\displaystyle \frac{V}{∥V∥}}\right)\right)\hfill \\ \hfill & \le f\left(d_E\left({\displaystyle \frac{U}{∥U∥}},{\displaystyle \frac{Z}{∥Z∥}}\right)\right)+f\left(d_E\left({\displaystyle \frac{Z}{∥Z∥}},{\displaystyle \frac{V}{∥V∥}}\right)\right)\hfill \\ \hfill & =d\left(U,Z\right)+d\left(Z,V\right)\hfill \end{array}$$

(7)

where $Z\in \mathcal{V}$ is a non-zero vector. Finally, this metric is bounded by the Cauchy–Schwarz inequality.    □

This metric is suitable for data recovery applications to quantify the similarity of two information vectors. For choosing q, it is considered that a significant value better detects subtle differences in data vectors but increases the computational error metric due to computational limitations.

2.2. Hopfield Neural Network as Associative Memory

The main idea of unsupervised neural network algorithms is to train or test with the correlation between vectors or matrices of information to minimize an energy function. For a database ${\mathcal{B}}_{N,m}$ with matrix X and a unit column vector $\xi \in {\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m$, in which the information may be missing or corrupted, is required to recover the correct information column vector of the database starting with the value in $\xi$. To solve this problem, the authors in [23] found an interesting energy functional as follows:

$$E\left(\xi \right):=\underset{E_2\left(\xi \right)\mathrm{concave}\mathrm{function}}{\underbrace{-{\beta }^{-1}\mathrm{lse}\left(\beta ,X^{⊺}\xi \right)}}+\underset{E_1\left(\xi \right)\mathrm{convex}\mathrm{function}}{\underbrace{{\displaystyle \frac{1}{2}}{\xi }^{⊺}\xi +{\beta }^{-1}logN+{\displaystyle \frac{1}{2}}{M}^2}}$$

(8)

where M is the most significant Euclidean norm of all stored patterns and:

$$\mathrm{lse}\left(\beta ,z\right):={\beta }^{-1}log\left(\sum _{i=1}^{N}exp\left(z_i\right)\right)$$

(9)

with $\beta >0$ and $z:={\left[\begin{array}{cccc}{z}_1& z_2& \dots & z_N\end{array}\right]}^{⊺}$. It is necessary to find $\xi$ as a value that minimizes Equation (8). The functional has concave and convex terms in which the Concave-Convex Procedure (CCCP, see [24]) is applied:

$${\nabla }_{\xi }{E}_1\left({\xi }^{\left(k+1\right)}\right)=-{\nabla }_{\xi }{E}_2\left({\xi }^{\left(k\right)}\right)$$

(10)

where $k\in \mathbb{N}$ is the iteration number. Substituting yields:

$${\nabla }_{\xi }\left({\displaystyle \frac{1}{2}}{\xi }^{⊺}\xi +{\beta }^{-1}logN+{\displaystyle \frac{1}{2}}{M}^2\right)\left({\xi }^{\left(k+1\right)}\right)={\nabla }_{\xi }\left({\beta }^{-1}\mathrm{lse}\left(\beta ,X^{⊺}\xi \right)\right)\left({\xi }^{\left(k\right)}\right)$$

(11)

taking the gradients:

$${\xi }^{\left(k+1\right)}=X\mathrm{softmax}\left(\beta X^{⊺}{\xi }^{\left(k\right)}\right)$$

(12)

where the Equation (12) generates information for each iteration in the following form:

$$\mathrm{softmax}\left(z\right):={\left[\begin{array}{cccc}{\mathrm{softmax}}_1\left(z\right)& {\mathrm{softmax}}_2\left(z\right)& \dots & {\mathrm{softmax}}_N\left(z\right)\end{array}\right]}^{⊺}$$

(13)

with the calculation of the softmax function as follows.

$${\mathrm{softmax}}_j\left(z\right):={\displaystyle \frac{exp\left(z_j\right)}{{\sum }_{i=1}^{N}exp\left(z_i\right)}}$$

(14)

for every $j\in ⟦1,N⟧$.

     For procedure in Equation (12) choice $\beta$ large enough (see [24]), this is important to obtain a fast convergence.

The recovery of the data by the HNN network can be obtained by the Algorithm 1, which follows the idea presented in [23]. This algorithm obtains energy minimization (see Theorem 3 in [24]). For this, construct the matrix X with the vector ${\mathcal{B}}_{n,m}$ for which we calculate recursively r times the value of $\xi$, and at the end, we select the value of $\xi$ that minimizes the Equation (8).

Algorithm 1 Information recovery with HNN-based method.

Input:  Set of information normalized vectors ${\mathcal{B}}_{n,m}={\left\{X_i\right\}}_{i=1}^N$, $\xi$ a corrupted information normalized vector, r iterations.      1: Compute $X=\left[\begin{array}{ccccc}{X}_1& X_2& X_3& \cdots & X_N\end{array}\right]$.      for k=1:r do            $\xi =X\mathtt{softmax}\left(\beta X^{⊺}\xi \right)$      end for Output:  $\xi$.

2.3. Interpretation

The procedure in Equation (12) represents a correlation vector with a maximum approximation having a softmax function produced by the inner products with elements of ${\mathcal{B}}_{n,m}$ and $\xi$, i.e.:

$$X^{⊺}{\xi }^{\left(k\right)}={\left[\begin{array}{cccc}〈X_1,{\xi }^{\left(k\right)}〉& 〈X_2,{\xi }^{\left(k\right)}〉& \dots & 〈X_N,{\xi }^{\left(k\right)}〉\end{array}\right]}^{⊺}$$

(15)

Then, the softmax function returns a normalized discrete distribution of the inner products, and ${\xi }^{\left(k+1\right)}$ results in a linear combination of elements of ${\mathcal{B}}_{n,m}$ with the closest component to ${\xi }^{\left(k\right)}$. In every iteration, this distribution tends to a specific element or oscillates around a set of vectors that minimize Equation (8) (see [24]).

Proposition 2. 

The Algorithm 1 minimizes the metric $d\left(X_i,{\xi }^{\ast }\right)$ for every $X_i\in {\mathcal{B}}_{n,m}$.

Proof. 

An important property of the softmax function is

$$\sum _{j=1}^N\mathrm{softmax}\left(z_j\right)=1$$

(16)

for $z_i<z_j$ implies $\mathrm{softmax}\left(z_i\right)<\mathrm{softmax}\left(z_j\right)$, then, in every iteration, k obtains a linear convex combination. Consider the identity of the following equation:

$${\xi }^{\left(k+1\right)}=\sum _{i=1}^N{a}_i^{\left(k\right)}{X}_i$$

(17)

with ${\sum }_{i=1}^N{a}_i^{\left(k\right)}=1$ where $a_i^{\left(k\right)}:={\mathrm{softmax}}_i\left(\beta X^{⊺}{\xi }^{\left(k\right)}\right)$ and consider $a_r^{\left(k\right)}:={max}_i\left\{a_i^{\left(k\right)}:i\in ⟦1,N⟧\right\}$ by the properties, the index r is choose like the maximum in the set $\left\{\beta 〈X_i,{\xi }^{\left(k\right)}〉:i\in ⟦1,N⟧\right\}$ for a fixed $\beta >0$, by properties of a metric and Cauchy–Schwarz inequality implies minimizing the set ${\mathcal{A}}^{\left(k\right)}:=\{d\left(X_i,{\xi }^{\left(k\right)}\right):i\in ⟦1,N⟧\}$ for every k iteration then the distribution ${\left\{a_i^{\left(k\right)}\right\}}_{i=1}^N$ converges or oscillates and $a_r^{\left(k\right)}$ converges or oscillates around minimum points of ${\mathcal{A}}^{\left(k\right)}$ (see [24]).    □

A necessary consequence of this procedure is that the convergence does not guarantee a closed solution of the minimization problem because it is not necessarily $d(X_i,{\xi }^{\left(k\right)})=0$ for $i\ne r$ then $a_i$ is not necessarily zero.

3. Proposed Procedure

Inspired by the interpretation of the Algorithm 1, it is possible to generate a method containing a simple procedure for information recovery. The authors propose algorithms that generate information recovery in only one iteration as follows:

$${\xi }^{\ast }=X_r$$

(18)

where $r:=\left\{j:{min}_i\left\{d\left(X_i,\xi \right)\right\},i\in ⟦1,N⟧\right\}$.

     Equation (18) represents the non-iterative methodology of information recovery by simultaneously comparing the actual information with the whole database. This methodology may have fewer computational operations compared to the Algorithm 1.

Proposition 3. 

For a database ${\mathcal{B}}_{n,m}$ and a normalized column vector $\xi \in {\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m$ the method in Equation (18) minimizes the metric $d\left(X_i,{\xi }^{\ast }\right)$ for every $X_i\in {\mathcal{B}}_{n,m}$.

The construction of the procedure using the Algorithm 2 allows obtaining proof of this proposition and does not require further considerations. It is essential to mention that this minimum is not necessarily unique.

The procedure compares the actual information with the information contained in the database using a metric as stated in the Equation (4), and at the end, r is selected, which includes the information from the database that has the slightest difference between the data. These steps can follow as:

Algorithm 2 Recovery Procedure (RP).

Input:  Set of information vectors ${\mathcal{B}}_{n,m}={\left\{X_i\right\}}_{i=1}^N$, $\xi$ a corrupted information vector.   1: Compute the set $\left\{d\left(X_i,\xi \right):X_i\in {\mathcal{B}}_{n,m}\right\}$.   2: Select an index r that minimizes the set in Step 1. Output:  $X_r$.

3.1. Capacity of Reconstruction

Definition 2. 

The capacity of reconstruction of ${\mathcal{B}}_{n,m}$ is defined as:

$$\begin{array}{c}\mathfrak{C}\left({\mathcal{B}}_{n,m}\right):=\\ min\left\{d\left(X_i,X_j\right):X_i,X_j\in {\mathcal{B}}_{n,m};i\ne j;i,j\in ⟦1,n⟧\right\}\end{array}$$

(19)

A brief interpretation is that consider the worst case of confusion to select the correct index r in Algorithm 1 or Algorithm 2 for some $\xi$. That is $\xi =X^{\ast }+\Delta ϵ$ where $X^{\ast }\in {\mathcal{B}}_{n,m}$ is the correct vector information and $\Delta ϵ$ is noise or data corruption. If $\Delta ϵ$ is such that $\xi$ is closest to another information vector $X^{\ast \ast }\in {\mathcal{B}}_{n,m}$, then the algorithms return the incorrect recovery data vector.

If the capacity of reconstruction is small, then there exists a the possibility that some corrupted information vectors cannot be reconstructed adequately.

3.2. Extension to Weighted Information Recovery Procedure

To improve the idea presented in the previous algorithms, the authors propose adding a filter with multiplicative properties applied to specific information sections to focus on certain areas of interest, e.g., parts of the information that have less damage or corruption.

For the application of the filter in the information, we have the following vector properties:

Definition 3. 

For $U,V\in {\mathbb{R}}^m$ the Hadamard product $U\circ V:{\mathbb{R}}^m\times {\mathbb{R}}^m\to {\mathbb{R}}^m$ is defined as:

$${\left[U\circ V\right]}_i:=u_i{v}_i$$

(20)

where U and V are vectors with information of the following form:

$$\begin{array}{cc}\hfill U& :={\left[\begin{array}{cccc}{u}_1& u_2& \dots & u_m\end{array}\right]}^{⊺},\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill V& :={\left[\begin{array}{cccc}{v}_1& v_2& \dots & v_m\end{array}\right]}^{⊺}\hfill \end{array}$$

(22)

The operation in Equation (20) represents the classic element-by-element product in vectors.

Definition 4. 

The operation $\mathcal{F}\circ U$ filter information from vector $U\in {\mathbb{R}}^m$ where $\mathcal{F}\in {\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m$, indicating that it is weighted filter for U. In extension to a database, defines ${\mathcal{FB}}_{n,m}:=\mathcal{F}\circ {\mathcal{B}}_{n,m}$ as a weighted filtered database.

These filters highlight the relevant information in the vector to be retrieved and ignore the terms that are not of great interest. Radial Gaussian functions and Heaviside functions, among others, can also give these filters. For example, Figure 1 presents on the left an image that has gone through a process to convert it into vector information; in the center of the figure is the figure when a Gaussian filter is applied, and on the right is the same image with the application of a Heaviside filter.

The extension of the Algorithm 2 to which the passing of the information through a filter is added can be performed with the following steps:

The Algorithm 3 has the information as an array of vectors, and by applying the filter as established in Definition 4 generates vectors with specific details of the actual data to compare with the available database, step 2 calculate the difference between the actual data and the data in the database using the metric proposed in the Equation (4) and finally, selected r where the slightest between data difference is yielded in step 3.

Algorithm 3 Weighted Recovery Procedure (WRP).

Input:  Set of information vectors ${\mathcal{B}}_{n,m}={\left\{X_i\right\}}_{i=1}^N$, $\xi$ a corrupted information vector and $\mathcal{F}$ a weighted filter.   1: Compute ${\mathcal{FB}}_{n,m}$ and $\varsigma =\mathcal{F}\circ \xi$.   2: Compute the set $\left\{d\left(Y_i,\varsigma \right):Y_i\in {\mathcal{FB}}_{n,m}\right\}$.   3: Select an index r that minimizes the set in Step 1. Output:  $X_r$.

4. Results

Numerical Example

This section will generate information recovery by constructing vectors from image data. The algorithms proposed earlier have been applied to image recovery since the database includes images. For this purpose, consider a database that contains 40 grayscale images, each with a size of $200\times 200$ pixels. The image database is available in [25] and is illustrated in Figure 2.

In this database, each image is a matrix where a reshape is applied to convert the information into vectors that are found in ${\left[{\mathbb{R}}^{+}\cup \left\{0\right\}\right]}^m$ with m = 40,000.

Image recovery is then applied using the algorithm presented for HNN. At the top of the Figure 3 are the images containing corrupted or damaged information, while at the bottom is the recovery of the image containing the filter using Algorithm 2. Note that the reconstruction is efficient except for the last image, where an image that does not belong to the database was intentionally introduced. Then, the algorithm takes the information that most closely represents the image from the information that does belong to the database.

With the results obtained in Figure 3 generate the

Table 1. Brief statistics comparing the algorithms.

Algorithm Used	Image 1	Image 2	Image 3	Image 4
Corrupted Data with Initial Metric	0.854	0.848	0.940	0.759
Algorithm 1 with Final Metric (1 iteration)	0.372	0.261	0.143	0.588
Algorithm 1 with Final Metric (10 iterations)	0.092	0.175	0.025	0.113
Algorithm 2 with Final Metric (single iteration)	0.000	0.000	0.000	0.759

that compares the algorithms proposed in this work, where the metric with $q=10$ and $\beta =200$ is used. Two different numbers of iterations have been considered for the Algorithm 1, but there is only one iteration for applying the Algorithm 2. Table 1 correlates the corrupted image data and the image data in the database. In the first row, where this correlation shows results close to zero, it marks a closeness between the images. However, in this way, it cannot determine who has more accuracy because they all have high correlations. Because of this, the norm in the Equation (4) is used to know who has more closeness between the data of the actual image and the data in the database, obtaining in the second row the closeness between the pictures when one iteration is performed in the algorithm for HNN. Also, the third row shows the closeness of results when there are more iterations, so the results are closer to zero. Finally, in the last row, the closeness between images is determined using Algorithm 2, and the correct image is obtained in a single iteration because the difference is zero.

The metric compares between the correct and recovered images, except for image 4, where the comparison is with the closest image in the database. Note that for the first three images, iterating with the Algorithm 1 converges to zero, but the proposed Algorithm 2 achieves convergence in a single iteration where it reaches the selection of the correct image considering the database. For image four, the algorithms attempt to recover the house in the database, not the girl and the teddy bear, so the final metric is non-zero in the proposed algorithm. This property reveals that the algorithm can correctly recover the information if there is information in the database.

In example 2, we consider a given corrupted image with different Gaussian noise levels, as shown in the first row of Figure 4, where it has randomly uniform degradation contamination set to zero oscillating with $19\%$, $38\%$, $57\%$, $76\%$, and $95\%$ of corrupted information from left to right, respectively. These percentages mark the probability that a pixel is corrupted or not. Applying Algorithm 2 calculates the metric between corrupted information and the information in the database, where the result is the nearest image to the actual data. The results using Algorithm 2 are shown in the second row of Figure 4. A Heaviside filter is applied for the corrupted image, as shown in Figure 5, together with Algorithm 3 to work with the essential information of each image, generating the third-row results in Figure 4.

It is clarified that the filter should be applied where the information is most relevant according to the context of the importance of the data.

Table 2. Comparison of the Algorithms 2 and 3.

Algorithm	Image	Image	Image	Image	Image
Used	1	2	3	4	5
Percentage probability of pixel degradation	19%	38%	57%	76%	95%
Algorithm 2 with Final Metric (1 iteration)	0.00	0.00	0.00	0.00	0.64
Algorithm 3 with Final Metric (1 iteration)	0.00	0.00	0.00	0.00	0.00

shows a comparison of the recovery of the images by the algorithms proposed by the authors in this work, where the results of recovery of the information are observed, although it has different levels of damage in the original data, without forgetting that this is achieved by only one iteration in its operation and reviewing the metric dictates that the difference tends to zero. The first row of the Table 2 contains information on the probability of damage contained in the pixels in each image. In contrast, rows two and three show the difference between the real data and the information in the database, so it can be determined that the trend toward zero value means the actual information is recovered. However, in the last column, it can be observed that when a filter is applied while the algorithm is being performed, it improves the image recovery capacity, generating the correct image, while when the filter has not applied, the algorithm gives the closest information as it can be observed in the Figure 4.

5. Discussion

The application of the algorithms for the information recovery procedure by a database proposed in this work manages to recover the data without the need to perform iterative computational operations. Applying the Equation (18), fewer computational operations have been performed compared with the procedure using HNN, where, although the recovery of the data can be obtained, several steps (iterations) are required to have convergence. In applying the non-iterative methodology, it is possible to visualize that good information recovery is achieved, even if it is damaged or corrupted, so the application of the algorithm is still good, even if it is only one or a few steps to obtain that information. It is also worth mentioning that adding a filter to focus on a section of the data to be recovered does not generate more computational operations. It maintains the advantage of data recovery with a high correlation, so the procedure is still achieved in a single iteration.

Although the results obtained by the algorithms presented in this work are reasonable, there is a significant dependence on the database used to perform the information recovery, so it is possible to think of future work in applying generative information tools to reconstruct the corrupted data. Also, when a large part of the information is lost, it is possible not to obtain a good recovery of these by the algorithms, so it is possible to add tools that select the essential information of the data, such as the principal component analysis (PCA).

6. Conclusions

Hopfield’s neural networks as associative memories can recover information contained in a database, and this idea serves as the primary basis for the formation of the algorithms developed in this work. It is important to mention that the database can contain information of any type, such as audio or video or, as illustrated in this work, image information. With the analysis of the results obtained in the examples presented in Section 4, it is observed that it was possible to recover information from a database containing images in a single iteration, even when it contains corrupted information. Applying the Algorithm 3 shows that the procedure can focus on the critical information contained in the image by applying a filter, but this still gets results in a single iteration. Since the procedures proposed in this work are performed in a single iteration, fewer computational operations are present when executed, which may present an advantage against iterative algorithms. On the other hand, applying the metric presented in the Equation (4) can demonstrate whether there is a good convergence and shows more detail on the difference between the actual data and the data obtained in the information recovery.