Fast Sparse Coding for Range Data Denoising with Sparse Ridges Constraint

Abstract

Light detection and ranging (LiDAR) sensors have been widely deployed on intelligent systems such as unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) to perform localization, obstacle detection, and navigation tasks. Thus, research into range data processing with competitive performance in terms of both accuracy and efficiency has attracted increasing attention. Sparse coding has revolutionized signal processing and led to state-of-the-art performance in a variety of applications. However, dictionary learning, which plays the central role in sparse coding techniques, is computationally demanding, resulting in its limited applicability in real-time systems. In this study, we propose sparse coding algorithms with a fixed pre-learned ridge dictionary to realize range data denoising via leveraging the regularity of laser range measurements in man-made environments. Experiments on both synthesized data and real data demonstrate that our method obtains accuracy comparable to that of sophisticated sparse coding methods, but with much higher computational efficiency.

1. Introduction

In addition to their increasing proliferation and the growing importance of their roles in numerous applications (including 3D reconstruction and landscape surveying), light detection and ranging (LiDAR) sensors have recently been widely deployed on intelligent systems such as unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) to perform localization, obstacle detection, and navigation tasks. Consequently, research into range data processing with competitive performance in terms of both accuracy and efficiency has attracted increasing attention. Despite the simplicity and directness in obtaining 3D information, such range data often suffer from noise due to the reflectance property of an object’s surfaces, or due to electrical and mechanical disturbances. Therefore, efficient denoising for range data remains a critical problem.

Some range data denoising methods directly originate from their counterparts for images by regarding the depth value of range data as intensity, and readers can refer to [1] for detailed discussions. However, such methods typically cause artifacts of vertex drifting, which reduce the geometric regularity [2,3]. Techniques designed specifically for range data (e.g., the 3D points cloud) can be roughly classified into point-based [4,5] and mesh-based [6,7] methods. However, estimating normal vectors from noisy data for mesh denoising appears to be a chicken-and-egg problem and remains an open topic [8,9]. In [10,11], range data and color images were fused for denoising, and promising results were reported. However, the prerequisites of these methods only hold when expensive sensors are applied or extra efforts are devoted to registering range data and images.

Sparse coding (SC) has revolutionized signal processing and led to state-of-the-art performance in a variety of applications, including image or video denoising, inpainting [12,13], restoration [14], and synthesis [15], face recognition [16], and anomaly detection [17]. These successes are mainly due to the fact that signal (images or image patches) have naturally sparse representations with respect to appropriate bases [18]. In recent years, such SC techniques have been applied to range data [19,20] and obtained competitive results. However, dictionary learning, which plays the central role in SC techniques, is computationally demanding, resulting in its limited applicability in real-time systems [21,22].

Based on the work of [20], in which reflectance information was synergized with depth data to facilitate the enforcement of adaptive sparsity constraint (therein, the depth and reflectance information that are complementary to each other work together to estimate the informative level of each patch, followed by adaptive dictionary learning which assigns more atoms to represent the patches with more information), we propose SC algorithms without dictionary learning to realize range data denoising via leveraging the regularity of laser range measurements in man-made environments. Specifically, a second-order differential transformation is applied to extract ridges that can be reconstructed and enhanced using a pre-learned ridge dictionary with sparsity constraint. Instead of performing refinement on the coefficients and dictionary alternately and iteratively [12,14,19,20,22], our method can achieve a one-step closed-form solution, resulting in much improved efficiency. Experiments on both synthesized data and real data were conducted to verify the effectiveness of our method, together with performance comparison against other state-of-the-art methods.

2. Our Method

2.1. Notation and Preliminaries

Matrices and vectors are shown in bold capital and bold lower-cased fonts, respectively. Sets are denoted with calligraphic fonts (e.g., $\mathcal{S}$), and their cardinality is denoted as $\left|\mathcal{S}\right|$. For a vector $\mathit{x}\in {\mathbb{R}}^m$, $\mathit{x}\left[i\right]$ is the i-th entry, ${\mathit{x}}_{\mathcal{S}}$ is the subvector of $\mathit{x}$ corresponding to the entries with indices in $\mathcal{S}$. We define ${\ell }_2$-norm, ${\left|\right|\mathit{x}\left|\right|}_2={\sum }_{i=1,\cdots ,m}\mathit{x}{\left[i\right]}^2$; ${\ell }_1$-norm, ${\left|\right|\mathit{x}\left|\right|}_1={\sum }_{i=1,\cdots ,m}\left|\mathit{x}\left[i\right]\right|$; ${\ell }_0$-norm, ${\left|\right|\mathit{x}\left|\right|}_0=\left|supp(\mathit{x})\right|$ (i.e., the number of nonzero elements). $sign(\mathit{x})$ is a sign vector with entries: $sign(\mathit{x}[i])=1$ if $\mathit{x}\left[i\right]>0$, $sign(\mathit{x}[i])=-1$ if $\mathit{x}\left[i\right]<0$, and zero otherwise. Based on the sign operator, we define the shrinkage operator:

$$\begin{array}{c}\hfill {\mathcal{T}}_{\alpha }(\mathit{x})\left[i\right]={(|\mathit{x}\left[i\right]|-\alpha )}_{+}sign(\mathit{x}\left[i\right])=max(|\mathit{x}\left[i\right]|-\alpha ,0)sign(\mathit{x}\left[i\right]).\end{array}$$

For a matrix $\mathit{X}\in {\mathbb{R}}^{m\times n}$ and an index set $\mathcal{S}$, ${\mathit{X}}_{\mathcal{S}}$ is the submatrix containing only the rows of $\mathit{X}$ with indices in $\mathcal{S}$. $vec(\mathit{X})$ denotes the column-wise vectorization of matrix $\mathit{X}$. Letters d and r in superscript refer to the components related to the depth and reflectance information, respectively. Letters v and h in subscript refer to the components related to the vertical and horizontal directions, respectively.

Ridge detection for range measurements. We first take a scan from a 2D laser scanner as an example to introduce our operator of ridge detection and then extend the approach to 3D depth profiles. In Figure 1, we plot the depth profile $\mathit{z}\in {\mathbb{R}}^n$ of a typical indoor environment, which is obtained using a standard planar range finder measured with (known) discrete angles. Clearly, the profile is sufficiently regular and contains a few corners. Considering three consecutive points at coordinates $(x_{i-1},z_{i-1})$, $(x_i,z_i)$, and $(x_{i+1},z_{i+1})$, there is a corner at i if $\frac{z_{i+1}-z_i}{x_{i+1}-x_i}\ne \frac{z_i-z_{i-1}}{x_i-x_{i-1}}$. In the following, we assume that $x_i-x_{i-1}=1$ for all i (this assumption comes without loss of generality since we can define it at arbitrary resolution). Therefore, the corner detection for the 2D profile is to find those indices i such that $z_{i-1}-2z_i+z_{i+1}\ne 0$. To make the notation more compact, we introduce the second-order difference operator:

$${\mathit{D}}_{2nd}=\left[\begin{array}{cccccc}1& -2& 1& 0& \cdots & 0\\ 0& 1& -2& 1& \cdots & 0\\ ⋮& 0& \ddots & \ddots & \ddots & 0\\ 0& \cdots & 0& 1& -2& 1\end{array}\right]\in {\mathbb{R}}^{(n-2)\times n}.$$

(1)

Thus, a simple operation ${\mathit{D}}_{2nd}·\mathit{z}$ can detect corners. For a 3D depth profile $\mathit{Z}\in {\mathbb{R}}^{r\times c}$, we estimate the ridges of vertical and horizontal directions ${\mathit{R}}_v$ and ${\mathit{R}}_h$, respectively:

$$\begin{array}{c}\hfill {\mathit{R}}_v={\mathit{D}}_{v2nd}·\mathit{Z}\in {\mathbb{R}}^{(r-2)\times c},\\ \hfill {\mathit{R}}_h=\mathit{Z}·{\mathit{D}}_{h2nd}^T\in {\mathbb{R}}^{r\times (c-2)},\end{array}$$

(2)

where the matrices ${\mathit{D}}_{v2nd}$ and ${\mathit{D}}_{h2nd}$ are the same as ${\mathit{D}}_{2nd}$ in Equation (1), but with suitable dimensions. To combine the two equations of (2), we reformulate it as below:

$$\left[\begin{array}{c}vec({\mathit{R}}_h)\\ vec({\mathit{R}}_v)\end{array}\right]=\left[\begin{array}{c}{\mathit{I}}_c\otimes {\mathit{D}}_{v2nd}\\ {\mathit{D}}_{h2nd}\otimes {\mathit{I}}_r\end{array}\right]·vec(\mathit{Z})={\mathit{D}}_{vh}·vec(\mathit{Z}).$$

(3)

Here, ⊗ is the Kronecker product, and ${\mathit{D}}_{vh}\in {\mathbb{R}}^{2(r·c-r-c)\times (r·c)}$.

2.2. Adaptive SC and Its ${\ell }_0,{\ell }_1$ Solutions

For each patch ${\mathit{r}}_i$ (in the vectorized form) in the depth ridge map, we formulate the following adaptive SC model:

$$\begin{array}{cc}\hfill \underset{{\mathit{x}}_{vi},{\mathit{x}}_{hi}}{min}& {\left|\left|{\mathit{r}}_{vi}-{\mathit{D}}_L·{\mathit{x}}_{vi}\right|\right|}_2+{\left|\left|{\mathit{r}}_{hi}-{\mathit{D}}_L·{\mathit{x}}_{hi}\right|\right|}_2,\hfill \\ \hfill s.t.& \forall 1⩽i⩽N,{\left|\left|{\mathit{x}}_{vi}\right|\right|}_0⩽k_{vi},{\left|\left|{\mathit{x}}_{hi}\right|\right|}_0⩽k_{hi},\hfill \end{array}$$

(4)

where ${\mathit{r}}_{vi}$ and ${\mathit{r}}_{hi}$ represent the patches from vertical and horizontal ridge maps, respectively. ${\mathit{D}}_L$ is the fixed pre-learned dictionary, as shown in Figure 2, which is obtained by applying the basic SC algorithm [19] on 40 pre-selected ridge maps of man-made scenarios (here, we display our pre-learned dictionary with the optimal setting of parameters as obtained in Section 3.1.1, namely the number of dictionary atoms is 1024 and the size of each atom is 8 × 8). In [23], different dictionaries (including off-the-shelf ones such as discrete cosine transform (DCT) basis, wavelets basis, and dictionaries obtained via learning—either pre-learned or learned from the data itself) were tested for image denoising. Although the dictionary learned from the data itself reported the best denoising accuracy, it was also the most computationally demanding. In addition to our pre-learned dictionary, we also displayed the DCT dictionary, Gabor wavelets dictionary in Figure 2. The parameter tweaking of such dictionaries and their denoising performance are discussed in Section 3.1.1. ${\mathit{x}}_{vi}$ and ${\mathit{x}}_{hi}$ correspond to the sparse coefficient vectors. $k_{vi}$ and $k_{hi}$ are the given sparsity controlling parameters. Instead of setting these parameters as constants for all patches, we set them adaptively according to the informative level of each patch (see Section 3.1.1 for details) in a manner similar to that of [20]. Given $k_{vi}$ and $k_{hi}$, the popular orthogonal-matching-pursuit (OMP) algorithm is applied to solve such ${\ell }_0$ optimization problem. Moreover, the batch-OMP (BOMP) technique is exploited to further improve the efficiency. As the problem of (4) is non-convex and NP-hard, we reformulate its relaxation as below:

$$\begin{array}{cc}\hfill \underset{{\mathit{x}}_{vi},{\mathit{x}}_{hi}}{min}& {\left|\left|{\mathit{r}}_{vi}-{\mathit{D}}_L·{\mathit{x}}_{vi}\right|\right|}_2+{\lambda }_{vi}{\left|\left|{\mathit{x}}_{vi}\right|\right|}_1\hfill \\ \hfill +& {\left|\left|{\mathit{r}}_{hi}-{\mathit{D}}_L·{\mathit{x}}_{hi}\right|\right|}_2+{\lambda }_{hi}{\left|\left|{\mathit{x}}_{hi}\right|\right|}_1.\hfill \end{array}$$

(5)

Here, ${\lambda }_{vi}$ and ${\lambda }_{hi}$, which balance the representation fidelity term and the sparsity penalty term, are again estimated adaptively (see Section 3.1.1 for details). By stacking all the patches, we obtain Equation (6):

$$\begin{array}{cc}\hfill \underset{{\mathit{X}}_v,{\mathit{X}}_h}{min}& {\left|\left|[{\mathit{r}}_{v1},\cdots ,{\mathit{r}}_{vN}]-{\mathit{D}}_L·{\mathit{X}}_v\right|\right|}_2+\sum _{i=1}^N{\lambda }_{vi}{\left|\left|{\mathit{x}}_{vi}\right|\right|}_1\hfill \\ \hfill +& {\left|\left|[{\mathit{r}}_{h1},\cdots ,{\mathit{r}}_{hN}]-{\mathit{D}}_L·{\mathit{X}}_h\right|\right|}_2+\sum _{i=1}^N{\lambda }_{hi}{\left|\left|{\mathit{x}}_{hi}\right|\right|}_1,\hfill \end{array}$$

(6)

in which the neighboring patches are extracted with half-overlapping. To solve such ${\ell }_1$ optimization problem, the least-absolute-shrinkage-and-selection-operator (LASSO) algorithm was proposed, which has proven to find the global optimizer.

Clearly, Equations (4)–(6) differ from previous methods [12,14,19,20,22] in one aspect, which is that ${\mathit{D}}_L$ is fixed. Thus, ${\mathit{X}}_v$ and ${\mathit{X}}_h$ can be obtained with a one-step closed-form solution instead of performing refinement on the coefficients and dictionary alternately and iteratively, thus resulting in much improved efficiency. With ${\mathit{X}}_v$ and ${\mathit{X}}_h$ in hand, we reconstruct each patch ${\widehat{\mathit{r}}}_{vi}$ and ${\widehat{\mathit{r}}}_{hi}$. When all patches are processed, the average value is used for the locations occupied by multiple patches, and then we obtain the refined ridge maps ${\widehat{\mathit{R}}}_v$ and ${\widehat{\mathit{R}}}_h$. Next, we can recover $\widehat{\mathit{Z}}$ by performing the inverse operation of Equation (3). However, due to the dependency of the rows of ${\mathit{D}}_{vh}$, the system is under-determined. Therefore, we need to incorporate the boundary conditions to recover $\widehat{\mathit{Z}}$ from its refined second-order difference, as shown in Equation (7):

$$\left[\begin{array}{c}{\mathit{z}}_N\\ vec({\mathit{R}}_h)\\ vec({\mathit{R}}_v)\end{array}\right]=\left[\begin{array}{c}{\mathit{I}}_N{\mathit{0}}_{N\times (r·c-N)}\\ {\mathit{D}}_{vh}\end{array}\right]·vec(\mathit{Z})=\left[\begin{array}{c}\mathit{A}\\ {\mathit{D}}_{vh}\end{array}\right]·vec(\mathit{Z}),$$

(7)

where the available N boundary points are incorporated, and six boundary points are used in our work. A more rigorous theoretical proof of the minimum necessary boundary points is out of the scope of this work. Performing the inverse operation on (7), we can recover $\widehat{\mathit{Z}}$ as Equation (8):

$$vec(\widehat{\mathit{Z}})={\left[\left[\begin{array}{c}{\mathit{A}}^T{\mathit{D}}_{vh}^T\end{array}\right]\left[\begin{array}{c}\mathit{A}\\ {\mathit{D}}_{vh}\end{array}\right]\right]}^{-1}\left[\begin{array}{c}{\mathit{A}}^T{\mathit{D}}_{vh}^T\end{array}\right]\left[\begin{array}{c}{\mathit{z}}_N\\ vec({\mathit{R}}_h)\\ vec({\mathit{R}}_v)\end{array}\right].$$

(8)

Here, all the matrices $\mathit{A}$, ${\mathit{D}}_{vh}$, their transposes, and their inverses can be pre-computed to significantly reduce the overall processing time. We now summarize all previous operations as Algorithm 1. In fact, our algorithm includes two versions of implementation: our-BOMP and our-LASSO.

In Algorithm 2, the convergence criterion is well-defined as ${\left|\left|\mathit{x}\right|\right|}_0⩽k$. In Algorithm 3, the convergence criterion is that the relative update difference of ${\mathit{x}}_k$ is less than ${10}^{-3}$.

Algorithm 1 Our Sparse Coding for Depth Data Denoising

Input: Depth map $\mathit{Z}$, dictionary ${\mathit{D}}_c$, parameters $k_1$, $k_2$, ${\lambda }_v$, ${\lambda }_h$;  Output: Restored $\widehat{\mathit{Z}}$ of Equation (8); 1: Estimate the ridge maps ${\mathit{R}}_V$ and ${\mathit{R}}_H$ as Equation (2); 2: Extract and vectorize ridge map patches ${\mathit{r}}_{vi}$, ${\mathit{r}}_{hi}$, $1⩽i⩽N$; 3: //Line 5 & 7 apply different algorithms to estimate ${\mathit{x}}_{vi}$, ${\mathit{x}}_{hi}$. 4: //Line 5 belongs to the version of our-BOMP. 5: Apply the Batch-OMP algorithm to solve Equation (4); 6: //Line 7 belongs to the version of our-LASSO. 7: Apply the LASSO algorithm to solve Equation (5); 8: Obtain the refined ridge maps ${\widehat{\mathit{R}}}_v$ and ${\widehat{\mathit{R}}}_h$ via reconstruction; 9: Recover $\widehat{\mathit{Z}}$ via applying Equation (8).

Algorithm 2 Batch-orthogonal-matching-pursuit [24] to solve: ${min}_{\mathit{x}}{\left|\left|\mathit{y}-\mathit{D}·\mathit{x}\right|\right|}_{2}s.t.{\left|\left|\mathit{x}\right|\right|}_0⩽k$

Input: $\mathit{y}$, $\mathit{D}$, k;  Output: $\mathit{x}$; 1: Initialization: $\mathit{x}=\mathit{0}$, $\mathit{r}=\mathit{y}$, $\mathcal{S}=\{⌀\}$; 2: while stopping criterion not met do 3: $k={argmax}_k|{\mathit{d}}_k^T·\mathit{r}|$; 4: $\mathcal{L}=\mathcal{S}$; 5: $\mathcal{S}=\left\{k\right\}+\mathcal{L}$; 6: ${\mathit{G}}_{\mathcal{S}}={\mathit{D}}_{\mathcal{S}}^T{\mathit{D}}_{\mathcal{S}}$, ${\mathit{G}}_{\mathcal{L}}={\mathit{D}}_{\mathcal{L}}^T{\mathit{D}}_{\mathcal{L}}$; 7: //Line 8 obtains ${\mathit{G}}_{\mathcal{S}}^{-1}$ using progressive Cholesky update. 8: ${\mathit{x}}_{\mathcal{S}}={\mathit{G}}_{\mathcal{S}}^{-1}{\mathit{D}}_{\mathcal{S}}^T\mathit{y}$; the key updating function to estimate ${\mathit{G}}_{\mathcal{S}}^{-1}$ is: $${\mathit{G}}_{\mathcal{S}}^{-1}=\left[\begin{array}{cc}{\mathit{G}}_{\mathcal{L}}^{-1}+\frac{1}{a}{\mathit{G}}_{\mathcal{L}}^{-1}{\mathit{D}}_{\mathcal{L}}^T{\mathit{d}}_k{\mathit{d}}_k^T{\mathit{D}}_{\mathcal{L}}{\mathit{G}}_{\mathcal{L}}^{-1}& -\frac{1}{a}{\mathit{G}}_{\mathcal{L}}^{-1}{\mathit{D}}_{\mathcal{L}}^T{\mathit{d}}_k\\ -\frac{1}{a}{\mathit{d}}_k^T{\mathit{D}}_{\mathcal{L}}{\mathit{G}}_{\mathcal{L}}^{-1}& \frac{1}{a}\end{array}\right]$$ here, $a={\mathit{d}}_k^T{\mathit{d}}_k-{\mathit{d}}_k^T{\mathit{D}}_{\mathcal{L}}{\mathit{G}}_{\mathcal{L}}^{-1}{\mathit{D}}_{\mathcal{L}}^T{\mathit{d}}_k$ 9: $\mathit{r}=\mathit{y}-{\mathit{D}}_{\mathcal{S}}{\mathit{x}}_{\mathcal{S}}$; 10: end while

Algorithm 3 Fast iterative shrinkage-thresholding algorithm (FISTA) [25] to solve LASSO problem: ${\mathit{min}}_{\mathit{x}}{\left|\left|\mathit{y}-\mathit{D}·\mathit{x}\right|\right|}_{\mathit{2}}+\mathit{\lambda }{\left|\left|\mathit{x}\right|\right|}_{\mathit{1}}$

Input: $\mathit{y}$, $\mathit{D}$, $\lambda$;  Output: $\mathit{x}$; 1: Initialization: ${\mathit{x}}_0=\mathit{0}$, ${\mathit{z}}_1={\mathit{x}}_0$, $L=max(eig({\mathit{D}}^T\mathit{D}))$, $t_0=1$, $k=0$; 2: while not converged do 3: $k=k+1$; 4: $grad({\mathit{z}}_k)={\mathit{D}}^T\mathit{D}{\mathit{z}}_k-{\mathit{D}}^T\mathit{y}$; 5: ${\mathit{x}}_k={\mathcal{T}}_{\frac{\lambda }{L}}({\mathit{z}}_k-\frac{1}{L}grad({\mathit{z}}_k))$; 6: //see Section 2.1 for the definition of ${\mathcal{T}}_{\alpha }(\mathit{x})$. 7: $t_k=(1+\sqrt{1+4·t_{k-1}^2})/2$; 8: ${\mathit{z}}_{k+1}={\mathit{x}}_k+\frac{t_{k-1}-1}{t_k}({\mathit{x}}_k-{\mathit{x}}_{k-1})$; 9: end while

3. Experiments and Analysis

We performed experiments on both synthesized data and actual data recorded using a laser scanner. In the synthesized data experiments where the ground truth was available, the setting of important parameters was investigated, followed by denoising evaluation. In the real data experiments, the results are also demonstrated for visual assessment, in addition to the performance comparison. Similar to [14,20], the peak signal-to-noise ratio (PSNR) value is employed to evaluate the performance. Here, we clarify that our work is run on an ASUS Pro Notebook with a 2.4 GHz Intel Quad-Core i7-4500U processor and 12 GB RAM executing MATLAB code in parallel.

3.1. Experiments on Synthesized Data

Similar to [20,26], we designed a scene of a square box having sides of one meter in a room, as shown in Figure 3, where all information is known as the ground truth.

3.1.1. Important Parameters

To achieve plausible performance in terms of both accuracy and efficiency, the following parameters play the central role: the sparsity controlling parameters $k_{vi}$, $k_{hi}$, ${\lambda }_{vi}$, ${\lambda }_{hi}$, dictionary dimension d (which indicates that the patch size is $\sqrt{d}\times \sqrt{d}$ pixels), and its atom number $n_D$ ($r=\frac{n_D}{d}$ is the redundancy of the dictionary).

Applying the same parameter tweaking strategy as that in [20], we recommend the sparsity controlling parameters as shown in

Table 1. Setting of sparsity controlling parameters. BOMP: batch orthogonal-matching-pursuit; LASSO: least-absolute-shrinkage-and-selection-operator.

Parameters	Our–BOMP	Our–LASSO
	${\mathit{k}}_{\mathit{vi}},{\mathit{k}}_{\mathit{hi}}$	${\mathit{\lambda }}_{\mathit{vi}},{\mathit{\lambda }}_{\mathit{hi}}$
Max value	$k_{max}=6$	${\lambda }_{max}=2.2$
Min value	$k_{min}=1$	${\lambda }_{min}=0.2$
Adaptive value *	$k_i^v={\alpha }_i^v{k}_{min}+{\beta }_i^v{k}_{max}$	${\lambda }_i^v={\beta }_i^v{\lambda }_{min}+{\alpha }_i^v{\lambda }_{max}$
	$k_i^h={\alpha }_i^h{k}_{min}+{\beta }_i^h{k}_{max}$	${\lambda }_i^h={\beta }_i^h{\lambda }_{min}+{\alpha }_i^h{\lambda }_{max}$

* ${\alpha }_i^v,{\beta }_i^v,{\alpha }_i^h,{\beta }_i^h$ are related to the informative estimation of each patch.

. For each patch $P_j$ ($j=v$ or h, from vertical or horizontal ridge maps, respectively), the parameters ${\alpha }_{vi},{\beta }_{vi}$ (or ${\alpha }_{hi},{\beta }_{hi}$) are related to its informative-level measurement ($InM$), which is estimated as below:

$$\begin{array}{c}{D}_{en}\left(P_v^d\right)=\sum _{i=1}^{\sqrt{d}}\sum _{j=1}^{\sqrt{d}}\left|P_v^d\left[i,j\right]\right|,\hfill \\ R_{en}\left(P^r\right)=\sum _{i=1}^{\sqrt{d}}\sum _{j=1}^{\sqrt{d}}GradMag(P^r)\left[i,j\right],GradMag(P^r)\left[i,j\right]=\sqrt{{(\frac{\partial P^r[i,j]}{\partial x})}^2+{(\frac{\partial P^r[i,j]}{\partial y})}^2},\hfill \\ InM(P_v^d)=D_{en}\left(P_v^d\right)·exp(\sigma (R_{en}\left(P^r\right))-0.5).\hfill \end{array}$$

(9)

Here, $GradMag(P^r)$ is the gradient magnitude map of patch $P^r$, and $\sigma (·)$ is the sigmoid function $\sigma (·)=\frac{1}{1+exp(·)}$. For the input from $(-\infty ,+\infty )$, the output of $\sigma (·)$ is in the range $(0,1)$. In our work, as the value of $R_{en}$ cannot be negative, the output of $\sigma (R_{en})$ is in the range $[0.5,1)$. The output of $exp(\sigma (R_{en}(P^r))-0.5))$ is in the range $[1,e^{0.5})$. With $InM(P_v^d),InM(P_h^d)$ for all patches in hand, we determine the sparsity controlling parameters via computing ${\alpha }_{vi},{\beta }_{vi},{\alpha }_{hi},and{\beta }_{hi}$ as below:

$$\begin{array}{c}In{M}_{min}=min\{InM(P_{vi}^d),InM(P_{hi}^d)\},i=1,\cdots ,N,\hfill \\ In{M}_{max}=max\{InM(P_{vi}^d),InM(P_{hi}^d)\},i=1,\cdots ,N,\hfill \\ {\alpha }_{ji}=\frac{In{M}_{max}-InM(P_{ji}^d)}{In{M}_{max}-In{M}_{min}},j=vorh,\hfill \\ {\beta }_{ji}=\frac{InM(P_{ji}^d)-In{M}_{min}}{In{M}_{max}-In{M}_{min}},j=vorh.\hfill \end{array}$$

(10)

In Table 1, the positions of ${\alpha }_{ji}$ and ${\beta }_{ji}$ are exchanged in Our-BOMP and Our-LASSO because $k_{ji}$ and ${\lambda }_{ji}$ enforce the sparsity constraint in different manners to adaptively determine the number of dictionary atoms permitted for the patches with different informative levels. For example, to assign more atoms to represent the patch with more information, $k_{ji}$ should increase to push up the sparsity limit, whereas ${\lambda }_{ji}$ should decrease to penalize the sparsity constraint term to a lower degree.

In addition to $k_{ji},{\lambda }_{ji}$, we tested different combinations of patch size and redundancy to find the most reasonable settings of d and r. Specifically, $\sqrt{d}$ can be 4, 8, 16, or 24, and r can be 4, 8, 16, or 24. Thus, a total of 16 combinations were tested. As shown in Figure 4a, dictionaries with redundancy 16 consistently achieved better denoising results than dictionaries with redundancy 4, 8, and 24, when their patch sizes were the same. Figure 4b shows the computation time. Clearly, the time increased dramatically with increasing patch size and redundancy. Taking both the denoising performance and computation time into account, we conclude that the dictionary setting of $\sqrt{d}$ = 8 and r = 16 (namely 1024 atoms) can yield decent outputs, while allowing fast computation. Such parameters are used throughout this paper, except where indicated. In

Table 2. Denoising results (peak signal-to-noise ratio (PSNR) value) using different dictionaries (unit: dB).

Dictionary	Dictionary with Different Dimensions
	$\mathit{r}=10,\sqrt{\mathit{d}}=8$	$\mathit{r}=16,\sqrt{\mathit{d}}=8$	$\mathit{r}=16,\sqrt{\mathit{d}}=16$
DCT dictionary	32.26	32.37	32.17
Gabor Wavelet dictionary	32.46	32.58	32.92
Our pre-learned dictionary	32.86	33.75	33.81

, we further report the denoising result using different fixed dictionaries: our pre-learned dictionary, DCT dictionary, and Gabor wavelets dictionary. Clearly, the results using our pre-learned dictionary were better those using off-the-shelf ones.

3.1.2. Denoising Results

The denoising results of our work on the synthesized data (see Figure 3d in which 100% dense Gaussian noises with standard deviation of 0.02 m and gray-scale value 5 for range and reflectance data respectively were added) are reported in

Table 3. Denoising results on the synthesized data. RMS: root mean square; SC: sparse coding.

Method	Only Gaussian Noise			Gaussian Noise and Outliers
	PSNR (dB)	RMS (mm)	Time * (s)	PSNR (dB)	RMS (mm)	Time * (s)
Our–BOMP	33.72	2.06	0.08	33.28	2.14	0.09
Our–LASSO	33.78	2.04	0.12	33.35	2.12	0.15
Adaptive SC	33.83	1.98	227.38	33.71	2.04	239.62
Basic SC	33.17	2.17	153.62	33.02	2.56	172.38
Trilateral filter	32.37	2.49	0.04	32.12	2.67	0.06

* Time indicates the computation time for each method.

, together with the comparison against available competitive methods including Trilateral filter [26], Basic SC [19], and Adaptive SC [20]. Moreover, in addition to the Gaussian noise, sparse outliers (5% of the total positions, which were randomly distributed) were added to the scene, as shown in Figure 3e, and the denoising results are also reported in Table 3. In addition to the PSNR criteria, the root mean square (RMS, with units of millimeters) error indicating the difference between the recovered range data and the ground truth is also included, since it can give readers a more direct indication of how well the contaminated range information was restored by these methods. Moreover, the computational time of each method is also reported. As shown in Table 3, our methods outperformed Trilateral filter, Basic SC, and obtained comparable results to Adaptive SC in terms of denoising accuracy (the best results were obtained by Adaptive SC with PSNR = 33.83 and RMS = 1.98), while achieving significantly higher efficiency than Adaptive SC. Moreover, Our-BOMP even worked slightly more efficiently than Our-LASSO, achieving the processing speed of approximately 13 fps for the range image with resolution 800 × 800.

3.2. Denoising Data from Laser Scanner

This set of experiments used the Brown range image database [27] captured with a Riegl LMS-Z210 laser scanner, its field of view (FoV) is ${80}^{\circ }$ vertically and ${259}^{\circ }$ horizontally. We focused only on the 15 indoor scenes (e.g., classrooms and theatres). To deal with (and benefit from) such a large horizontal FoV, similar to the multiple local projection method in [19], we partitioned the scene into several parts with ${80}^{\circ }$ FoV vertically and horizontally. Note that in the Brown range image database, the depth of each measuring point is saved sequentially, left to right and up to down. Thus, the image location only indicates the order of saving and it was necessary to perform local projection to obtain reasonable images. The consecutive parts were half-overlapping along the horizontal direction, then each part was projected to an image plane orientated to the center. Finally, the average of all the restored parts was computed to restore the whole scene. In the following, the denoising results on such real data are reported and discussed. Similar to Section 3.1, Gaussian noise was added to the depth and reflectance data, and the results on three representative scenes are shown in Figure 5.

As can be observed in Figure 5, the results of Trilateral filter were good in the planar regions, but contained a great deal of noise in the edge regions. Our method, Basic SC, and Adaptive SC are all based on SC, and obtained consistent results in all regions. Moreover, the results of our method in the third column were slightly worse than those of columns 1 and 2, because the curved structure of the scene in the third column posed a greater challenge to our sparse ridge assumption. In

Table 4. Denoising results on real data.

Method	Only Gaussian Noise			Gaussian Noise and Outliers
	PSNR (dB)	RMS (mm)	Time * (s)	PSNR (dB)	RMS (mm)	Time * (s)
Our–BOMP	35.37	2.31	0.86	35.17	2.43	0.89
Our–LASSO	35.43	2.27	1.15	35.21	2.37	1.28
Adaptive SC	35.72	2.03	918.71	35.68	2.08	972.33
Basic SC	34.28	2.83	538.37	34.16	2.97	572.81
Trilateral filter	32.13	3.27	0.16	32.02	3.75	0.18

* Time indicates the computation time for each method.

, the average denoising results on all 15 indoor scenes of the dataset are reported. Similar to the results on the synthesized data, our method outperformed Trilateral filter and Basic SC consistently, and obtained comparable results to those of Adaptive SC in terms of denoising accuracy, while achieving significantly higher efficiency than Adaptive SC. Compared with the results in Table 3, the results of Table 4 deteriorated as the structure of real scenes was more complex than the synthetic scene. Similar to the previous section, in addition to the Gaussian noise, sparse outliers were added to the scene, and the denoising results are also reported in Table 4.

4. Conclusions

In this study, based on our previous work [20], we propose SC-based algorithms in which a pre-learned ridge dictionary is applied to realize range data denoising by leveraging the regularity of laser range measurements in man-made environments. Experiments on both synthesized data and real data demonstrate that our method obtained accuracies comparable to those of sophisticated SC methods with much higher efficiency, achieving approximately 13 fps (for resolution of 800 × 800) with the resource of a lightweight laptop computer. Therefore, our method can be applied for real-time systems in man-made environments. Furthermore, we are trying to implement our method with GPU acceleration, intending to further increase the efficiency and also save the precious payload for unmanned systems, especially UAVs.