# Tensor Decomposition for Salient Object Detection in Images

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

**F**is decomposed into the sum of a low-rank matrix $L\in {\Re}^{N\times D}$ and sparse matrix $S\in {\Re}^{N\times D}$ by solving optimization problem in Equation (1). $||\cdot |{|}_{*}$ stands for the nuclear norm which computes the sum of the singular values of a matrix. It is a convex relaxation of the matrix rank function. $||\cdot |{|}_{1}$ represents the l

_{1}-norm to promote the sparsity. λ is a constant to balance the tradeoff between the low-rank term and the sparse term.

## 3. Proposed Tensor Decomposition Method for Saliency Detection

_{1}-norm for the high-dimensional tensor case with the order greater than three. Based on the relationship between high-dimensional tensor and matrix, we use the following tensor trace norm [25] for a n-order tensor $\mathbf{X}$:

_{1}-norm:

_{1}-norm) of a high-dimensional tensor is a linear combination of the trace norms (or l

_{1}-norms) of all matrices unfolded along each mode. Note that, the l

_{1}-norm definition in this article is different from the general norm definition of a tensor in [19].

_{1}-norm along each tube fibers, i.e., ${\mathbf{S}}_{ij:}$, of tensor $\mathbf{S}$ is used to measure the saliency of corresponding segmented region in the i-th row and j-th column of the superpixels. The final saliency map is then accordingly generated and normalized to a 0 to 1 scale image.

## 4. Experimental Results

#### 4.1. Benchmark Datasets and Evaluation Criteria

_{β}-measure. Specifically, MAE was used to measure the average pixel-wise absolute difference between detected saliency map and corresponding ground-truth saliency map. The OR was used to measure the importance of complete detection of salient objects. The F

_{β}-measure is defined as

_{β}= (1 + β

^{2})·P·R/(β

^{2}·P + R)

^{2}= 0.3 is to ensure more stress on precision than recall [33]. The weighted F

_{β}–measure (WF) was also used to measure the importance of saliency detection performance. Precision (P) equals the percentage of salient pixel correctly assigned. Recall (R) is the ratio between the correctly detected salient pixels and true saliency pixel in the ground-truth saliency map.

#### 4.2. Efficient of the Heuristic Tensor Decomposition Process

#### 4.3. Saliency Detection Performance Comparison

## 5. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Peng, H.; Li, B.; Ling, H.; Hu, W.; Xiong, W.; Maybank, S.J. Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 39, 818–832. [Google Scholar] [CrossRef] - Liu, T.; Yuan, Z.; Sun, J.; Wang, J.; Zheng, N.; Tang, X.; Shum, H.-Y. Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell.
**2011**, 33, 353–367. [Google Scholar] - Mai, L.; Niu, Y.; Liu, F. Saliency aggregation: A data-driven approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 1131–1138. [Google Scholar]
- Jiang, H.; Wang, J.; Yuan, Z.; Wu, Y.; Zheng, N.; Li, S. Salient object detection: A discriminative regional feature integration approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2083–2090. [Google Scholar]
- Zhao, R.; Ouyang, W.; Li, H.; Wang, X. Saliency detection by multi-context deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1265–1274. [Google Scholar]
- Li, G.; Yu, Y. Deep contrast learning for salient object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 478–487. [Google Scholar]
- Hou, Q.; Cheng, M.-M.; Hu, X.; Borji, A.; Tu, Z.; Torr, P.H. Deeply supervised salient object detection with short connections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 3203–3212. [Google Scholar]
- Cheng, M.-M.; Mitra, N.J.; Huang, X.; Torr, P.H.; Hu, S.-M. Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2015**, 37, 569–582. [Google Scholar] [CrossRef] - Perazzi, F.; Krähenbühl, P.; Pritch, Y.; Hornung, A. Saliency filters: Contrast based filtering for salient region detection. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 733–740. [Google Scholar]
- Shen, X.; Wu, Y. A unified approach to salient object detection via low rank matrix recovery. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 853–860. [Google Scholar]
- Wang, W.; Wang, Y.; Huang, Q.; Gao, W. Measuring visual saliency by site entropy rate. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2368–2375. [Google Scholar]
- Guo, Y.; Wang, F.; Liu, P.; Xin, J.; Zheng, N. Multi-scale point set saliency detection based on site entropy rate. In Proceedings of the Pacific Rim Conference on Multimedia, Xi’an, China, 15–16 September 2016; pp. 366–375. [Google Scholar]
- Zhang, L.; Ai, J.; Jiang, B.; Lu, H.; Li, X. Saliency detection via absorbing markov chain with learnt transition probability. IEEE Trans. Image Process.
**2018**, 27, 987–998. [Google Scholar] [CrossRef] - Ishikura, K.; Kurita, N.; Chandler, D.M.; Ohashi, G. Saliency detection based on multiscale extrema of local perceptual color differences. IEEE Trans. Image Process.
**2018**, 27, 703–717. [Google Scholar] [CrossRef] [PubMed] - Candès, E.J.; Li, X.; Ma, Y.; Wright, J. Robust principal component analysis? J. ACM
**2011**, 58, 11. [Google Scholar] [CrossRef] - Yan, J.; Zhu, M.; Liu, H.; Liu, Y. Visual saliency detection via sparsity pursuit. IEEE Signal Process. Lett.
**2010**, 17, 739–742. [Google Scholar] [CrossRef] - Zheng, Q.; Yu, S.; You, X.; Peng, Q.; Yuan, W. Coarse-to-fine salient object detection with low-rank matrix recovery. arXiv
**2018**, arXiv:1805.07936. [Google Scholar] - Tang, C.; Wang, P.; Zhang, C.; Li, W. Salient object detection via weighted low rank matrix recovery. IEEE Signal Process. Lett.
**2017**, 24, 490–494. [Google Scholar] [CrossRef] - Kolda, T.G.; Bader, B.W. Tensor decompositions and applications. SIAM Rev.
**2009**, 51, 455–500. [Google Scholar] [CrossRef] - Sidiropoulos, N.D.; De Lathauwer, L.; Fu, X.; Huang, K.; Papalexakis, E.E.; Faloutsos, C. Tensor decomposition for signal processing and machine learning. IEEE Signal Process.
**2017**, 65, 3551–3582. [Google Scholar] [CrossRef] - Jia, C.; Fu, Y. Low-rank tensor subspace learning for RGB-D action recognition. IEEE Trans. Image Process.
**2016**, 25, 4641–4652. [Google Scholar] [CrossRef] [PubMed] - Fanaee-T., H.; Gama, J. Tensor-based anomaly detection: An interdisciplinary survey. Knowl.-Based Syst.
**2016**, 98, 130–147. [Google Scholar] [CrossRef] [Green Version] - Chen, J.; Mitra, U. A tensor decomposition technique for source localization from multimodal data. In Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15–20 April 2018; pp. 4074–4078. [Google Scholar]
- Hillar, C.J.; Lim, L.-H. Most tensor problems are NP-hard. J. ACM
**2013**, 60, 45. [Google Scholar] [CrossRef] - Liu, J.; Musialski, P.; Wonka, P.; Ye, J. Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell.
**2013**, 35, 208–220. [Google Scholar] [CrossRef] [PubMed] - Wright, J.; Ganesh, A.; Rao, S.; Peng, Y.; Ma, Y. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 6 December 2010; pp. 2080–2088. [Google Scholar]
- Simoncelli, E.P.; Freeman, W.T. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In Proceedings of the International Conference on Image Processing, Washington, DC, USA, 23–26 October 1995; pp. 444–447. [Google Scholar]
- Feichtinger, H.G.; Strohmer, T. Gabor Analysis and Algorithms: Theory and Applications; Springer Science and Business Media: New York, NY, USA, 2012. [Google Scholar]
- Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Süsstrunk, S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell.
**2012**, 34, 2274–2282. [Google Scholar] [CrossRef] [PubMed] - Zhu, W.; Liang, S.; Wei, Y.; Sun, J. Saliency optimization from robust background detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 2814–2821. [Google Scholar]
- Batra, D.; Kowdle, A.; Parikh, D.; Luo, J.; Chen, T. Interactively co-segmentating topically related images with intelligent scribble guidance. Int. J. Comput. Vis.
**2011**, 93, 273–292. [Google Scholar] [CrossRef] - Yan, Q.; Xu, L.; Shi, J.; Jia, J. Hierarchical saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 1155–1162. [Google Scholar]
- Achanta, R.; Hemami, S.; Estrada, F.; Süsstrunk, S. Frequency-tuned salient region detection. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA, 20–25 June 2009; pp. 1597–1604. [Google Scholar]
- Zou, W.; Kpalma, K.; Liu, Z.; Ronsin, J. Segmentation driven low-rank matrix recovery for saliency detection. In Proceedings of the 24th British Machine Vision Conference (BMVC), Bristol, UK, 9 September 2013; pp. 1–13. [Google Scholar]
- Kim, J.; Han, D.; Tai, Y.-W.; Kim, J. Salient region detection via high-dimensional color transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 883–890. [Google Scholar]

**Figure 2.**Saliency map comparison. (

**a**) Image; (

**b**) tensor decomposition original (TDO) saliency map (with sub-problem solutions integrated); (

**c**) ground-truth saliency map; (

**d**) sub-problem 1 saliency map; (

**e**) sub-problem 2 saliency map; (

**f**) sub-problem 3 saliency map.

**Figure 3.**Quantitative comparison on three datasets in terms of F-measure curve. (

**a**) F-measure comparison on iCoSeg dataset; (

**b**) F-measure comparison on ECSSD dataset; (

**c**) F-measure comparison on MASRA10K dataset.

**Figure 4.**Visual comparison of saliency maps of different methods. (

**a**) Image; (

**b**) ground-truth; (

**c**) TDS; (

**d**) TDO; (

**e**) WLRR; (

**f**) SMD; (

**g**) SLR; (

**h**) DRFI; (

**i**) RBD; (

**j**) HCT.

**Table 1.**Result on iCoSeg dataset in terms of weighted F

_{β}–measure (WF), mean absolute error (MAE), overlapping ration (OR), and area under the RO curve (AUC).

Metric | TDS | TDO | SMD | WLRR | SLR | DRFI | RBD | HCT |
---|---|---|---|---|---|---|---|---|

WF↑ | 0.641 | 0.568 | 0.611 | 0.602 | 0.473 | 0.592 | 0.599 | 0.464 |

OR↑ | 0.601 | 0.564 | 0.598 | 0.578 | 0.505 | 0.582 | 0.588 | 0.519 |

AUC↑ | 0.839 | 0.836 | 0.822 | 0.843 | 0.805 | 0.839 | 0.827 | 0.833 |

MAE↓ | 0.134 | 0.162 | 0.138 | 0.147 | 0.179 | 0.139 | 0.138 | 0.179 |

**Table 2.**Result on extended complex scene saliency dataset (ECSSD) dataset in terms of WF, MAE, OR, and AUC.

Metric | TDS | TDO | SMD | WLRR | SLR | DRFI | RBD | HCT |
---|---|---|---|---|---|---|---|---|

WF↑ | 0.551 | 0.502 | 0.517 | 0.500 | 0.442 | 0.517 | 0.490 | 0.430 |

OR↑ | 0.520 | 0.506 | 0.523 | 0.498 | 0.474 | 0.527 | 0.494 | 0.457 |

AUC↑ | 0.815 | 0.818 | 0.775 | 0.819 | 0.764 | 0.780 | 0.752 | 0.755 |

MAE↓ | 0.185 | 0.208 | 0.227 | 0.211 | 0.252 | 0.217 | 0.225 | 0.249 |

Metric | TDS | TDO | SMD | WLRR | SLR | DRFI | RBD | HCT |
---|---|---|---|---|---|---|---|---|

WF↑ | 0.713 | 0.652 | 0.704 | 0.659 | 0.601 | 0.666 | 0.685 | 0.582 |

OR↑ | 0.706 | 0.690 | 0.741 | 0.696 | 0.692 | 0.723 | 0.716 | 0.674 |

AUC↑ | 0.849 | 0.849 | 0.847 | 0.853 | 0.840 | 0.857 | 0.834 | 0.847 |

MAE↓ | 0.108 | 0.131 | 0.104 | 0.130 | 0.141 | 0.114 | 0.108 | 0.143 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhou, J.; Tao, Y.; Liu, X.
Tensor Decomposition for Salient Object Detection in Images. *Big Data Cogn. Comput.* **2019**, *3*, 33.
https://doi.org/10.3390/bdcc3020033

**AMA Style**

Zhou J, Tao Y, Liu X.
Tensor Decomposition for Salient Object Detection in Images. *Big Data and Cognitive Computing*. 2019; 3(2):33.
https://doi.org/10.3390/bdcc3020033

**Chicago/Turabian Style**

Zhou, Junxiu, Yangyang Tao, and Xian Liu.
2019. "Tensor Decomposition for Salient Object Detection in Images" *Big Data and Cognitive Computing* 3, no. 2: 33.
https://doi.org/10.3390/bdcc3020033