Error Estimations for Total Variation Type Regularization
Abstract
:1. Introduction
2. Notation
3. Basic Error Estimations
3.1. Stability
3.2. Consistency
3.3. Convergence Rate
4. Improved Convergence Rate
4.1. Performance under Sparsity Assumption
4.2. Performance if Sparsity Assumption Fails
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Donoho, D.L. Compressed sensing. IEEE T. Inform. Theory 2006, 52, 1289–1306. [Google Scholar] [CrossRef]
- Candes, E.J.; Tao, T. Decoding by linear programming. IEEE T. Inform. Theory 2005, 51, 4203–4215. [Google Scholar] [CrossRef] [Green Version]
- Lustig, M.; Donoho, D.L.; Santos, J.M.; Pauly, J.M. Compressed Sensing MRI. IEEE Signal Proc. Mag. 2008, 25, 72–82. [Google Scholar] [CrossRef]
- Haupt, J.; Bajwa, W.U.; Rabbat, M.; Nowak, R. Compressed Sensing for Networked Data. IEEE Signal Proc. Mag. 2008, 25, 92–101. [Google Scholar] [CrossRef]
- Yin, W.; Osher, S.; Goldfarb, D.; Darbon, J. Bregman iterative algorithms for ℓ1-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 2008, 1, 143–168. [Google Scholar] [CrossRef] [Green Version]
- Cai, J.F.; Osher, S.; Shen, Z. Split Bregman methods and frame based image restoration. Multiscale Model. Sim. 2009, 8, 337–369. [Google Scholar] [CrossRef]
- Van den Berg, E.; Friedlander, M.P. Probing the Pareto Frontier for Basis Pursuit Solutions. SIAM J. Sci. Comput. 2008, 31, 890–912. [Google Scholar] [CrossRef] [Green Version]
- Chen, S.S.; Donoho, D.L.; Saunders, M.A. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 1998, 20, 33–61. [Google Scholar] [CrossRef]
- Candès, E.J.; Romberg, J.; Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 2006, 52, 489–509. [Google Scholar] [CrossRef] [Green Version]
- Candès, E.J. The restricted isometry property and its implications for compressed sensing. CR Math. 2008, 346, 589–592. [Google Scholar]
- Tibshirani, R. Regression Shrinkage and Selection via the Lasso. J. R. Stat. Soc. B 1996, 58, 267–288. [Google Scholar] [CrossRef]
- Daubechies, I.; Defrise, M.; Mol, C.D. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 2004, 57, 1413–1457. [Google Scholar] [CrossRef] [Green Version]
- Daubechies, I.; Devore, R.; Fornasier, M.; Güntürk, C.S. Iteratively reweighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. 2010, 38, 1–38. [Google Scholar] [CrossRef] [Green Version]
- Bot, R.I.; Hofmann, B. The impact of a curious type of smoothness conditions on convergence rates in ℓ1-regularization. Eur. J. Math. Comput. Appl. 2013, 1, 29–40. [Google Scholar]
- Jin, B.; Maass, P. Sparsity regularization for parameter identification problems. Inverse Probl. 2012, 28, 123001. [Google Scholar] [CrossRef]
- Grasmair, M.; Haltmeier, M.; Scherzer, O. Sparse regularization with ℓq penalty term. Inverse Probl. 2008, 24, 055020. [Google Scholar] [CrossRef] [Green Version]
- Hans, E.; Raasch, T. Global convergence of damped semismooth Newton methods for 1 Tikhonov regularization. Inverse Probl. 2015, 31, 025005. [Google Scholar] [CrossRef]
- Lorenz, D.A.; Schiffler, S.; Trede, D. Beyond convergence rates: Exact recovery with the Tikhonov regularization with sparsity constraints. Inverse Probl. 2011, 27, 085009. [Google Scholar] [CrossRef]
- Lorenz, D.A. Convergence rates and source conditions for Tikhonov regularization with sparsity constraints. J. Inverse III-Pose P 2008, 16, 463–478. [Google Scholar] [CrossRef] [Green Version]
- Rubinov, A.M.; Yang, X.Q.; Bagirov, A.M. Penalty functions with a small penalty parameter. Optim. Methods Softw. 2002, 17, 931–964. [Google Scholar] [CrossRef]
- Cai, J.F.; Dong, B.; Osher, S.; Shen, A.Z. Image Restoration: Total Variation, Wavelet Frames and Beyond. J. Am. Math. Soc. 2012, 25, 1033–1089. [Google Scholar] [CrossRef] [Green Version]
- Rudin, L.I.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Phys. D 1992, 60, 259–268. [Google Scholar] [CrossRef]
- Jun, L.; Huang, T.Z.; Gang, L.; Wang, S.; Lv, X.-G. Total variation with overlapping group sparsity for speckle noise reduction. Neurocomputing 2019, 216, 502–513. [Google Scholar]
- Van den Berg, P.M.; Kleinman, R.E. A total variation enhanced modified gradient algorithm for profile reconstruction. Inverse Probl. 1995, 11, L5. [Google Scholar] [CrossRef]
- Clason, C.; Jin, B.; Kunisch, K. A Duality-Based Splitting Method for ℓ1-TV Image Restoration with Automatic Regularization Parameter Choice. SIAM J. Sci. Comput. 2010, 32, 1484–1505. [Google Scholar] [CrossRef]
- Cai, J.F.; Xu, W.Y. Guarantees of total variation minimization for signal recovery. Inform. Infer. 2015, 4, 328–353. [Google Scholar] [CrossRef] [Green Version]
- Needell, D.; Ward, R. Stable Image Reconstruction Using Total Variation Minimization. SIAM J. Imaging Sci. 2012, 6, 1035–1058. [Google Scholar] [CrossRef] [Green Version]
- Sun, T.; Yin, P.; Cheng, L.; Jiang, H. Alternating direction method of multipliers with difference of convex functions. Adv. Comput. Math. 2018, 44, 723–744. [Google Scholar] [CrossRef]
- Sun, T.; Jiang, H.; Cheng, L.; Zhu, W. Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems. IEEE Trans. Signal Proc. 2018, 66, 5380–5391. [Google Scholar] [CrossRef] [Green Version]
- Leonov, A.S. On the total-variation convergence of regularizing algorithms for ill-posed problems. Comput. Math. Math. Phys. 2007, 47, 732–747. [Google Scholar] [CrossRef]
- Tikhonov, A.; Leonov, A.; Yagola, A. Nonlinear Ill-Posed Problems; De Gruyter: Berlin, Germany, 2011; pp. 505–512. [Google Scholar] [CrossRef]
- Hào, D.N.; Quyen, T.N.T. Convergence rates for total variation regularization of coefficient identification problems in elliptic equations I. Inverse Probl. 2011, 27, 075008. [Google Scholar] [CrossRef]
- Hào, D.N.; Quyen, T.N.T. Convergence rates for total variation regularization of coefficient identification problems in elliptic equations II. J. Math. Anal. Appl. 2012, 388, 593–616. [Google Scholar] [CrossRef] [Green Version]
- Ciegis, R.; Sev, A.J. Nonlinear Diffusion Problems In Image Smoothing. Math. Model. Anal. 2005, 1, 381–388. [Google Scholar]
- Grasmair, M.; Scherzer, O.; Haltmeier, M. Necessary and sufficient conditions for linear convergence of ℓ1-regularization. Commun. Pure Appl. Math. 2011, 64, 161–182. [Google Scholar] [CrossRef]
- Burger, M.; Osher, S. Convergence rates of convex variational regularization. Inverse Probl. 2004, 20, 1411. [Google Scholar] [CrossRef]
- Burger, M.; Flemming, J.; Hofmann, B. Convergence rates in ℓ1-regularization if the sparsity assumption fails. Inverse Probl. 2013, 29, 025013. [Google Scholar] [CrossRef] [Green Version]
- Flemming, J.; Hofmann, B.; Veselić, I. A unified approach to convergence rates for ℓ1-regularization and lacking sparsity. J. Inverse III-Pose P 2015, 24, 139–148. [Google Scholar] [CrossRef]
- Anzengruber, S.W.; Hofmann, B.; Mathé, P. Regularization properties of the sequential discrepancy principle for Tikhonov regularization in Banach spaces. Appl. Anal. 2014, 93, 1382–1400. [Google Scholar] [CrossRef]
- Luo, Z.Q.; Tseng, P. On the convergence of the coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl. 1992, 72, 7–35. [Google Scholar] [CrossRef] [Green Version]
- Luo, Z.Q.; Tseng, P. Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim. 1992, 2, 43–54. [Google Scholar] [CrossRef] [Green Version]
- Jin, B.; Lorenz, D.A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM J. Numer. Anal. 2010, 48, 1208–1229. [Google Scholar] [CrossRef] [Green Version]
- Jin, B.; Zou, J. Iterative parameter choice by discrepancy principle. IMA J. Numer. Anal. 2012, 32, 1714–1732. [Google Scholar] [CrossRef]
- Jin, B.; Lorenz, D.A.; Schiffler, S. Elastic-Net Regularization: Error estimates and Active Set Methods. Inverse Probl. 2009, 25, 115022. [Google Scholar] [CrossRef] [Green Version]
- Bredies, K.; Lorenz, D.A. Linear convergence of iterative soft-thresholding. J. Fourier Anal. Appl. 2008, 14, 813–837. [Google Scholar] [CrossRef] [Green Version]
- Foucart, S.; Rauhut, H. A Mathematical Introduction to Compressive Sensing; Springer: New York, NY, USA, 2013. [Google Scholar]
- Anzengruber, S.W.; Ramlau, R. Convergence rates for Morozov’s discrepancy principle using variational inequalities. Inverse Probl. 2011, 27, 105007–105024. [Google Scholar] [CrossRef] [Green Version]
- Flemming, J. Generalized Tikhonov Regularization and Modern Convergence Rate Theory in Banach Spaces; Shaker Verlag GmbH: Düren, Germany, 2012. [Google Scholar]
- Hofmann, B.; Mathé, P. Parameter choice in Banach space regularization under variational inequalities. Inverse Probl. 2012, 6, 1035–1058. [Google Scholar] [CrossRef] [Green Version]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, K.; Huang, C.; Yuan, Z. Error Estimations for Total Variation Type Regularization. Mathematics 2021, 9, 1373. https://doi.org/10.3390/math9121373
Li K, Huang C, Yuan Z. Error Estimations for Total Variation Type Regularization. Mathematics. 2021; 9(12):1373. https://doi.org/10.3390/math9121373
Chicago/Turabian StyleLi, Kuan, Chun Huang, and Ziyang Yuan. 2021. "Error Estimations for Total Variation Type Regularization" Mathematics 9, no. 12: 1373. https://doi.org/10.3390/math9121373
APA StyleLi, K., Huang, C., & Yuan, Z. (2021). Error Estimations for Total Variation Type Regularization. Mathematics, 9(12), 1373. https://doi.org/10.3390/math9121373