# A New Filter Nonmonotone Adaptive Trust Region Method for Unconstrained Optimization

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The New Algorithm

Algorithm 1. A new filter nonmonotone adaptive trust region method. |

Step 0. (Initialization) An initial point ${x}_{0}\in {R}^{n}$ and a symmetric matrix ${B}_{0}\in {R}^{n}\times {R}^{n}$ are given. The constants $0<{\mu}_{1}<{\mu}_{2}<1$, $0<{\eta}_{\mathrm{min}}\le {\eta}_{\mathrm{max}}<1$, $\tau >0$, $N>0$, $\epsilon >0$, ${\eta}_{\mathrm{min}}\in [0,1)$ and ${\eta}_{\mathrm{max}}\in [{\eta}_{\mathrm{min}},1)$ are also given. Step 1. If $\Vert {g}_{k}\Vert \le \epsilon $, then stop. Step 2. Solve the subproblem (2) to find the trial step ${d}_{k}$. Step 3. Choose ${w}_{ki}\in \left[0,1\right]$, which satisfies $\sum _{i=1}^{\mathrm{min}\{k,m\}}{w}_{ki}}=1$. Compute ${R}_{k}$, ${\widehat{\rho}}_{k}$, and ${\widehat{\rho}}_{k}{}^{\prime}$, respectively. Step 4. Test the trial step. If ${\widehat{\rho}}_{k}\ge {\mu}_{1}$, then set ${x}_{k+1}={x}_{k}^{+}$. Otherwise compute ${g}_{k}^{+}=\nabla f({x}_{k}^{+})$; if ${x}_{k}^{+}$ is acceptable by the filter $F$, then ${x}_{k+1}={x}_{k}^{+}$, add ${g}_{k}^{+}=\nabla f({x}_{k}^{+})$ into the filter $F$. Otherwise, find the step length ${\alpha}_{k}$ satisfying (4), set ${x}_{k+1}={x}_{k}+{\alpha}_{k}{d}_{k}$. end(if) end(If) Step 5. Update the trust region radius by ${\Delta}_{k+1}={c}_{k+1}{\Vert {g}_{k+1}\Vert}^{\gamma}$, where ${c}_{k+1}$ is updated by (12). Step 6. Compute the new Hessian approximation ${B}_{k+1}$ by a modified BFGS method formula. Set $k\text{}=\text{}k\text{}+\text{}1$, and return to Step 1. |

## 3. Convergence Analysis

**Assumption**

**1.**

**Remark**

**1.**

**Remark**

**2.**

**Lemma**

**1.**

**Proof.**

**Lemma**

**2.**

**Proof.**

**Lemma**

**3.**

**Proof.**

**Lemma**

**4.**

**Proof.**

**Lemma**

**5.**

**Proof.**

**Lemma**

**6.**

**Proof.**

**Theorem**

**1.**

**Proof.**

## 4. Local Convergence

**Theorem**

**2.**

**Proof.**

## 5. Preliminary Numerical Experiments

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Mo, J.T.; Zhang, K.C.; Wei, Z.X. A nonmonotone trust region method for unconstrained optimization. Appl. Math. Comput.
**2005**, 171, 371–384. [Google Scholar] [CrossRef] - Ou, Y.G.; Zhou, Q.; Lin, H.C. An ODE-based trust region method for unconstrained optimization problems. J. Comput. Appl. Math.
**2009**, 232, 318–326. [Google Scholar] [CrossRef] [Green Version] - Wang, X.Y.; Tong, J. A Nonmonotone Adaptive Trust Region Algorithm with Fixed Stepsize for Unconstrained Optimization Problems. Math. Appl.
**2009**, 3, 496–500. [Google Scholar] - Hang, D.; Liu, M. On a Fixed Stepsize Nonmonotonic Self-Adaptive Trust Region Algorithm. J. Southwest China Norm. Univ.
**2013**, 38. [Google Scholar] [CrossRef] - Sartenaer, A. Automatic determination of an initial trust region in nonlinear programming. SIAM J. Sci. Comput.
**1997**, 18, 1788–1803. [Google Scholar] [CrossRef] [Green Version] - Zhang, X.S.; Zhang, J.L.; Liao, L.Z. An adaptive trust region method and its convergence. Sci. China
**2002**, 45, 620–631. [Google Scholar] [CrossRef] [Green Version] - Shi, Z.J.; Guo, J.H. A new trust region methods for unconstrained optimization. Comput. Appl. Math.
**2008**, 213, 509–520. [Google Scholar] [CrossRef] [Green Version] - Kimiaei, M. A new class of nonmonotone adaptive trust-region methods for nonlinear equations with box constraints. Calcolo
**2017**, 54, 769–812. [Google Scholar] [CrossRef] - Amini, K.; Shiker Mushtak, A.K.; Kimiaei, M. A line search trust-region algorithm with nonmonotone adaptive radius for a system of nonlinear equations. Q. J. Oper. Res.
**2016**, 4, 132–152. [Google Scholar] [CrossRef] - Peyghami, M.R.; Tarzanagh, D.A. A relaxed nonmonotone adaptive trust region method for solving unconstrained optimization problems. Comput. Optim. Appl.
**2015**, 61, 321–341. [Google Scholar] [CrossRef] - Deng, N.Y.; Xiao, Y.; Zhou, F.J. Nonmonotone Trust Region Algorithm. J. Optim. Theory Appl.
**1993**, 76, 259–285. [Google Scholar] [CrossRef] - Ahookhoosh, M.; Amini, K.; Peyghami, M. A nonmonotone trust region line search method for large scale unconstrained optimization. Appl. Math. Model.
**2012**, 36, 478–487. [Google Scholar] [CrossRef] - Zhang, H.C.; Hager, W.W. A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim.
**2004**, 14, 1043–1056. [Google Scholar] [CrossRef] [Green Version] - Gu, N.Z.; Mo, J.T. Incorporating nonmonotone strategies into the trust region for unconstrained optimization. Comput. Math. Appl.
**2008**, 55, 2158–2172. [Google Scholar] [CrossRef] [Green Version] - Fletcher, R.; Leyffer, S. Nonlinear programming without a penalty function. Math. Program.
**2002**, 91, 239–269. [Google Scholar] [CrossRef] - Gould, N.I.; Sainvitu, C.; Toint, P.L. A filter-trust-region method for unconstrained optimization. SIAM J. Optim.
**2005**, 16, 341–357. [Google Scholar] [CrossRef] [Green Version] - Gould, N.I.; Leyffer, S.; Toint, P.L. A multidimensional filter algorithm for nonlinear equations and nonlinear least-squares. SIAM J. Optim.
**2004**, 15, 17–38. [Google Scholar] [CrossRef] [Green Version] - Wächter, A.; Biegler, L.T. Line search filter methods for nonlinear programming and global convergence. SIAM J. Optim.
**2005**, 16, 1–31. [Google Scholar] [CrossRef] - Miao, W.H.; Sun, W. A filter trust-region method for unconstrained optimization. Numer. Math. J. Chin. Univ.
**2007**, 19, 88–96. [Google Scholar] - Zhang, Y.; Sun, W.; Qi, L. A nonmonotone filter Barzilai-Borwein method for optimization. Asia Pac. J. Oper. Res.
**2010**, 27, 55–69. [Google Scholar] [CrossRef] - Fatemi, M.; Mahdavi-Amiri, N. A filter trust-region algorithm for unconstrained optimization with strong global convergence properties. Comput. Optim. Appl.
**2012**, 52, 239–266. [Google Scholar] [CrossRef] - Nocedal, J.; Wright, S.J. Numerical Optimization; Springer: New York, NY, USA, 2006. [Google Scholar]
- Andrei, N. An unconstrained optimization test functions collection. Environ. Sci. Technol.
**2008**, 10, 6552–6558. [Google Scholar] - Pang, S.; Chen, L. A new family of nonmonotone trust region algorithm. Math. Pract. Theory.
**2011**, 10, 211–218. [Google Scholar] - Dolan, E.D.; Moŕe, J.J. Benchmarking optimization software with performance profiles. Math. Program.
**2002**, 91, 201–213. [Google Scholar] [CrossRef]

Problem | $\mathit{n}$ | ${\mathit{n}}_{\mathit{f}}/{\mathit{n}}_{\mathit{i}}$ | |||||
---|---|---|---|---|---|---|---|

ANTRFS [9] | CPU | FSNATR [20] | CPU | Algorithm 1 | CPU | ||

Ext.Rose | 4 | 755/382 | 2.755795 | 168/88 | 0.322036 | 87/58 | 0.104316 |

Ext. Beale | 4 | 25/13 | 0.008651 | 41/21 | 0.069185 | 18/16 | 0.028946 |

Penalty i | 2 | 18/10 | 0.087532 | 18/10 | 0.067020 | 17/14 | 0.032533 |

Pert.Quad | 6 | 28/25 | 0.058921 | 25/13 | 0.058700 | 18/17 | 0.035631 |

Raydan 1 | 8 | 18/10 | 0.015109 | 38/20 | 0.105928 | 39/20 | 0.070292 |

Raydan 2 | 4 | 21/11 | 0.015356 | 13/8 | 0.012729 | 11/6 | 0.017449 |

Diagonal 1 | 10 | 13/8 | 0.009493 | 35/18 | 0.070199 | 27/26 | 0.064282 |

Diagonal 2 | 10 | 56/29 | 0.017841 | 58/30 | 0.119385 | 57/29 | 0.083905 |

Diagonal 3 | 50 | 200/101 | 1.926143 | 182/92 | 1.232287 | 127/126 | 1.849887 |

Hager | 10 | 27/14 | 0.049037 | 27/14 | 0.048247 | 33/17 | 0.071906 |

Gen. Trid 1 | 20 | 967/484 | 3.536055 | 50/26 | 0.432577 | 47/24 | 0.217367 |

Ext.Trid 1 | 10 | 27/14 | 0.013890 | 29/15 | 0.128696 | 18/12 | 0.071580 |

Ext. TET | 50 | 13/7 | 0.203093 | 16/9 | 0.031416 | 17/9 | 0.119907 |

Diadonal 4 | 100 | 7/4 | 0.035933 | 9/5 | 0.343849 | 5/4 | 0.146901 |

Ext.Him | 100 | 29/15 | 0.147102 | 25/13 | 0.208976 | 29/28 | 0.409463 |

Gen. White | 50 | 785/576 | 10.47342 | 771/429 | 9.940880 | 443/228 | 5.741535 |

Ext. Powell | 16 | 1567/787 | 7.266044 | 794/404 | 2.148929 | 496/337 | 1.208253 |

Full Hessian FH3 | 100 | 11/6 | 0.053598 | 11/6 | 0.084726 | 8/7 | 0.088831 |

Ext.BD1 | 100 | 51/27 | 0.210790 | 50/28 | 0.739621 | 21/15 | 0.261978 |

Pert. Quad | 200 | 91/66 | 2.547689 | 87/44 | 2.421596 | 57/56 | 2.405979 |

Extended Hiebert | 16 | 1821/1000 | 9.819290 | 175/143 | 2.456780 | 135/68 | 0.527388 |

Quadratic QF1 | 4 | 15/8 | 0.007903 | 17/9 | 0.017025 | 11/10 | 0.010983 |

FLETCHCR34 | 36 | 210/123 | 1.847519 | 150/91 | 0.950314 | 165/83 | 1.786160 |

ARWHEAD | 200 | 297/150 | 37.928050 | 29/15 | 0.317976 | 15/12 | 0.317976 |

NONDIA | 50 | 75/39 | 0.368280 | 92/47 | 0.544079 | 51/35 | 0.307129 |

DQDRTIC | 50 | 67/38 | 0.51243 | 53/28 | 0.341435 | 32/30 | 0.318596 |

EG2 | 200 | 32/17 | 0.319954 | 28/16 | 0.373764 | 49/35 | 2.633184 |

Bro.Tridiagonal | 200 | 2797/1504 | 441.453385 | 744/398 | 119.570838 | 69/35 | 1.539657 |

A.Per.Quad | 16 | 73/47 | 0.144890 | 63/32 | 0.132644 | 45/26 | 0.128349 |

Pert.Trid.Quad | 100 | 330/166 | 10.985321 | 325/163 | 9.663929 | 289/156 | 8.521700 |

Ext.DENSCH | 100 | 37/19 | 0.190549 | 43/22 | 0.398777 | 128/82 | 5.638770 |

SINCOS | 100 | 4303/2152 | 198.717544 | 1303/952 | 142.543185 | 65/36 | 1.122092 |

BIGGSB1 | 10 | 1949/1042 | 8.466655 | 329/195 | 0.676394 | 275/185 | 0.376394 |

ENGVAL1 | 200 | 788/487 | 139.949938 | 643/406 | 99.088596 | 474/472 | 88.401960 |

EDENSCH | 100 | 474/238 | 25.639664 | 45/26 | 0.407574 | 37/23 | 0.930150 |

CUBE | 100 | 430/220 | 21.53234 | 357/198 | 20.93564 | 280/147 | 13.946540 |

BDEXP | 100 | 476/369 | 34.54797 | 452/356 | 24.569196 | 22/21 | 0.550708 |

GENHUMPS | 100 | 532/321 | 3.27453 | 412/213 | 0.475453 | 1014/537 | 1.235720 |

QUARTC | 100 | 57/32 | 1.035734 | 43/22 | 0.443325 | 18/17 | 0.326680 |

Gen. PSC1 | 500 | 198/212 | 10.457624 | 51/54 | 9.562354 | 51/54 | 8.539801 |

Ext. PSC1 | 500 | 15/15 | 1.254327 | 15/15 | 1.0983452 | 13/13 | 1.562763 |

Variably dim. | 500 | 41/27 | 2.9578243 | 21/16 | 1.4536982 | 17/15 | 1.093456 |

DIXMAANA | 1000 | 21/21 | 1.457893 | 21/21 | 1.237642 | 20/20 | 1.025372 |

SINQUAD | 1000 | 1582/1063 | 187.563723 | 1995/1215 | 135.872354 | 912/579 | 100.458723 |

DIXMAANJ | 1000 | 2415/2398 | 431.253485 | 2320/2311 | 410.253485 | 2246/2132 | 397.256732 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wang, X.; Ding, X.; Qu, Q.
A New Filter Nonmonotone Adaptive Trust Region Method for Unconstrained Optimization. *Symmetry* **2020**, *12*, 208.
https://doi.org/10.3390/sym12020208

**AMA Style**

Wang X, Ding X, Qu Q.
A New Filter Nonmonotone Adaptive Trust Region Method for Unconstrained Optimization. *Symmetry*. 2020; 12(2):208.
https://doi.org/10.3390/sym12020208

**Chicago/Turabian Style**

Wang, Xinyi, Xianfeng Ding, and Quan Qu.
2020. "A New Filter Nonmonotone Adaptive Trust Region Method for Unconstrained Optimization" *Symmetry* 12, no. 2: 208.
https://doi.org/10.3390/sym12020208