Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
3.7
2.3
Journals
Algorithms
Special Issues
Numerical Optimization in Honor of the 60th Birthday of Marko...
share announcement

Numerical Optimization in Honor of the 60th Birthday of Marko M. Mäkelä

A special issue of Algorithms (ISSN 1999-4893). This special issue belongs to the section "Algorithms for Multidisciplinary Applications".

Deadline for manuscript submissions: 30 June 2024 | Viewed by 3907

Special Issue Editors

Dr. Sona Taheri

E-Mail Website
Guest Editor
School of Science, RMIT University, Melbourne, VIC 3001, Australia
Interests: nonsmooth optimization; data mining; machine learning
Special Issues, Collections and Topics in MDPI journals
Dr. Kaisa Joki

E-Mail Website
Guest Editor
Department of Mathematics and Statistics, University of Turku, 20014 Turku, Finland
Interests: nonsmooth optimization; DC optimization; nonconvex optimization; machine learning and data analysis
Dr. Napsu Karmitsa

E-Mail Website
Guest Editor
Department of Computing, University of Turku, 20014 Turku, Finland
Interests: nonsmooth optimization; large-scale optimization; machine learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Numerical optimization is a branch of mathematical and numerical analysis. In practice, every problem in science, engineering, economics, medicine, and machine learning (to name a few) can be formulated as an optimization problem. However, not all optimization problems are alike. Each problem possesses unique characteristics, such as a special structure of the objective, continuous/discrete variables, various constraints, or the multiobjective nature of the problem that requires specialized tools and techniques to be tackled effectively. By recognizing and understanding these distinctive characteristics, we can unlock the full potential of optimization and develop powerful, efficient numerical methods.

To advance the field of numerical optimization, this Special Issue aims to bring together state-of-the-art techniques and methods. We invite innovative submissions from all areas of numerical optimization and beyond, with a specific focus on the research interests of Prof. Marko M. Mäkelä. These interests encompass numerical methods for nonsmooth nonconvex optimization, including DC optimization, multiobjective optimization, mixed integer nonlinear programming, and global optimization, as well as the theoretical foundations supporting these methods. Additionally, we are interested in applications within these fields, especially those emerging in the industry.

Dr. Sona Taheri
Dr. Kaisa Joki
Dr. Napsu Karmitsa
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • numerical optimization methods
  • nonsmooth optimization
  • nonconvex optimization
  • DC programming
  • mixed integer programming
  • global optimization

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

21 pages, 2540 KiB  
Article
Analysis of a Two-Step Gradient Method with Two Momentum Parameters for Strongly Convex Unconstrained Optimization
by Gerasim V. Krivovichev and Valentina Yu. Sergeeva
Algorithms 2024, 17(3), 126; https://doi.org/10.3390/a17030126 - 18 Mar 2024
Viewed by 752
Abstract
The paper is devoted to the theoretical and numerical analysis of the two-step method, constructed as a modification of Polyak’s heavy ball method with the inclusion of an additional momentum parameter. For the quadratic case, the convergence conditions are obtained with the use [...] Read more.
The paper is devoted to the theoretical and numerical analysis of the two-step method, constructed as a modification of Polyak’s heavy ball method with the inclusion of an additional momentum parameter. For the quadratic case, the convergence conditions are obtained with the use of the first Lyapunov method. For the non-quadratic case, sufficiently smooth strongly convex functions are obtained, and these conditions guarantee local convergence.An approach to finding optimal parameter values based on the solution of a constrained optimization problem is proposed. The effect of an additional parameter on the convergence rate is analyzed. With the use of an ordinary differential equation, equivalent to the method, the damping effect of this parameter on the oscillations, which is typical for the non-monotonic convergence of the heavy ball method, is demonstrated. In different numerical examples for non-quadratic convex and non-convex test functions and machine learning problems (regularized smoothed elastic net regression, logistic regression, and recurrent neural network training), the positive influence of an additional parameter value on the convergence process is demonstrated. Full article
(This article belongs to the Special Issue Numerical Optimization in Honor of the 60th Birthday of Marko M. Mäkelä)
►▼ Show Figures

Figure 1

26 pages, 679 KiB  
Article
Deep Neural Networks Training by Stochastic Quasi-Newton Trust-Region Methods
by Mahsa Yousefi and Ángeles Martínez
Algorithms 2023, 16(10), 490; https://doi.org/10.3390/a16100490 - 20 Oct 2023
Viewed by 1211
Abstract
While first-order methods are popular for solving optimization problems arising in deep learning, they come with some acute deficiencies. To overcome these shortcomings, there has been recent interest in introducing second-order information through quasi-Newton methods that are able to construct Hessian approximations using [...] Read more.
While first-order methods are popular for solving optimization problems arising in deep learning, they come with some acute deficiencies. To overcome these shortcomings, there has been recent interest in introducing second-order information through quasi-Newton methods that are able to construct Hessian approximations using only gradient information. In this work, we study the performance of stochastic quasi-Newton algorithms for training deep neural networks. We consider two well-known quasi-Newton updates, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (BFGS) and the symmetric rank one (SR1). This study fills a gap concerning the real performance of both updates in the minibatch setting and analyzes whether more efficient training can be obtained when using the more robust BFGS update or the cheaper SR1 formula, which—allowing for indefinite Hessian approximations—can potentially help to better navigate the pathological saddle points present in the non-convex loss functions found in deep learning. We present and discuss the results of an extensive experimental study that includes many aspects affecting performance, like batch normalization, the network architecture, the limited memory parameter or the batch size. Our results show that stochastic quasi-Newton algorithms are efficient and, in some instances, able to outperform the well-known first-order Adam optimizer, run with the optimal combination of its numerous hyperparameters, and the stochastic second-order trust-region STORM algorithm. Full article
(This article belongs to the Special Issue Numerical Optimization in Honor of the 60th Birthday of Marko M. Mäkelä)
►▼ Show Figures

Figure 1

21 pages, 565 KiB  
Article
Bundle Enrichment Method for Nonsmooth Difference of Convex Programming Problems
by Manlio Gaudioso, Sona Taheri, Adil M. Bagirov and Napsu Karmitsa
Algorithms 2023, 16(8), 394; https://doi.org/10.3390/a16080394 - 21 Aug 2023
Viewed by 913
Abstract
The Bundle Enrichment Method (BEM-DC) is introduced for solving nonsmooth difference of convex (DC) programming problems. The novelty of the method consists of the dynamic management of the bundle. More specifically, a DC model, being the difference of two convex piecewise [...] Read more.
The Bundle Enrichment Method (BEM-DC) is introduced for solving nonsmooth difference of convex (DC) programming problems. The novelty of the method consists of the dynamic management of the bundle. More specifically, a DC model, being the difference of two convex piecewise affine functions, is formulated. The (global) minimization of the model is tackled by solving a set of convex problems whose cardinality depends on the number of linearizations adopted to approximate the second DC component function. The new bundle management policy distributes the information coming from previous iterations to separately model the DC components of the objective function. Such a distribution is driven by the sign of linearization errors. If the displacement suggested by the model minimization provides no sufficient decrease of the objective function, then the temporary enrichment of the cutting plane approximation of just the first DC component function takes place until either the termination of the algorithm is certified or a sufficient decrease is achieved. The convergence of the BEM-DC method is studied, and computational results on a set of academic test problems with nonsmooth DC objective functions are provided. Full article
(This article belongs to the Special Issue Numerical Optimization in Honor of the 60th Birthday of Marko M. Mäkelä)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop