Machine Learning Applications in High Dimensional Stochastic Control

A special issue of Algorithms (ISSN 1999-4893).

Deadline for manuscript submissions: closed (30 August 2022) | Viewed by 2059

Special Issue Editor

School of Mathematical and Physical Sciences, University of Technology Sydney, Sydney 2007, Australia
Interests: machine learning; optimization; algorithms, stochastich control

Special Issue Information

Dear Colleagues,

Decision-theoretic planning is naturally formulated and solved using a discrete-time stochastic control framework. This theory provides a fundamental and intuitive formalism not only for sequential decision optimization but also for diverse learning problems. One of the main research areas here deals with the following situation: Starting from a random terminal value of a sequential decision problem, a non-linear recursion computes the value of a strategy at a given time from its expectation one-time step ahead.  Such nesting in the calculations causes a deviation from the true value, which is inevitably progressing through all time steps of the backward induction. This error propagation is typically hard to quantify and control, particularly if the dimension of the decision variables is high, causing a variety of problems -- usually referred to as the curse of dimensionality. To deal with the dimension problems, numerous approaches have been suggested, ranging from the discretization of the state space to approximations of functions on this space. These techniques are usually combined with Monte-Carlo methods. However, many important problems still remain computationally infeasible. The proposed special issue of "Algorithms" focuses on a modern development: Unlike in the classical approximate dynamic programming, an optimal policy can be approached using neural networks techniques. This idea benefits from a huge pool of knowledge accumulated in the Machine Learning area and raises the hope that with some work, we soon can address control problems whose dimension is out of reach today.

I cordially invite your contributions to this exciting development.

Dr. Juri Hinz
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (1 paper)

Order results
Result details
Select all
Export citation of selected articles as:

Research

19 pages, 508 KiB  
Article
An Algorithm for Making Regime-Changing Markov Decisions
by Juri Hinz
Algorithms 2021, 14(10), 291; https://doi.org/10.3390/a14100291 - 04 Oct 2021
Viewed by 1541
Abstract
In industrial applications, the processes of optimal sequential decision making are naturally formulated and optimized within a standard setting of Markov decision theory. In practice, however, decisions must be made under incomplete and uncertain information about parameters and transition probabilities. This situation occurs [...] Read more.
In industrial applications, the processes of optimal sequential decision making are naturally formulated and optimized within a standard setting of Markov decision theory. In practice, however, decisions must be made under incomplete and uncertain information about parameters and transition probabilities. This situation occurs when a system may suffer a regime switch changing not only the transition probabilities but also the control costs. After such an event, the effect of the actions may turn to the opposite, meaning that all strategies must be revised. Due to practical importance of this problem, a variety of methods has been suggested, ranging from incorporating regime switches into Markov dynamics to numerous concepts addressing model uncertainty. In this work, we suggest a pragmatic and practical approach using a natural re-formulation of this problem as a so-called convex switching system, we make efficient numerical algorithms applicable. Full article
(This article belongs to the Special Issue Machine Learning Applications in High Dimensional Stochastic Control)
Show Figures

Figure 1

Back to TopTop