Machine Learning-Based Model Predictive Control for Collaborative Production Planning Problem with Unknown Information

Abstract

In industrial production planning problems, the accuracy of the accessible market information has the highest priority, as it is directly associated with the reliability of decisions and affects the efficiency and effectiveness of manufacturing. However, during a collaborative task, certain private information regarding the participants might be unknown to the regulator, and the production planning decisions thus become biased or even inaccurate due to the lack of full information. To improve the production performance in this specific case, this paper combines the techniques of machine learning and model predictive control (MPC) to create a comprehensive algorithm with low complexity. We collect the historical data of the decision-making process while the participants make their individual decisions with a certain degree of bias and analyze the collected data using machine learning to estimate the unknown parameter values by solving a regression problem. Based on an accurate estimate, MPC helps the regulator to make optimal decisions, maximizing the overall net profit of a given collaborative task over a future time period. A simulation-based case study is conducted to validate the performance of the proposed algorithm in terms of estimation accuracy. Comparisons with individual and pure MPC decisions are also made to verify its advantages in terms of increasing profit.

1. Introduction

In modern industry, the participants of a manufacturing task usually work in a collaborative mode, which substantially increases productivity [1]. The individual decisions of the participants contribute to changes in state variables within the industrial process, which in turn affect their own task performance. To address this decision-making process, a dynamic model of the given collaborative task is established to represent the interaction and communication between the participants [2]. Within these collaborative tasks, each participant naturally makes individual decision to optimize their own task performance, if no decision is made by the regulator. However, the optimal decisions for the overall optimal task performance usually conflict with these individual decisions according to the Nash Equilibrium [3]. Furthermore, the participants might even make individual decisions with a certain bias and fail to optimize their individual benefits due to a lack of professional experience. Therefore, an essential responsibility for regulators is to optimize the overall task performance rather than any individual’s performance, which creates production planning problems in a collaborative mode.

The class of production planning problems aims at making appropriate decisions for an industrial manufacturing process to optimize the performance of certain tasks and yield various practical benefits, such as maximum net profits [4,5,6], fuzzy multiple objectives [7,8,9], economic production quantity [10] and optimal emission policy [11,12]; see [13,14,15,16,17] for detailed overviews of production planning problems. Most existing production planning methods assume that all information within their problem formulations are known as a priori. Nevertheless, this paper focuses on the production planning problems in collaborative tasks, which contain private information regarding participants that might be unknown to the regulator. In this sense, the regulator cannot obtain the full scope of accurate market information to make optimal decisions for participants, which is an obstacle to making reliable decisions.

To accurately estimate the parameter values of unknown private information, the data-driven technique of machine learning is considered in this paper, which is a collection of approaches for making inferences based upon historical data [18]. This approach extracts useful information from a realistic data set to tune the weighting parameters of a trained model, meaning that this model asymptotically returns desired information to the users [19]. This advantage unlocks intelligent solutions to certain design problems that can hardly be characterized using classical methods due to their complexity and fuzziness [20]. Numerous machine learning procedures have been proposed in different research areas, such as image classification [21,22], data classification and clustering [23,24,25], system identification [26], natural language processing [27], autonomous driving [28] and fault diagnosis [29,30]. Most of these state-of-the-art machine learning approaches are designed for classification tasks with various data types, such as images, audio and video. However, this paper addresses a parameter estimation task by solving a regression problem based on the recorded biased individual decisions, and a “gradient descent”-based machine learning pipeline is proposed to fulfill this task in an efficient manner.

The technique of MPC is employed in this paper to solve the specific production planning problem based on the estimated parameter values. We generally exploit the model and current state information to predict future information so that an appropriate decision is made to optimize the overall task performance over a future time period, which is computationally efficient [31]. Because of this key feature, various MPC methods have been implemented to improve the task performance of broad applications, such as sludge processes [32], micro-grid [33] and power electronics [34]; see [35,36,37] for more details about MPC. Although some existing works in the literature—e.g., [38,39,40]—handle the production planning problem with uncertain variables, their problem formulations were different from that discussed in this paper with disembedded machine learning or MPC methods. Moreover, this machine learning-based MPC method is capable of extracting the target information from a complex but real historical data set and carrying out the estimation procedure in an off-line fashion. It outperforms other on-line learning based MPC methods—e.g., [41,42,43,44]—in terms of its small computation load.

To overcome the aforementioned problems, this paper combines the techniques of machine learning and MPC to aid regulators to compute optimal decisions for the participants, solving the production planning problem with a maximum overall net profit. Note that some preliminary results of this paper have been reported in [45], but significant extensions are proposed in this paper. A detailed discussion is made of technical issues such as Remarks 1–4, an entirely new section (Section 4, Implementation Instructions) is added to help the reader to understand the behind-screen parameter tuning mechanism of the algorithm, the case study section employs a newly established model to handle the problem, and the results have been improved compared to those in the published paper because of the new contributions added in this paper. The contributions of this paper are highlighted as follows:

• The production planning problem is formulated using a discrete time system, with task performance judged by net profit;
• A gradient descent machine learning procedure with an adaptive learning scheme is developed to estimate the unknown parameters of the revenue in Q using historical data via solving a regression problem;
• An MPC method uses the estimated values of Q as its user-defined weight factors to predict the optimal decisions to maximize net profit;
• A machine learning-based MPC algorithm with low complexity is proposed and validated in a simulation-based case study;
• A comparison with individual and pure MPC decisions is performed to show the increase in profit.

The nomenclature is listed as follows:

${\mathbb{R}}^n$	is an n-dimensional Euclidean vector space,
${\mathbb{R}}^{m\times n}$	is an $m\times n$ real matrix space,
$a_{ij}$	is the element in ith row and jth column of A,
$\mathbf{diag}(·)$	is a diagonal matrix of its argument,
$\rho (·)$	is the spectral radius of a given matrix,
$\mathsf{\Omega }$	is an input constraint set,
$f(·)$	is a cost function of the performance index,
${\parallel x\parallel }_{\infty }$	is the infinity norm of the vector x,
${\mathcal{P}}_{\mathsf{\Omega }}(x)$	is the projection of x onto the set $\mathsf{\Omega }$,
$\mathcal{N}(·)$	is the representation of normal distribution,
$\widehat{x}$	is the estimated value of x,
$\mathcal{L}(·)$	is a loss function,
$\mathbb{Z}$	is the set of integer numbers,
$\mathbb{N}$	is the set of non-negative integer numbers,
s	is the production demand,
Q	is the weighting parameters of the revenue,
R	is the weighting parameters of the productivity effort,
P	is the decision bias parameters of the participants,
$\mathsf{\Pi }$	is the benchmark of the unknown parameters.

2. Problem Formulation

This section introduces the system dynamics and uses certain cost functions to represent the desired task performance. Then, the production planning problem with a collaborative mode and unknown information is formulated, and the effect of unknown parameters is discussed.

2.1. System Dynamics

In this paper, the interaction among participants within a given collaborative manufacturing task is modeled by a discrete time system with the following state space representation:

$$x(k+1)=Ax(k)+Bu(k).$$

(1)

The system input $u(k)\in {\mathbb{R}}^n$ and the system state $x(k)\in {\mathbb{R}}^n$ denote the productivity decisions and market sizes of all n participants at sample time index k, respectively. The matrices A and B are of compatible dimensions; i.e., $A,B\in {\mathbb{R}}^{n\times n}$.

The future state $x(k+1)$ of the discrete time system (1) is triggered by the current state $x(k)$ and the current input $u(k)$. In other words, the market size of each participant at the next sample time depends on the current market sizes of the participants as well as the current productivity decision. Therefore, the individual decision of each participant influences the market sizes of others, and their market size is also affected by the individual decisions of others, which gives rise to the system dynamics. If $\rho (A)⩽1$, the discrete time system (1) is stable. The matrix B is diagonal, i.e.,

$$B=\mathbf{diag}(b_1,b_2,\dots ,b_n),$$

(2)

as the decision of an individual participant does not directly affect the market sizes of other participants, and its diagonal elements are non-negative since the productivity clearly has a positive relationship with respect to the market size.

Remark 1.

The problem formulation in this paper can be naturally extended to other non-linear systems without difficulties. However, this kind of extension requires extra modifications to the solution to the production planning problem, which is not discussed in this paper.

2.2. System Constraints

In practice, it should be realized that the system variables have specific physical limits that give rise to different forms of system constraints. For example, the system input

$$u(k)={[u_1(k),u_2(k),\dots ,u_n(k)]}^{\top }$$

(3)

represents the productivity of the participants at sample time index k, and all its elements $u_i(k)$, $i=1,\dots ,n$ must be non-negative and have an upper bound for maximum productivity. The input constraint set $\mathsf{\Omega }$ usually represents the acceptable system input range. In this paper, the input saturation constraint $u(k)\in \mathsf{\Omega }$, i.e.,

$$\mathsf{\Omega }=\{u\in {\mathbb{R}}^n:0⩽u_i⩽{\overline{u}}_i,i=1,\dots ,n\}$$

(4)

is considered and used to compute the solutions of the production planning problem in latter sections, where ${\overline{u}}_i$ denotes the input upper bound.

2.3. Production Planning Design Objectives

In a manufacturing task, the overall task performance is generally represented by a cost function $f(u,x,k)$ consisting of the productivity decision u, the market size x and the sample time index k. For example, the change in market size is denoted as

$$f(u,x,k)=x(k+1)-x(k),$$

(5)

and the maximum productivity is denoted as

$$f(u,x,k)={∥u(k)∥}_{\infty }.$$

(6)

Note that the cost function $f(u,x,k)$ does not necessarily need to incorporate all variables of u, x and k, and its exact definition can change in terms of the target performance index to be optimized. To improve the task performance, the regulator should make an appropriate decision for the participants to optimize a certain performance index. This design objective brings significant practical benefits within the given task. Therefore, the production planning problem explored in this paper is formulated in the following definition.

Definition 1.

TheProduction Planning Problemrefers to the choice of an appropriate productivity decision $u(k)$ of a discrete time system (1) at sample time index k, such that the desired cost function $f(u,x,k)$ is optimized over the time period $[1,N]$; i.e.,

$$\begin{array}{cc}\hfill u=arg\underset{u}{max}& \sum _{k=1}^{N}f(u,x,k),\hfill \\ \hfill s.t.& x(k+1)=Ax(k)+Bu(k),\hfill \\ \hfill & u(k)\in \mathsf{\Omega },k=0,\dots ,N-1.\hfill \end{array}$$

(7)

To exemplify this design objective, this paper takes the overall net production profit of a collaborative manufacturing task to be the task performance, defined as

$$f(u,x,k)=g(x,k)-h(u,k-1),$$

(8)

where $g(x,k)$ and $h(u,k)$ are the revenue function and the effort function with respect to the sample time index k. This cost function makes sense in practice as the production effort made at sample time index $k-1$ brings the revenue in the next sample time index k.

The revenue function $g(x,k)$ is in a quadratic form,

$$g(x,k)=-\frac{1}{2}{x}^{\top }(k)Qx(k)+s^{\top }(k)x(k),$$

(9)

where the diagonal matrix

$$Q=\mathbf{diag}(q_1,q_2,\dots ,q_n)$$

(10)

contains the constant weighting parameters of the revenue for each participant, and the vector

$$s(k)=[s_1(k),s_2(k),\dots ,s_n(k)]$$

(11)

represents the production demands of the participants at sample time index k, which is a known time-varying value. In the problem formulation, the production demand $s(k)$ is the required order number of the products and does not have a direct relationship with the market size $x(k)$, which denotes the actual number of products made by the factory. However, they together contribute to the revenue function $g(x,k)$. According to the quadratic form (9), the revenue of each participant first increases to a peak value and then decreases along its market size axis. Due to the fact of excess demand, the overall revenue decreases after a certain market size. Note that the revenue is zero for zero market size, which is realistic in practice. The production demand is not a fixed value as the demand for certain products may vary with respect to the sample time index k. In addition, the production demand has a positive effect on the revenue, as an increase of production demand increases the product price.

The effort function $h(x,k)$ is in a quadratic form,

$$h(u,k)=\frac{1}{2}{u}^{\top }(k)Ru(k),$$

(12)

where the diagonal matrix

$$R=\mathbf{diag}(r_1,r_2,\dots ,r_n)$$

(13)

contains the constant weighting parameters of the productivity effort for the participants. Based on the quadratic form (12), the effort of each participant increases quadratically as its productivity increases from the value of zero. This quadratic form is realistic as high productivity generally leads to extra costs in terms of human power, machine damage and energy consumption.

To solve the production planning problem (7), a reliable system model (1) and accurate values of R, Q and $s(k)$ are required. However, unknown parameters occur frequently in the practical production planning problem. In this specific case, the participators naturally attempt to optimize their private performance index, and their individual decisions might even conflict with the optimal decisions made by the regulator for the overall task performance. Therefore, they might not share their private revenue weighting parameters $q_i$ with the regulator or provide inaccurate values. Thus, the matrix Q is generally unknown or inaccurate to the regulator, which restricts the reliability of optimal decisions.

To handle this problem, the data-driven method of machine learning is applied to estimate these unknown parameters by extracting and analyzing the information from the recorded historical data, and MPC is then used to provide optimal decisions for the regulator based on estimated parameter values.

3. Machine Learning-Based MPC

In this section, a formula is derived to model the individual decisions of the participants under the free decision making condition. A gradient descent machine learning training procedure with low complexity is proposed to estimate the unknown parameters using the recorded historical data and the obtained formula. Then, MPC is considered to solve the problem (7) based on the estimated values of the parameters to yield the optimal decisions.

3.1. Individual Decision Modeling

A free decision making condition is now supposed for the participators, meaning that there are no optimal decisions received from the regulator. Therefore, they have the authorization to make individual decisions for themselves. It is not surprising that the individual decisions are often biased from their optimal value due to the low response to market environment change and the restricted insight of the participators. In this sense, the decision bias matrix

$$P=\mathbf{diag}(p_1,p_2,\dots ,p_n),$$

(14)

is introduced to provide the decision bias parameters of the participants. These parameters $p_i$, $i=1,\dots ,n$ are unknown random values with a normal distribution $\mathcal{N}({\mu }_i,{\sigma }_i^2)$, where ${\mu }_i$ is the mean value and ${\sigma }_i$ is the standard deviation. The decision bias parameters represent the inherent difference between actual individual decisions and associated optimal options. By introducing these parameters, a representation formula of the individual decisions is derived and shown in the next theorem.

Theorem 1.

Under the free decision making condition, the $i^{th}$ participant attempts to maximize their individual task performance, defined by the cost function

$$f_i(x_i,u_i,k)=-\frac{1}{2}{q}_i{x}_i^2(k+1)+s_i(k+1)x_i(k+1)-\frac{1}{2}{r}_i{u}_i^2(k),$$

(15)

at sample time index k, and the individual decision is

$$u_i(k)={\mathcal{F}}_i({\mathsf{\Pi }}_i,s(k+1),x(k)),$$

(16)

where ${\mathsf{\Pi }}_i=(q_i,p_i)$ involves the unknown parameters of the participant and function ${\mathcal{F}}_i$ is defined by

$${\mathcal{F}}_i({\mathsf{\Pi }}_i,s,x)={\mathcal{P}}_{{\mathsf{\Omega }}_i}((1+p_i)\frac{b_i{s}_i-b_i{q}_i{\sum }_{j=1}^n{a}_{ij}{x}_j}{q_i{b}_i^2+r_i}),$$

(17)

with the projection operator

$${\mathcal{P}}_{{\mathsf{\Omega }}_i}(u)=arg\underset{{\tilde{u}}_i}{min}\left|u_i-{\tilde{u}}_i\right|,s.t.0⩽{\tilde{u}}_i⩽{\overline{u}}_i.$$

(18)

Proof. 

Under the free decision making condition, each participant makes their decision actively and focuses on maximizing their individual cost function (15), which gives rise to

$$\underset{u_i}{min}\frac{1}{2}{q}_i{x}_i^2-s_i(k+1)x_i+\frac{1}{2}{r}_i{u}_i^2,s.t.x_i=\sum _{j=1}^n{a}_{ij}{x}_j(k)+b_i{u}_i.$$

(19)

The problem (19) is a quadratic programming problem, and the partial differentiation of the cost function is performed with respect to $u_i$ to find the identical stationary point. As the problem is convex, this stationary point is the global minimum and gives the unconstrained solution

$$(b_i{s}_i(k+1)-b_i{q}_i\sum _{j=1}^n{a}_{ij}{x}_j(k))/(q_i{b}_i^2+r_i).$$

(20)

Using the decision bias parameter $p_i$, the actual decision of each participant is

$$(1+p_i)(b_i{s}_i(k+1)-b_i{q}_i\sum _{j=1}^n{a}_{ij}{x}_j(k))/(q_i{b}_i^2+r_i).$$

(21)

To satisfy the input constraint, the projection operator ${\mathcal{P}}_{{\mathsf{\Omega }}_i}(·)$ is used to provide the individual decision (16).    □

The above theorem derives a representation formula of the individual decisions under the free decision making condition. It not only attempts to optimize the individual task performance of each participant but also incorporates the decision bias into the individual decision making processes. Moreover, the representation formula of the individual decision vector is stated in the next corollary.

Corollary 1.

According to the Formula (16) in Theorem 1, the overall individual decisions are equivalently written into the vector form

$$u(k)=\mathcal{F}(\mathsf{\Pi },s(k+1),x(k)),$$

(22)

where $\mathsf{\Pi }=(Q,P)$ denotes the benchmark of the unknown parameters and function $\mathcal{F}$ is defined by

$$\mathcal{F}(\mathsf{\Pi },s,x)={\mathcal{P}}_{\mathsf{\Omega }}((I+P){(BQB+R)}^{-1}(Bs-BQAx)),$$

(23)

with the projection operator

$${\mathcal{P}}_{\mathsf{\Omega }}(u)=arg\underset{\tilde{u}\in \mathsf{\Omega }}{min}∥u-\tilde{u}∥.$$

(24)

Proof. 

The vector form (22) is naturally derived from (16) with standard matrix calculations and derivations.    □

3.2. Gradient Descent Machine Learning

In industry, it is normal practice to record historical production information. Suppose that the historical data are recorded over m sample times under the free decision making condition, and the data set $\mathsf{\Lambda }$ with m elements is generated and stored. Each element $\lambda =(u,x,s)\in \mathsf{\Lambda }$ contains the decision, the state and the production demand, which are linked by the function $\mathcal{F}$ as

$$u=\mathcal{F}(\mathsf{\Pi },s,x)$$

(25)

according to Corollary 1. However, $\mathsf{\Pi }$ is unknown in the general sense, and the estimate $\widehat{\mathsf{\Pi }}=(\widehat{Q},\widehat{P})$ is employed in this paper to denote all unknown parameters within the function $\mathcal{F}(·)$. In this sense, an accurate estimate $\widehat{\mathsf{\Pi }}$ definitely provides a decision $\widehat{u}=\mathcal{F}(\widehat{\mathsf{\Pi }},s,x)$x), which approaches the real value u. This fact motivates the data-driven technique of machine learning to estimate the benchmark $\mathsf{\Pi }=(Q,P)$. The machine learning problem and its design objective are stated in the following definition.

Definition 2.

Themachine learning problemaims at iteratively updating the estimate $\widehat{\mathsf{\Pi }}$ such that the decision loss function

$$\mathcal{L}(\widehat{\mathsf{\Pi }},\lambda )=∥u-\mathcal{F}(\widehat{\mathsf{\Pi }},s,x)∥$$

(26)

is minimized for all $\lambda \in \mathsf{\Lambda }$.

In the above problem definition, the decision loss denotes the difference between the decision $\widehat{u}$ computed from the estimate $\widehat{\mathsf{\Pi }}$ and the recorded historical decision u. If the decision loss is sufficiently small, the estimated decision $\widehat{u}$ approximates u and the estimate $\widehat{\mathsf{\Pi }}$ is considered as an approximation of the benchmark $\mathsf{\Pi }$. Once an accurate estimate $\widehat{\mathsf{\Pi }}$ is available, more reliable model information becomes possible for the later MPC procedure as it considers the values of $\widehat{Q}$ while solving the optimization problem.

Furthermore, the decision loss function of each participant is defined as

$${\mathcal{L}}_i({\widehat{\mathsf{\Pi }}}_i,\lambda )=∥u_i-{\mathcal{F}}_i({\widehat{\mathsf{\Pi }}}_i,s,x)∥,$$

(27)

the sum of which gives rise to the overall decision loss function; i.e.,

$$\mathcal{L}(\widehat{\mathsf{\Pi }},\lambda )=\sum _{i=1}^n{\mathcal{L}}_i({\widehat{\mathsf{\Pi }}}_i,\lambda ).$$

(28)

The widely used method of machine learning is an iterative training procedure with several training epochs and an initial estimate

$${\widehat{\mathsf{\Pi }}}^0=({\widehat{Q}}^0,{\widehat{P}}^0).$$

(29)

This method randomly splits a total number of $\eta$ elements from the data set $\mathsf{\Lambda }$ into a training set ${\mathsf{\Lambda }}_{train}$, and the rest are sorted into a testing set ${\mathsf{\Lambda }}_{test}$[46]. At each training epoch, it updates the estimate $\widehat{\mathsf{\Pi }}$ using each element in the training set ${\mathsf{\Lambda }}_{train}$ to reduce the decision loss (26). At the end of each training epoch, the decision loss (26) of the elements in the testing set ${\mathsf{\Lambda }}_{test}$, i.e.,

$${\iota }_i^j=\mathcal{L}(\widehat{\mathsf{\Pi }},{\lambda }_i^j),{\lambda }_i^j\in {\mathsf{\Lambda }}_{test},i=1,\dots ,m-\eta ,$$

(30)

is computed to evaluate the training performance of the machine learning method; in other words, the decision loss, where the subscript $j\in \mathbb{N}$ denotes the training epoch number. Since the testing set ${\mathsf{\Lambda }}_{test}$ is independent of the training set ${\mathsf{\Lambda }}_{train}$, it is possible to use the evaluation results to double-check the training performance of the machine learning design using external data from the training set, which avoids the limitation of over-fitting.

A gradient descent machine learning training procedure is proposed to reduce the decision loss (26) with respect to the elements in the training set ${\mathsf{\Lambda }}_{train}^j$ during each training epoch. The training procedure of each training element is illustrated in the following theorem:

Theorem 2.

For $\lambda \in {\mathsf{\Lambda }}_{train}^j$, the estimate $\widehat{\mathsf{\Pi }}$ is updated by ${\widehat{\mathsf{\Pi }}}^{{}^{\prime }}=({\widehat{Q}}^{{}^{\prime }},{\widehat{P}}^{{}^{\prime }})$ using the gradient descent update

$$\left[\begin{array}{c}{\widehat{q}}_i^{{}^{\prime }}\\ {\widehat{p}}_i^{{}^{\prime }}\end{array}\right]=\left[\begin{array}{c}{\widehat{q}}_i\\ {\widehat{p}}_i\end{array}\right]-{\gamma }_i\left[\begin{array}{c}{\nabla }_{{\widehat{q}}_i}{\mathcal{L}}_i(·)\\ {\nabla }_{{\widehat{p}}_i}{\mathcal{L}}_i(·)\end{array}\right],i=1,\dots ,n,$$

(31)

such that the overall decision loss is reduced; i.e.,

$$\mathcal{L}({\widehat{\mathsf{\Pi }}}^{{}^{\prime }},\lambda )⩽\mathcal{L}(\widehat{\mathsf{\Pi }},\lambda ).$$

(32)

where ${\nabla }_{{\widehat{q}}_i}\mathcal{L}(·)$ and ${\nabla }_{{\widehat{p}}_i}\mathcal{L}(·)$ are partial derivatives of (26) with respect to ${\widehat{q}}_i$ and ${\widehat{p}}_i$, and ${\gamma }_i$ is the adaptive learning rate defined by

$${\gamma }_i={\alpha }^{{\beta }_i}\gamma ,$$

(33)

where $0<\alpha <1$ and $\gamma >0$ are constant scalars, and ${\beta }_i$ is the smallest non-negative integer such that

$${\mathcal{L}}_i({\widehat{\mathsf{\Pi }}}_i^{{}^{\prime }},\lambda )⩽{\mathcal{L}}_i({\widehat{\mathsf{\Pi }}}_i,\lambda ).$$

(34)

Proof. 

Since the individual decision loss reduction of all participants in (34) is guaranteed by the adaptive learning rate choice (33), the overall decision loss naturally reduces according to (28).    □

This gradient descent machine learning training procedure aims at updating the estimate $\widehat{\mathsf{\Pi }}$ to reduce the decision loss of all elements in the training set and obtains a reliable estimate of the unknown parameter values after sufficient training epochs.

Remark 2.

K-fold cross-validation is recommended if the amount of historical data is relatively small. Different splits of the data enable the cross validation using all elements within the historical data set [47]. The adaptive learning rate choice (33) is a mechanism used to ensure both the maximum gradient descent step size and the reduction of the decision loss. Other alternative learning rate choices can be made using bisection or back-stepping methods.

Remark 3.

For each element $\lambda \in \mathsf{\Lambda }$, there exist infinite choices for $\widehat{\mathsf{\Pi }}$ to reduce its decision loss $\mathcal{L}(\widehat{\mathsf{\Pi }},\lambda )$ to zero, but there only exists a rather small range for $\widehat{\mathsf{\Pi }}$ in the search space to obtain a sufficiently small overall decision loss. The trial and error method is used in (31) to reduce the decision loss of all $\lambda \in {\mathsf{\Lambda }}_{train}$ in a random order, with the aim of minimizing the overall decision loss. However, its convergence performance along with the training number cannot always be guaranteed due to the independence of the elements in Λ; i.e., an update of $\widehat{\mathsf{\Pi }}$ may reduce $\mathcal{L}(\widehat{\mathsf{\Pi }},{\lambda }_i)$ but increase $\mathcal{L}(\widehat{\mathsf{\Pi }},{\lambda }_{i+1})$.

Remark 4.

Note that other non-iterative methods, such as echo-state networks, extreme learning machines or kernel ridge regression, are also capable of solving the regression design objective shown in Definition 2.

Remark 5.

In more general production planning problem, due to different system dynamics, the function $\mathcal{F}(·)$ may become complex/fuzzy. To handle the machine learning problem, the deep neural network model is suggested to be utilized to estimate the unknown values.

3.3. MPC Production Planning Problem

The machine learning procedure in Section 3.2 is capable of determining the unknown parameter values for the production planning problem. Therefore, an analytic representation of the problem formulation becomes available, and the regulator can make the optimal decisions for the participants to improve the overall task performance. To solve the production planning problem, the technique of MPC is applied at each sample time index k to update the optimal decisions over a future time interval $[k,k+\mathsf{\Delta }k]$ by solving the optimization problem   

$$\begin{array}{cc}\hfill \underset{u}{min}& \sum _{i=k}^{k+\mathsf{\Delta }k-1}\frac{1}{2}{x}^{\top }(i+1|k)\widehat{Q}x(i+1|k)-s^{\top }(i+1)x(i+1|k)\hfill \\ \hfill & +\frac{1}{2}{u}^{\top }(i)Ru(i),\hfill \\ \hfill s.t.& x(i+1|k)=Ax(i|k)+Bu(i),u(i)\in \mathsf{\Omega },\hfill \\ \hfill & u=[u(k),u(k+1),\dots ,u(k+\mathsf{\Delta }k-1)],\hfill \end{array}$$

(35)

where $x(i+1|k)$ is the predicted state at sample time index $i+1$ based on the real information $x(k)$ at k and the planned input $u(i)$ over the period $[k,i]$. At each sample time, MPC uses the current input $u(k)$, the current state $x(k)$ and the system dynamics to predict the future states $x(i+1|k)$, $i=k,\dots ,k+\mathsf{\Delta }k-1$, over the prediction time period $[k+1,k+\mathsf{\Delta }k]$. An appropriate solution is chosen to optimize the cost function consisting of these predicted values and replace the previous decision (if it exists) with this new decision along with the future time period.

The optimal decisions aim at maximizing the overall net profit of the collaborative manufacturing task, while the individual decisions focus on the net profit of an individual participant. A comparison between these decisions is made in Section 5 via a case study, and the benefits of using the optimal decisions are illustrated in details.

4. Implementation Instructions

The instructions are provided in this section for implementing the solutions in Section 3, and a comprehensive algorithm is then proposed to provide a solution to the production planning problem with a collaborative mode and unknown information.

4.1. Instructions on Projection Solution

According to the definition (22) of the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}(·)$, the unconstrained decision is projected into the given decision constraint set $\mathsf{\Omega }$, which generates the constrained decision. The decision saturation constraint (4) is element-wise and the constraint handling procedure is straightforward. The next proposition provides the analytic solution of the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}(·)$ with respect to the decision saturation constraint (4) as an example.

Proposition 1.

If the constraint set Ω is in the form of the input saturation constraint (4), the analytic solution of the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}(·)$ defined in (22) is given by

$$u_i(k)=\left\{\begin{array}{c}0,\mathit{if}{\tilde{u}}_i(k)<0,\\ {\lambda }_i,\mathit{if}{\tilde{u}}_i(k)>{\overline{u}}_i,\\ {\tilde{u}}_i(k),\mathit{otherwise},\end{array}\right.i=1,\dots ,n.$$

(36)

Proof. 

As the input saturation constraint (4) is element-wise, the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}$ is computed by solving the problem

$$u_i(k)=\underset{\tilde{u}}{min}\left|u_i-{\tilde{u}}_i\right|,s.t.0⩽u_i⩽{\overline{u}}_i,$$

(37)

for $i=1,\dots ,n$, which yields the analytic solution (36).    □

The above proposition provides the analytic solution of ${\mathcal{P}}_{\mathsf{\Omega }}(·)$ based on the input saturation constraint (4). The solutions of the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}(·)$ with other types of input constraints can be handled in a similar way.

4.2. Instructions on Initial Estimate Choice

The machine learning training procedure is carried out in an iterative manner, and the decision loss function in (26) is generally non-linear and non-convex. Although the gradient descent training procedure (31) guarantees the reduction of the decision loss for the various possible initial estimate choices ${\widehat{\mathsf{\Pi }}}^0$, the initial estimate choice definitely affects the training performance. The estimate might converge to a local minimum point with a relatively large decision loss if the initial estimate is not chosen properly.

To achieve the desired training performance, the estimate search space is predefined as

$$\begin{array}{cc}\hfill \mathsf{\Theta }& =\{\widehat{\mathsf{\Pi }}\in {\mathbb{R}}^{n\times n}\times {\mathbb{R}}^{n\times n}:{\overline{q}}_i⩽{\widehat{q}}_i⩽{\underline{q}}_i,{\overline{p}}_i⩽{\widehat{p}}_i⩽{\underline{p}}_i\hfill \\ \hfill & i=1,\dots ,n\},\hfill \end{array}$$

(38)

where the scalars ${\overline{q}}_i$, ${\overline{p}}_i$, ${\underline{q}}_i$ and ${\underline{p}}_i$, $i=1,\dots ,n$, are appropriately chosen as the upper and lower bounds of the search space. In this sense, two different initial estimate choices are introduced, and the first one is defined by

Definition 3.

Thecentral initial estimate choiceis used to specify all initial values as the central values of their intervals; i.e.,

$$q_i^0=\frac{{\overline{q}}_i+{\underline{q}}_i}{2},p_i^0=\frac{{\overline{p}}_i+{\underline{p}}_i}{2},i=1,\dots ,n.$$

(39)

This choice ensures the initial distance to the benchmark $\mathsf{\Pi }$ does not appear to be too large and provides a relatively fair initial choice. Alternatively, the second choice is defined by

Definition 4.

Thegrid initial estimate choiceis used to specify the initial estimate choice as the best fitting solution to the grid search problem

$$\underset{\widehat{\mathsf{\Pi }}\in \breve{\mathsf{\Theta }}}{min}\sum _{i=1}^m\mathcal{L}(\widehat{\mathsf{\Pi }},{\lambda }_i),$$

(40)

where $\breve{\mathsf{\Theta }}$ is a discretized finite subset of Θ defined by

$$\begin{array}{cc}\hfill & \breve{\mathsf{\Theta }}=\{\widehat{\mathsf{\Pi }}\in {\mathbb{R}}^{n\times n}\times {\mathbb{R}}^{n\times n}:{\underline{q}}_i+\frac{m_q^i-0.5}{n_q}({\overline{q}}_i-{\underline{q}}_i),\hfill \\ \hfill & {\underline{p}}_i+\frac{m_p^i-0.5}{n_p}({\overline{p}}_i-{\underline{p}}_i),m_q^i\in [1,n_q],m_p^i\in [1,n_p],\hfill \\ \hfill & m_q^i,m_p^i\in \mathbb{Z},i=1,\dots ,n\}\hfill \end{array}$$

(41)

with sample numbers $n_q$ and $n_p$.

This word “grid” in the name of this choice implies that the sample numbers $n_q$ and $n_p$ should not be too large so that the total number of elements in $\breve{\mathsf{\Theta }}$ would not be excessive [48]. Therefore, the grid search problem (40) can be computed efficiently, capturing an approximate region of the global optimal solution to the minimum decision loss problem:

$$\underset{\widehat{\mathsf{\Pi }}\in \mathsf{\Theta }}{min}\mathcal{L}(\widehat{\mathsf{\Pi }},\lambda ),\forall \lambda \in \mathsf{\Lambda }.$$

(42)

However, this choice requires extra computational time to carry out the grid search as the number of the unknown parameters increases.

Remark 6.

These two initial choices both provide reliable initial values for the estimate for the gradient descent training procedure (31). However, a trade-off between computational efficiency and training performance improvement should be taken into account for practical implementation.

4.3. Instructions for Partial Derivative Estimation

Since the decision loss function $\mathcal{L}(·)$ has an analytic form (26), the value of its partial derivatives with respect to ${\widehat{q}}_i$ and ${\widehat{p}}_i$ can be computed using the syntax gradient in Matlab or autograd in Python. Alternatively, a numerical estimation

$$\begin{array}{c}\hfill {\nabla }_{{\widehat{q}}_i}\mathcal{L}(·)=\frac{\mathcal{L}(({\widehat{Q}}_i^{+},\widehat{P}),\lambda )-\mathcal{L}(({\widehat{Q}}_i^{-},\widehat{P}),\lambda )}{2\delta }\end{array}$$

(43)

is used to obtain the partial derivative, where

$$\begin{array}{cc}\hfill {\widehat{Q}}_i^{+}& =\mathbf{diag}({\widehat{q}}_1,{\widehat{q}}_2,\dots ,{\widehat{q}}_i+\delta ,\dots ,{\widehat{q}}_n),\hfill \\ \hfill {\widehat{Q}}_i^{-}& =\mathbf{diag}({\widehat{q}}_1,{\widehat{q}}_2,\dots ,{\widehat{q}}_i-\delta ,\dots ,{\widehat{q}}_n),\hfill \end{array}$$

and the scalar $\delta \in \mathbb{N}$ is sufficiently small. The partial derivative ${\nabla }_{{\widehat{p}}_i}\mathcal{L}(·)$ can be obtained in a similar way. This numerical estimation incurs less computational load and is still capable of giving approximate values of the derivatives even if the analytic representation of the decision loss function $\mathcal{L}(·)$ is not available in some extreme cases.

4.4. Instructions for MPC Problem Solution

The MPC problem (35) has a quadratic cost function, linear equality constraint and convex inequality constraint. Therefore, it is a quadratic programming problem, which is a convex optimization problem with a global optimal solution. There exist various standard solvers for quadratic programming problems, such as the solver quadprog in MATLAB. These standard solvers can directly provide the global optimal solution.

Furthermore, an alternative computational efficient solution is obtained by decoupling the input constraint from the optimization problem (35) and setting the prediction interval to a unit sample time; i.e., $\mathsf{\Delta }k=1$. The solution to the problem (35) is described as first solving the unconstrained problem

$$\begin{array}{cc}\hfill & \underset{u}{min}\frac{1}{2}x{(k+1|k)}^{\top }\widehat{Q}x(k+1|k)-s^{\top }(k+1)x(k+1|k)\hfill \\ \hfill & +\frac{1}{2}{u}^{\top }Ru,s.t.x(k+1|k)=Ax(k)+Bu,\hfill \end{array}$$

(44)

and projecting the solution into the input constraint set $\mathsf{\Omega }$ using ${\mathcal{P}}_{\mathsf{\Omega }}(·)$, which is illustrated as follows:

Proposition 2.

If the input constraint is decoupled from the problem (35) and $\mathsf{\Delta }k=1$, its alternative solution is

$$u(k)={\mathcal{P}}_{\mathsf{\Omega }}(\tilde{u}(k)),$$

(45)

where $\tilde{u}(k)$ is the solution of (44); i.e.,

$$\tilde{u}(k)={(B\widehat{Q}B+R)}^{-1}(Bs(k+1)-B\widehat{Q}Ax(k)).$$

(46)

Proof. 

As the input constraint is decoupled and $\mathsf{\Delta }k=1$, the problem (35) collapses to the problem (44). The problem (44) becomes an unconstrained problem by substituting the equality constraint $x(k+1|k)=Ax(k)+Bu$ into the cost function, which yields the solution (46). To handle the input constraint, the projection operator ${\mathcal{P}}_{\mathsf{\Omega }}(·)$ is considered to give the solution (46).    □

The alternative solution to problem (35) in Proposition 2 is computationally more efficient than the direct solution provided by the standard solvers. A trade-off between efficiency and accuracy should be considered by the production regulator according to the properties of the exact production planning scenario.

4.5. Production Planning Comprehensive Algorithm

The techniques of machine learning and MPC are combined to yield a comprehensive algorithm (Algorithm 1) to solve the production planning problem with a collaborative mode and unknown information. This algorithm first uses the gradient descent machine learning procedure to estimate the unknown parameters and applies MPC to obtain the optimal decisions for the participants at each sample time index k. Note that the scalar value N denotes the total number of sample times and $ϵ>0$ is a small scalar depending on the accuracy requirement of machine learning estimation. For a straightforward view, a block diagram of the machine learning-based MPC algorithm is shown in Figure 1.

Algorithm 1 Machine learning based MPC

Input: System dynamics (1), cost function $f(u,x,k)$, production demand $s(k)$, initial state $x_0$, initial estimate ${\widehat{\mathsf{\Pi }}}^0$, decision loss function $\mathcal{L}(·)$ and historical data set $\mathsf{\Lambda }$.

Output: Estimate $\widehat{Q}$, optimal decision $u(k)$.

1: initialization: Set training epoch number $j=1$, sample time index $k=0$ and randomly split $\eta$ elements of the set $\mathsf{\Lambda }$ to a training set ${\mathsf{\Lambda }}_{train}$ and the rest to a testing set ${\mathsf{\Lambda }}_{test}$.

2: while not ${\sum }_{i=1}^{m-\eta }{\iota }_i^j⩽ϵ$ do

3: Perform the training procedure (31) to update the estimate $\widehat{\mathsf{\Pi }}$ using all elements in ${\mathsf{\Lambda }}_{train}$.

4: Perform the evaluation (30) to compute the decision loss ${\iota }_i^j$, using all elements in ${\mathsf{\Lambda }}_{test}$.

5: Set $j=j+1$ to the next training epoch.

6: end while

7: Set the estimate $\widehat{Q}$ as the unknown matrix Q.

8: while $k<N$ do

9:   Solve the MPC problem (35) to obtain $u(k)$.

10: Record the optimal decisions $u(k)$.

11: Set $k=k+1$ to the next sample time index.

12: end while

13: return $\widehat{Q}$, $u(k)$, $k=0,\dots ,N-1$

5. Simulation-Based Case Study

In this section, a case study of a production planning problem is presented, and an evaluation is conducted to check the performance of Algorithm 1 in comparison to the performance under the conditions of individual decision making and pure MPC decision making.

5.1. Problem Design Specifications

In this paper, we consider the production planning scenario within a clothing factory as a case study. This factory has a total number of 10 departments ($n=10$; i.e., 10 participant) working in a collaborative mode. The modified dynamics of the discrete time system (1) of this factory are considered with

$$\begin{array}{cc}\hfill A& =\left[\begin{array}{cccccccccc}0.72& 0.00& 0.00& 0.00& 0.00& 0.05& 0.00& 0.00& 0.00& 0.00\\ 0.00& 0.53& 0.07& 0.00& 0.01& 0.10& 0.00& 0.00& 0.13& 0.00\\ 0.00& 0.00& 0.64& 0.00& 0.02& 0.00& 0.00& 0.00& 0.03& 0.00\\ 0.00& 0.02& 0.00& 0.28& 0.00& 0.08& 0.00& 0.03& 0.00& 0.00\\ 0.00& 0.00& 0.04& 0.00& 0.48& 0.00& 0.00& 0.00& 0.00& 0.01\\ 0.00& 0.15& 0.00& 0.00& 0.00& 0.33& 0.04& 0.00& 0.00& 0.16\\ 0.00& 0.00& 0.00& 0.34& 0.00& 0.00& 0.67& 0.00& 0.00& 0.00\\ 0.00& 0.23& 0.00& 0.00& 0.06& 0.00& 0.00& 0.41& 0.00& 0.07\\ 0.02& 0.00& 0.16& 0.00& 0.00& 0.04& 0.00& 0.00& 0.18& 0.00\\ 0.00& 0.04& 0.00& 0.00& 0.00& 0.07& 0.00& 0.00& 0.00& 0.19\end{array}\right],\hfill \\ \hfill B& =\mathbf{diag}(3.62,4.46,2.31,2.21,2.05,2.53,4.99,7.68,0.29,1.57).\hfill \end{array}$$

The above system dynamics demonstrate how the productivity decision of an identical participant affects the market size of other participants. The sample time period is one day, which means the regulator needs to plan productivity decisions one day in advance. The total number of sample times N is 365; i.e., one year. The initial state value is chosen as $x(0)=0$, and the input saturation constraint (4) is considered with ${\overline{u}}_i=1.5$ as the productivity of the participants with a maximum load. The benchmark values of matrices R and Q are listed as

$$\begin{array}{cc}\hfill R& =\mathbf{diag}(0.93,3.55,3.54,1.51,0.56,1.46,2.17,2.38,4.14,3.21),\hfill \\ \hfill Q& =\mathbf{diag}(1.03,2.35,1.14,3.44,5.01,0.86,1.47,1.68,1.96,2.41).\hfill \end{array}$$

The individual decision bias parameters $p_i$, $i=1,\dots ,n$ in matrix P are defined with mean values

$$\begin{array}{cc}\hfill & {\mu }_1=-0.30,{\mu }_2=-0.15,{\mu }_3=-0.55,{\mu }_4=-0.35,{\mu }_5=-0.25,\hfill \\ \hfill & {\mu }_6=0.60,{\mu }_7=-0.55,{\mu }_8=-0.35,{\mu }_9=0.15,{\mu }_{10}=0.60.\hfill \end{array}$$

as well as a standard deviation of ${\sigma }_i=0.01$. Since the parameters in the matrix Q are unknown to the regulator, the authors only apply these values as a benchmark to generate historical data and evaluate the estimation accuracy of the machine learning procedure. The same argument holds for the unknown matrix P.

In addition, the production demand of each department over the total decision making period $[0,365]$ is described as

$$s_i(k)={\kappa }_{i}sin(\frac{2\pi }{N}k+{\phi }_i)+{\tau }_i,i=1,\dots ,n,$$

(47)

where the amplitude ${\kappa }_i$, center amplitude ${\tau }_i$ and the phase ${\phi }_i$ are the elements of the vectors as follows:

$$\begin{array}{cc}\hfill \kappa & =[1.43,3.17,3.81,1.91,4.13,2.72,2.24,4.22,1.06,4.59],\hfill \\ \hfill \tau & =[7.02,6.92,4.15,5.29,4.53,3.62,8.79,5.22,6.11,9.52],\hfill \\ \hfill \phi & =[\frac{1}{2}\pi ,\frac{1}{2}\pi ,\frac{1}{2}\pi ,\frac{1}{2}\pi ,\frac{1}{2}\pi ,-\frac{1}{2}\pi ,-\frac{1}{2}\pi ,-\frac{1}{2}\pi ,-\frac{1}{2}\pi ,-\frac{1}{2}\pi ].\hfill \end{array}$$

This case study focuses on clothing manufacturing, so the production demand varies for different seasons in a calendar year. In this sense, the sinusoidal function (47) can reasonably represent the production demands of these departments. These seasonally varying demands contribute to the historical demand within the set $\mathsf{\Lambda }$ and are used to compute the optimal decisions in Section 5.2 and Section 5.3.

These problem design specifications fully replicate the industrial environment of cloth manufacturing. Therefore, the simulation results are capable of indicating the practical effectiveness and feasibility of the proposed algorithm.

5.2. Parameter Estimation Using Machine Learning

To obtain the historical data, the following data sampling procedure was employed: first of all, the benchmark matrices Q and P defined in Section 5.1 generated the individual decisions using the Formula (22) over a production planning period $[1,365]$, denoting a calendar year. Therefore, the historical data set $\mathsf{\Lambda }$ was established with a total number of 365 groups of historical data; i.e., ${\lambda }_i=(s_i,u_i,x_i)$, $i=1,\dots ,365$ ($m=365$). In this case study, the groups of data generated as above replicated the interaction between the production demands, individual decisions and market sizes of each department under the free decision making condition and were thus regarded as effective historical data for the later machine learning procedure.

The machine learning procedure of Algorithm 1 randomly split a total number of 100 ($\eta =100$) groups of the historical data into a training set ${\mathsf{\Lambda }}_{train}$ and the rest into a testing set ${\mathsf{\Lambda }}_{test}$. Then, it was capable of estimating the unknown parameter values of Q and P based on the obtained training set ${\mathsf{\Lambda }}_{train}$. The grid initial estimate choice in Definition 4 was carried out in the search space (38) defined by

$${\overline{q}}_i=10,{\overline{p}}_i=1,{\underline{q}}_i=0,{\underline{p}}_i=-1,$$

(48)

for $i=1,\dots ,10$. The values $\alpha =0.6$ and $\gamma =0.05$ were used to determine the adaptive learning rate choice (33).

The decision loss during the training procedure was plotted along the training number line for each department in Figure 2 and all asymptotically converged to sufficiently small values. Meanwhile, the decision loss of each department was computed using the testing data at the end of each training epoch and plotted as the colored lines in Figure 3. Note that all lines in this figure also asymptotically converged to sufficiently small values. This means that the estimated decision $\widehat{u}=\mathcal{F}(\widehat{\mathsf{\Pi }},s,x)$ gradually approached the real decision u, and thus the training procedure succeeded in providing an appropriate estimate $\widehat{\mathsf{\Pi }}$. The above evaluation demonstrates the feasibility and reliability of the machine learning procedure in terms of validation using testing data. The choice of the initial estimate affected the convergence performance of the 10 departments. For instance, the estimated values of the fourth department were sufficiently close to the benchmark values, and the convergence rate was much slower along the training epochs in comparison to others.

Furthermore, the updated estimate $\widehat{\mathsf{\Pi }}$ was compared with the benchmark values for each training update, and the results of the first department are plotted in Figure 4. For comparison, the corresponding benchmark values are plotted in the same figure as the dashed magenta lines (the mean value ${\mu }_1$ of $p_i$ is used for comparison). It is observed that the estimated values asymptotically converged to their benchmark values, which means that the converged estimate matched well with its benchmark. The converged estimates were

$$\begin{array}{cc}\hfill \widehat{Q}& =\mathbf{diag}(1.03,2.35,1.14,3.44,5.01,0.86,1.47,1.68,1.96,2.41),\hfill \\ \hfill \widehat{P}& =\mathbf{diag}(-0.30,-0.15,-0.55,-0.35,-0.25,0.60,-0.55,-0.35,0.15,0.60),\hfill \end{array}$$

which approached the benchmark values shown in Section 5.1. Therefore, the above results reveal the effectiveness of the machine learning procedure for estimating the unknown parameters. The estimated convergence performances of other departments were similar to those in Figure 4 and are thus omitted here for brevity.

In practice, the historical data of the production information should be recorded over a past time period rather than artificially generated using the benchmark, as in this case study. The performance of the machine learning procedure should be evaluated based on the convergence performance of the decision loss with respect to the testing data.

5.3. Decision Making Using MPC

Since the machine learning procedure in Section 5.2 provided an accurate estimate $\widehat{\mathsf{\Pi }}$ of the unknown parameter values, the information within the production problem made it feasible for the regulator to make the optimal decisions. The MPC decision making procedure of Algorithm 1 was performed for the production planning problem specified in Section 5.1 with a unit prediction interval ($\mathsf{\Delta }T=1$), using the given system dynamics (1) and the desired cost function (8).

The performance of the MPC decision making procedure was evaluated over the whole production planning period $[1,365]$, and the overall net profits of the factory at each sample time index are plotted as the magenta curve in Figure 5. Note that the profit in winter was approximately $30\%$ higher than that in summer. The net profit caused by the individual decisions was computed for the same production planning problem, and the pure MPC approach was utilized for this problem with an inaccurate estimate of a $\pm 50\%$ random variance to the benchmark values. These results are also plotted in this figure as the blue and red curves, respectively. These results suggest that the machine learning (ML)-based MPC decisions are able to increase the profit by over $10\%$ compared to the profit of the individual biased decisions. In comparison to the pure MPC decisions, it fully employs the technique of machine learning to obtain an accurate estimate of unknown parameters, increasing the annual profit by around $5\%$.

Furthermore, the optimal decisions of each department are plotted as the colored curves in Figure 6. Note that the optimal decisions were all within the saturation range $[0,1.5]$, which means that Algorithm 1 could handle the input constraint appropriately. Furthermore, the optimal decisions had a causal relationship with the production demand (47). In other words, a higher production demand encourages a larger decision value (production effort).

From the simulation results, Algorithm 1 can be seen to provide the optimal decisions within the constraint at each sample time index to increase the net profit. Therefore, Algorithm 1 achieved the desired practical benefits in solving the production planning problem. Note that the authors employed the alternative solution in Proposition 2 to solve problem (35), providing similar results to those obtained from quadprog.

6. Conclusions and Future Work

This paper focuses on the exact production planning problem with a collaborative mode and unknown information and proposes a machine learning-based MPC algorithm to solve this problem. This problem is defined using a mathematical form using a discrete time system and a net profit cost function. This paper records the historical data under the free decision making condition with a certain degree of decision bias to establish a dataset. A gradient descent machine learning procedure is proposed to estimate the unknown parameter values based on the elements in the dataset, and MPC uses the estimated parameters to make the optimal decisions by solving a quadratic optimization problem at each sample time index. These procedures together yield a comprehensive algorithm for this specific class of production planning problems. The algorithm is validated using a simulation-based case study to check its parameter estimation accuracy and task performance. In addition, a comparison with individual decision making and pure MPC decision making is conducted to show its advantages in terms of increasing profit.

Although the efficiency and effectiveness of the proposed algorithm are shown in this paper, some potential extensions can be made to strengthen the generality and reliability of this method. First of all, a general non-linear system model can be used to broaden the generality to estimate unknown information for other control methods, such as iterative learning control [49,50], and a target task performance other than the net profit is suggested to be considered to achieve alternative benefits. Moreover, a research study on a system with complex dynamic interactions among the participants and associated applications—e.g., a large scale non diagonal Q matrix—are under consideration for future works. Last but not least, other alternative frameworks, such as a real-time optimization framework and Economic MPC, will be further explored to confirm the wide applications of the proposed algorithm in the area of production planning.