Uniqueness of Optimal Power Management Strategies for Energy Storage Dynamic Models

Abstract

This paper contributes to the field of analytic and semi-analytic solutions for optimal power flow problems involving storage systems. Its primary contribution is a rigorous proof establishing the uniqueness of the “shortest path” optimal solution, a key element in this class of algorithms, building upon a graphical design procedure previously introduced. The proof is constructed through five consequential lemmas, each defining a distinct characteristic of the optimal solution. These characteristics are then synthesized to demonstrate the uniqueness of the optimal solution, which corresponds to the shortest path of generated energy within defined bounds. This proof not only provides a solid theoretical foundation for this algorithm class but also paves the way for developing analytic solutions to more complex optimal control problems incorporating storage. Furthermore, the efficacy of this unique solution is validated through two comparative tests. The first one uses synthetic data to benchmark the proposed solution in comparison to recent reinforcement learning algorithms, including actor–critic, PPO, and TD3. The second one compares the proposed solution to the optimal solutions derived from other numerical methods based on real-world data from an electrical vehicle storage device.

1. Introduction

Energy storage devices are emerging as an essential component in modern power systems [1,2]. These devices can be used in many applications to increase efficiency, reduce operation costs, limit the peak power generation, or increase stability. However, the optimal management of storage devices is often very challenging, since the optimal energy curve must be evaluated in the time-domain and therefore always involves a high-dimensional solution space.

As indicated in [3], many methods and algorithms have been developed to solve storage system optimization problems. For instance, linear programming approaches are used that are relatively simple and intuitive, relying on low computational resources but limited to problems in which the objective function is linear and the constraints are either linear equalities or linear inequalities. Work [4] aims to minimize the daily electricity cost of a campus central chiller plant, which includes electric chillers and thermal energy storage, by employing a hybrid optimization approach. Specifically, mixed-integer linear programming determines the optimal chiller operation, while dynamic programming manages the thermal energy storage. Alternatively, work [5] investigates the optimization of an islanded microgrid, incorporating renewable energy sources, diesel generators, and battery energy storage systems through the application of linear and mixed-integer programming. The linear programming formulation aims to minimize diesel generator power output while the mixed-integer programming formulation seeks to minimize total operational cost, with both formulations including battery state-of-charge constraints.

Dynamic programming is an example for traditional optimization method vastly in use. This method divides complex problems into a series of simpler sub-problems, with low computational overhead each, and solves them recursively. For example, in the energy market category, work [6] is concerned with minimizing the average cost of electricity used and investment in storage while satisfying demand in the dynamic pricing problem. The objective of [7] is to calculate the optimal control of a residential ESS, with and without local generation in real-time pricing problems. To address uncertainties, there are several approaches for deciding upon the optimal control. For example, Model Predictive Control (MPC) is one dominant method used in managing storage devices, while another involves framing the problem as a Markov Decision Process (MDP) and resolving it through stochastic dynamic programming. Unlike classical dynamic programming, which focuses on optimizing control policies for predetermined signals, stochastic dynamic programming focuses on optimizing control policies across a spectrum of potential signals. Consequently, rather than yielding a singular optimal solution, the algorithm produces an optimal control strategy that adapts in real time, according to actual measurements. On top of that, robust control techniques, particularly sliding mode control and its higher-order variants, are also gaining extensive attention. sliding mode control, in essence, is a nonlinear control technique designed to force a system’s state to reach and remain on a predefined surface. First, a mathematical function that defines the desired system behavior is formulated. Once the system state reaches this surface, it remains there despite disturbances. Following, the control law is designed to drive the system state from any initial condition to the sliding surface. Once on the sliding surface, the system follows a reduced-order dynamics known as sliding mode dynamics. The system becomes insensitive to parameter variations and disturbances, leading to high robustness. Several examples include [8] that introduces a super-twisting sliding mode control (STSMC) strategy for grid-side inverters (GSIs) in wind power generation systems to improve power delivery under unpredictable weather. The proposed method employs three integral-type STSMC controllers within a multi-loop GSI model to regulate active and reactive power exchange, mitigating chattering and improving robustness against parameter variations. Simulation and experimental results demonstrate that the STSMC significatnly improves the active/reactive power response and stability of three-phase voltage/current signals, with reduced harmonic content compared to proportional-integral control. Similarly, work [9] proposes a universal controller for battery energy storage systems in microgrids that uses impedance shaping and sliding mode control to address issues with grid-feeding and grid-forming voltage source converters. The controller adapts to different operating modes, improving performance in weak grid connections and during fault ride-through transients, while also ensuring a smooth transition between grid-connected and islanded modes. The proposed control method was evaluated through simulations in MATLAB R2024a Simulink. Different works utilize stochastic control methods for grid-connected storage device control problems. For instance, work [6] aims to minimize the average cost of electricity used and investment in storage while satisfying demand. Pontryagin’s Minimum Principle is yet another technique that is often used for managing storage devices. Various works leverage this powerful mathematical principle. As an example, if one examines control problems for grid-connected devices, ref. [10] or [11] focus on optimizing revenue from generated electricity in concentrated solar power plants. In [12], the objective is to calculate the optimal total expenses of a capacitor-type energy system. From a slightly different perspective, works [13,14] focus on managing and optimizing the power flows in the power system of a hybrid vehicle. The authors model the control problem as a constrained optimization problem and use Pontryagin’s Minimum Principle approach to solve it. In this work, a multi-objective optimization problem is considered, aiming to minimize both energy efficiency, in the sense of overall cost, and pollutant emissions for a given driving cycle. The model is tested in a simulated environment.

Furthermore, with the surge of machine learning tools and recent advancements in computational power, there is an emerging trend of using machine learning, specifically reinforcement learning algorithms, for storage management problems. For instance, ref. [15] introduces a deep reinforcement learning framework for optimizing battery storage, incorporating a lithium-ion battery degradation model and a noisy network architecture for effective action space exploration. In [16], Double Deep Q-Learning is used to optimize community battery storage in microgrids. More relevant works are [17,18].

It is evident from the literature review that various optimization methods have been explored for managing energy storage systems, each with its own strengths and limitations. For instance, dynamic programming suffers from the “curse of dimensionality”, making it computationally expensive for large-scale applications. Pontryagin’s Minimum Principle provides an analytical framework for solving optimal control problems, yet it relies on precise system modeling, making it less flexible in handling uncertainties and stochastic variations in energy demand and generation. Additionally, reinforcement learning has gained popularity for its ability to adapt to unknown system dynamics, yet model-free RL struggles with long-term dependencies, constraint enforcement, and convergence stability, especially in highly variable environments.

The shortest path method [19] offers a complementary approach that serves as an intuitive benchmark for evaluating energy storage optimization techniques. Unlike RL, PMP, or DP, the shortest path method provides an approximate but interpretable solution based on a graphical representation of energy trajectories. While it does not incorporate detailed system constraints or real-world transients, its smooth trajectories help identify structural properties of optimal solutions, making it a valuable qualitative tool. By comparing advanced algorithms against the shortest path benchmark, researchers can pinpoint areas where additional complexity is necessary, guiding the development of more computationally efficient and real-world-applicable optimization strategies.

In this light, the current paper continues several recent works that study optimal power management for energy storage dynamic models. Original work [19] provides a graphical design procedure that reveals the optimal power flow solution in a system that comprises a load and a network-connected storage device. Later works building upon this result provide a deeper understanding and intuitive characteristics of the optimal solution in several cases, such as optimal peak-shaving, scenario with storage devices connected by a network, and an extension of this initial result for cases of stochastic load profiles. However, the basic graphical design procedure presented in the initial work [19] was never proven in a rigorous manner.

Therefore, the purpose of this paper is to bridge this gap and to provide a mathematically complete proof for the central theorem in [19]. Nonetheless, this proof provides not only a sound theoretical foundation for this class of algorithms, but may also help develop additional analytic solutions for more complex optimal control problems involving storage systems. This unique solution is validated via two distinct comparative tests. The first one uses synthetic data to benchmark the proposed solution in comparison to recent reinforcement learning algorithms, including actor–critic, PPO, and TD3. The second one compares the proposed solution to the optimal solutions derived from other numerical methods based on real-world data from an electrical vehicle storage device. This comparative analysis emphasizes how the shortest path method can be used to learn about the performance bounds of other more advanced methods, since, as discussed in previous works, the shortest path method does indeed have the advantage of minimal computational complexity, thus it can serve as a benchmark for other methods, for instance, those new methods that make use of reinforcement learning.

It is important to note that while “uniqueness” often implies both exclusivity and optimality, in this case, we refer strictly to the mathematical uniqueness of the solution rather than its real-world superiority over alternative methods. The shortest path method, while proven to yield a unique solution within the given framework, does not necessarily mean that it is the most effective approach in practical applications, where additional constraints, uncertainties, and real-world imperfections may impact performance. The structure of the article is presented in Figure 1.

2. Main Result

The original problem explored in [19] concerns with continuous time, finite-horizon optimization problem that may be concisely formulated as follows:

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\int }_{\tau =0}^{T}F\left(P_g\left(\tau \right)\right)\mathrm{d}\tau ,\hfill \\ \hfill \mathrm{subject}\mathrm{to}{W}_g\left(t\right)& ={\int }_0^T{P}_g\left(\tau \right)\mathrm{d}\tau ,\hfill \\ \hfill W_L\left(t\right)& \le W_g\left(t\right)\le W_L\left(t\right)+W_{max},\hfill \\ \hfill W_g\left(0\right)& =W_L\left(0\right),\hfill \\ \hfill W_g\left(T\right)& =W_L\left(T\right),\hfill \end{array}$$

(1)

where $F\left(P_g\right)$ is a general convex cost function, $P_g\left(t\right)$ is the generated power, $W_L\left(t\right)$ is the energy consumption of the load, $W_g\left(t\right)$ is the generated energy, $[0,T]$ is the integration interval, where T is known, and $W_{max}$ is the capacity of the storage device. The solution for this problem may be obtained by the “shortest path” method, as explained in [19]. Our main objective in this paper is to provide an analytical and rigorous proof for this result, as formulated by the following theorem:

Theorem 1

(Unique Solution). Consider the optimization problem described in (1). If $F(·)$ is convex, then this problem has a unique solution $W_g^{*}\left(t\right)$, and this solution is independent of $F(·)$.

Proof. 

First, recall two lemmas from [19]:

Lemma 1

(Straight Flow (SF), [19]). If $\exists [a,b]\subseteq [0,T]$ such that $\forall t\in (a,b):W_L\left(t\right)<W_g\left(t\right)<W_L\left(t\right)+W_{max}$, then the optimal generated power $P_g\left(t\right)$ is constant in $[a,b]$ (when the generated energy is not constrained, it is a straight line).

Lemma 2

(Tangent And Continuous (TC), [19]). The curve of optimal generated energy must tangent its bounding constraints. In other words, at each time when the generated energy meets the constraints, the generated power is equal to the load power. Formally, $W_g\left(t\right)=W_L\left(t\right)\Rightarrow P_g\left(t\right)=P_L\left(t\right)$ and $W_g\left(t\right)=W_L\left(t\right)+W_{max}\Rightarrow P_g\left(t\right)=P_L\left(t\right)$. Moreover, the optimal generated power, $P_g\left(t\right)$, is a continuous function.

The aforementioned lemmas can be interpreted as follows:

• When not bounded, the generated energy is a straight line.
• When bounded, the generated energy is tangent to the constraint.

In [19], those two lemmas were used to assert that there is only one legal solution that includes both features, so together, the two features define the optimal solution. These lemmas tell us what happens to the generated energy when it is bounded and what happens when it is not. To complete the proof, we need to show that the times when the generated energy is bounded are unique.

The outline of the proof is depicted in Figure 2. To prove the uniqueness of the generated energy, additional lemmas are presented that serve as the basis of our proof. First, let us introduce the first added lemma, which is denoted by lemma number three, which shows that we can split the timeline into segments. Those intervals divide the power demand of the load into segments where this function is monotonically increasing or decreasing, as can be viewed in Figure 3. Moreover, inside those intervals, the optimal solution can be tangent to an adequate constraint; upper bound in an increasing interval and lower bound in a decreasing interval. The solution is tangential continuously to the constraint for a finite interval. One may conclude that the optimal solution can be described by the MI on which it tangents one of the constraints. As seen in Figure 2, the third lemma is based on the two lemmas from [19]. Finally, two additional new lemmas, denoted as the fourth and fifth lemmas, demonstrate that the times in which the optimal generated energy is bounded are unique. To prove this, it must be shown that the optimal generated energy is bounded inside a unique MI, which sets the unique times as required. Presented in Figure 2, those lemmas rely on the third lemma. Consequently, we combine the fourth and fifth lemmas to show the uniqueness of the optimal generated energy.

Let us set a new framework and present some formal definitions of the varied segments and points that characterize the load power consumption $P_L\left(t\right)$ and the optimal solution $W_g\left(t\right)$.

Definition 1

(Increasing Interval). A real valued function $f\left(t\right):[0,T]\to \mathbb{R}$ has an increasing interval $[c,d]\subseteq [0,T]$ if

1.

The function is increasing in $[c,d]$:

$\forall x,y\in [c,d]:x\le y\Rightarrow f\left(x\right)\le f\left(y\right)$.

2.

If the point $t=c$ is not at the edge of the domain of definition of the function, then the function is decreasing on the interval [0,c):

$c\ne 0\Rightarrow \exists {\delta }_1>0,\forall t\in [c-{\delta }_1,c):f\left(t\right)>f\left(c\right)$.

3.

If the point $t=d$ is not at the edge of the domain of definition of the function, then the function is decreasing from some point inside the interval $(d,T]$:

$d\ne T\Rightarrow \exists {\delta }_2>0,\forall x\in (d,d+{\delta }_2]:f\left(x\right)\le f\left(d\right)\wedge \exists y\in (d,d+{\delta }_2]:f\left(y\right)<f\left(d\right)$.

Definition 2

(Decreasing Interval). A real-valued function $f\left(t\right):[0,T]\to \mathbb{R}$ has a decreasing interval $[c,d]\subseteq [0,T]$ if

1.

The function is decreasing in $[c,d]$:

$\forall x,y\in [c,d]:x\le y\Rightarrow f\left(x\right)\ge f\left(y\right)$.

2.

If the point $t=c$ is not at the edge of the domain of definition of the function, then the function is increasing on the interval [0,c):

$c\ne 0\Rightarrow \exists {\delta }_1>0,\forall t\in [c-{\delta }_1,c):f\left(t\right)<f\left(c\right)$.

3.

If the point $t=d$ is not at the edge of the domain of definition of the function, then the function is increasing from some point inside the interval $(d,T]$:

$d\ne T\Rightarrow \exists {\delta }_2>0,\forall x\in (d,d+{\delta }_2]:f\left(x\right)\ge f\left(d\right)\wedge \exists y\in (d,d+{\delta }_2]:f\left(y\right)>f\left(d\right)$

An illustration of those definitions can be seen in Figure 3.

Definition 3

(Monotone Interval). A real-valued function $f\left(t\right):[0,T]\to \mathbb{R}$ has a monotone interval $[c,d]\subseteq [0,T]$ if $[c,d]$ is either a decreasing or an increasing interval of $f\left(t\right)$.

Definition 4

(Union Point). Continuous real valued functions $g\left(t\right), f\left(t\right):[0,T]\to \mathbb{R}$ have a union point in $t_m\in [0,T]$ if:

1.

$\exists {\delta }_1>0:\forall t\in [t_m-{\delta }_1,t_m),f\left(t\right)\ne g\left(t\right)$,

2.

$\exists {\delta }_2>0:\forall t\in [t_m,t_m+{\delta }_2],f\left(t\right)=g\left(t\right)$.

Definition 5

(Separation Point). Continuous real valued functions $g\left(t\right), f\left(t\right):[0,T]\to \mathbb{R}$ have a separation point in $t_s\in [0,T]$ if

1.

$\exists {\delta }_1>0:\forall t\in [t_s-{\delta }_1,t_s], f\left(t\right)=g\left(t\right)$,

2.

$\exists {\delta }_2>0:\forall t\in (t_s,t_s+{\delta }_2], f\left(t\right)\ne g\left(t\right)$.

2.1. The Third Lemma

Lemma 3

(Up Increasing, Down Decreasing (UIDD)). We let $W_g\left(t\right)$ be an optimal generated energy for (1).

If $\exists t_0:W_g\left(t_0\right)=W_L\left(t_0\right)+W_{max}$, then $\exists t_1<t_2\in [0,T]$ such that $t_0\in [t_1,t_2],\forall t\in [t_1,t_2]:W_g\left(t\right)=W_L\left(t\right)+W_{max}$ and $\exists [a,b)$ such that $[t_1,t_2]\subseteq [a,b)$ and $[a,b)$ is an increasing interval of $P_L\left(t\right)$.

Similarly, if $\exists t_3:W_g\left(t_3\right)=W_L\left(t_3\right)$, then $\exists t_4<t_5\in [0,T]$ such that $t_3\in [t_4,t_5],\forall t\in [t_4,t_5]:W_g\left(t\right)=W_L\left(t\right)$ and $\exists [c,d]$ such that $[t_4,t_5]\subseteq [c,d]$ and $[c,d]$ is a decreasing interval of $P_L\left(t\right)$.

Proof. 

First, let us prove that $\nexists t_0\in [0,T]$ such that $t_0$ is a part of an increasing interval of $P_L\left(t\right)$ and $W_g\left(t_0\right)=W_L\left(t_0\right)$.

We assume by negation that $\exists [a,b]\subseteq [0,T]$ which is an increasing interval of $P_L\left(t\right)$, and for which $\exists t_0\in [a,b]$ such that an optimal generated energy $W_g\left(t\right)$ has $W_g\left(t_0\right)=W_L\left(t_0\right)$. We can establish two opposing facts about $W_g\left(t\right)$ behavior and reach a contradiction.

$$\forall t\in [a,b]:W_g\left(t\right)=W_L\left(t\right).$$

(2)

Without loss of generality, we assume by negation $\exists t_1\in [a,b]$ such that $t_1>t_0$, $W_g\left(t_1\right)>W_L\left(t_1\right)$. Because $P_L\left(t\right)$ is increasing on $[t_0,t_1]\subseteq [a,b]$, we know $W_L\left(t\right)$ is convex in this section, and therefore the line connecting $W_L\left(t_0\right)=W_g\left(t_0\right)$ and $W_g\left(t_1\right)$ located above $W_L\left(t\right)$. Knowing that $W_g\left(t\right),W_L\left(t\right)$ are antiderivatives, they are continuous, and therefore $(W_g-W_L)\left(t\right)$ is continuous. This means (from the intermediate point theorem) that $\exists t_{sep}\in [t_0,t_1]$ such that 

$$\begin{array}{cc}\hfill \forall t& \in [t_0,t_{sep}]:W_g\left(t\right)-W_L\left(t\right)=0,\hfill \\ \hfill \exists \delta & >0 \mathrm{such} \mathrm{that} \forall 0<h<\delta :0<W_g(t_{sep}+h)-W_L(t_{sep}+h).\hfill \end{array}$$

(3)

Relying on the fact that ${\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}(W_g\left(t\right)-W_L\left(t\right))=P_g\left(t\right)-P_L\left(t\right)$ is continuous (from TC lemma, and therefore bounded on $[a,b]$), we also know we can choose $\delta >0$ such that

$$\forall 0<h<\delta :0<(W_g-W_L)(t_{sep}+h)<W_{max}.$$

(4)

So, we can conclude that

$$\begin{array}{cc}\hfill W_g\left(t_{sep}\right)& =W_L\left(t_{sep}\right),\hfill \\ \hfill \forall h\in (0,\delta )& :W_L(t_{sep}+h)<W_g(t_{sep}+h)<W_L(t_{sep}+h)+W_{max}.\hfill \end{array}$$

(5)

Therefore, from SF lemma, we know $W_g\left(t\right)$ must be a straight line in the time segment $\left(t_{sep},t_{sep}+{\displaystyle \frac{\delta }{2}}\right]$, and its slope satisfies

$$\begin{array}{cc}\hfill \forall t& \in \left(t_{sep},t_{sep}+{\displaystyle \frac{\delta }{2}}\right]:\hfill \\ \hfill P_g\left(t\right)& ={\displaystyle \frac{2\left(W_g\left(t_{sep}+\frac{\delta }{2}\right)-W_g\left(t_{sep}\right)\right)}{\delta }}\equiv p_{g0}.\hfill \end{array}$$

(6)

From TC lemma, we also know that $W_g\left(t_{sep}\right)=W_L\left(t_{sep}\right)\Rightarrow P_g\left(t_{sep}\right)=P_L\left(t_{sep}\right)$; however,

$$\begin{array}{cc}\hfill p_{g0}·{\displaystyle \frac{\delta }{2}}& ={\int }_{t_{sep}}^{t_{sep}+{\displaystyle \frac{\delta }{2}}}{P}_g\left(t\right)·\mathrm{d}t=\hfill \\ \hfill & =W_g\left(t_{sep}+{\displaystyle \frac{\delta }{2}}\right)-W_g\left(t_{sep}\right)>W_L\left(t_{sep}+{\displaystyle \frac{\delta }{2}}\right)-W_L\left(t_{sep}\right)=\hfill \\ \hfill & ={\int }_{t_{sep}}^{t_{sep}+{\displaystyle \frac{\delta }{2}}}{P}_L\left(t\right)\mathrm{d}t\iff \hfill \\ \hfill p_{g0}& >{\displaystyle \frac{2}{\delta }}{\int }_{t_{sep}}^{t_{sep}+{\displaystyle \frac{\delta }{2}}}{P}_L\left(t\right)\mathrm{d}t.\hfill \end{array}$$

(7)

Since $P_L\left(t\right)$ is increasing on $\left[t_{sep},t_{sep}+{\displaystyle \frac{\delta }{2}}\right]\subset [a,b]$, that means $P_g\left(t_{sep}\right)=p_{g0}>P_L\left(t_{sep}\right)$. This is a contradiction to the TC lemma, which proves that (2) holds. Next, we explicitly define the time $t_{big}\in [a,b]$ at which $W_g\left(t_{big}\right)>W_L\left(t_{big}\right)$. Notice the load power $P_L\left(t\right)$ is increasing on $[a,b]$, so $W_L\left(t\right)$ is convex there. This means that $\forall t_s,t_e\in [t_0,b],t_s<t_e$, the line connecting $W_L\left(t_s\right),W_L\left(t_e\right)$ is above $W_L\left(t\right)$, and it is strictly above $W_L\left(t\right)$ for some $t_s<t_e$. Because $P_L\left(t\right)$ is continuous on a compact interval, we know it is bounded. So,

$$\exists M\in \mathbb{R}:\forall t\in [0,T],|P_g\left(t\right)|<M.$$

(8)

Therefore, if we choose $t_s,t_e$ such that $t_e-t_s<W_{max}/M$, we know the line connecting $W_L\left(t_s\right),W_L\left(t_e\right)$ is also strictly below $W_L\left(t\right)+W_{max}$. Hence, according to SF lemma, if $W_g\left(t_s\right)=W_L\left(t_s\right)$ and $W_g\left(t_e\right)=W_L\left(t_e\right)$, it must be a straight line between those times. For that reason, looking at $t_{big}=(t_s+t_e)/2$ yields

$$W_g\left(t_{big}\right)=W_g\left({\displaystyle \frac{t_s+t_e}{2}}\right)>W_L\left({\displaystyle \frac{t_s+t_e}{2}}\right)=W_L\left(t_{big}\right).$$

(9)

Notice that (2) and (9) contradict each other; therefore, $\nexists t_0\in [a,b]$ such that $W_g\left(t_0\right)=W_L\left(t_0\right)$. An illustration of this argument is shown in Figure 4.

Note that the same argument can be made in case that $W_g\left(t\right)$ is equal to the upper energy band $W_L\left(t\right)+W_{max}$ in a decreasing interval of $P_L\left(t\right)$ by switching convexity for concavity solely. An illustration is shown in Figure 5.

Lastly, let us confirm that when the generated energy $W_g\left(t\right)$ is equal to a constraint, it must be equal to it for a continuous time segment inside the monotone interval of $P_L\left(t\right)$. Without loss of generality, we let $W_g\left(t_0\right)=W_L\left(t_0\right)+W_{max}$ for some $t_0\in [0,T]$. Using the fact that $\exists [a,b]\subseteq [0,T]$ such that $t_0\in [a,b]$ and $[a,b]$ is an increasing interval of $P_L\left(t\right)$, we let

$$A=\{t\in [a,b]\mid W_g\left(t\right)=W_L\left(t\right)+W_{max}\}.$$

(10)

Next, it is imperative to show that the set A is a segment. Since A is bounded, it has a supremum and infimum. Given that $W_g\left(t\right),W_L\left(t\right)$ are continuous, we also know the supremum and infimum are in A. We let

$$\begin{array}{c}\hfill min\left(A\right)=c,max\left(A\right)=d.\end{array}$$

(11)

We assume by negation $\exists t_1\in [c,d]:W_g\left(t_1\right)\ne W_L\left(t_1\right)+W_{max}$. Then,

$$\begin{array}{cc}\hfill W_g\left(c\right)& =W_L\left(c\right)+W_{max},\hfill \\ \hfill W_g\left(d\right)& =W_L\left(d\right)+W_{max},\hfill \\ \hfill W_g\left(t_1\right)& <W_L\left(t_1\right)+W_{max}.\hfill \end{array}$$

(12)

Inasmuch as $W_g\left(t\right)$ and $W_L\left(t\right)$ are continuous, we let $t_l$ be their first separation point after $t=c$. Two different scenarios need to be addressed as follows:

1.

If $\exists t_m\in [c,d]$ such that $t_m>t_l$ and $t_m$ is a union point of $W_L\left(t\right),W_g\left(t\right)$, since $[c,d]\subseteq [a,b]$ is in an increasing interval of $P_L\left(t\right)$, the following is produced:

$$P_g\left(t_m\right)=P_g\left(t_l\right)=P_L\left(t_l\right)<P_L\left(t_m\right).$$

(13)

This contradicts TC lemma at $t_m$. An illustration of this case can be seen in Figure 6.

2.

Otherwise, $\nexists t_m\in [c,d]$ such that $t_m>t_l$ and $t_m$ is a union point of $W_L\left(t\right),W_g\left(t\right)$. But then from SF lemma $P_g\left(t\right)$ is constant for $t\in [t_l,d]$, and

$$\begin{array}{cc}\hfill W_g\left(d\right)& =W_g\left(t_l\right)+{\int }_{t_l}^d{P}_g\left(t\right)\mathrm{d}t=\hfill \\ \hfill & =W_L\left(t_l\right)+W_{max}+{\int }_{t_l}^d{P}_L\left(t_l\right)\mathrm{d}t<\hfill \\ \hfill & <W_L\left(t_l\right)+W_{max}+{\int }_{t_l}^d{P}_L\left(t\right)\mathrm{d}t=W_L\left(d\right)+W_{max}.\hfill \end{array}$$

(14)

This contradicts (12). An illustration of this case can be seen in Figure 7.

Both cases reached a contradiction. Hence, $\mathrm{if}\nexists t_1\in [c,d]\mathrm{then}{W}_g\left(t_1\right)\ne W_L\left(t_1\right)+W_{max}$ and the set A is a segment. □

2.2. The Fourth Lemma

The proof of this lemma is based on the following definitions.

Definition 6

(Candidate Optimal Solution). We let $[t_a,t_b]\subseteq [0,T]$ be an interval. A differentiable function $W_g\left(t\right):[t_a,t_b]\to \mathbb{R}$ is a Candidate Optimal Solution (COS) on $[t_a,t_b]$ if

1.

$\forall t\in [t_a,t_b]:W_L\left(t\right)\le W_g\left(t\right)\le W_L\left(t\right)+W_{max}$,

2.

$W_g\left(t\right)$ satisfies SF lemma on $[t_a,t_b]$,

3.

$W_g\left(t\right)$ satisfies TC lemma on $[t_a,t_b]$.

Definition 7

(Reachable Monotone Interval). We let $[t_a,t_b]$ be a monotone interval of $P_L\left(t\right)$, where $t_b<T$. Also, we let $C(i,t)=W_L\left(t\right)+i·W_{max}$ be the optimization problem constraints for $i\in \{0,1\}$. We let $[t_c,t_d]$ be another monotone interval of $P_L\left(t\right)$ with $t_c>t_b$. Then, $[t_c,t_d]$ is reachable from $[t_a,t_b]$ if

1.

$\exists W_g\left(t\right):[t_a,t_d]\to \mathbb{R}$ such that $W_g\left(t\right)$ is a COS on $[t_a,t_d]$,

2.

$\exists t_{sep}\in [t_a,t_b],\exists i_0\in \{0,1\}$ such that $t_{sep}$ is a separation point of $W_g\left(t\right)$ and $C(i_0,t)$,

3.

$\exists t_{meet}\in [t_c,t_d],\exists i_1\in \{0,1\}$ such that $t_{meet}$ is a union point of $W_g\left(t\right)$ and $C(i_1,t)$,

4.

$\forall t\in [t_{sep},t_{meet}]:W_L\left(t\right)<W_g\left(t\right)<W_L\left(t\right)+W_{max}$.

Definition 8

(Routed Between Monotone Intervals). We let $[t_a,t_b]$ be a monotone interval of $P_L\left(t\right)$, where $t_b<T$. Also, let $C(i,t)=W_L\left(t\right)+i·W_{max}$ be the optimization problem constraints for $i\in \{0,1\}$. We let $[t_c,t_d]$ be a reachable monotone interval from $[t_a,t_b]$. A COS $W_g\left(t\right)$ on $[t_a,t_d]$ is said to be routed between $[t_a,t_b]$ and $[t_c,t_d]$ if

1.

$\exists t_{sep}\in [t_a,t_b],\exists i_0\in \{0,1\}$ such that $t_{sep}$ is a separation point of $W_g\left(t\right)$ and $C(i_0,t)$,

2.

$\exists t_{meet}\in [t_c,t_d],\exists i_1\in \{0,1\}$ such that $t_{meet}$ is a union point of $W_g\left(t\right)$ and $C(i_1,t)$,

3.

$\exists t_1\in [t_c,t_d]$ with $t_1>t_{meet}$ where $P_g\left(t_1\right)\ne P_g\left(t_{meet}\right)$.

Remark. 

A COS that is routed between an increasing interval and a decreasing interval is said to be discharging. A COS that is routed between a decreasing interval and an increasing interval is said to be charging.

Lemma 4

(Unique Between Monotone Intervals (UBMI)). We let $[t_a,t_b]$ and $[t_c,t_d]$ be two monotone intervals of $P_L\left(t\right)$ with $t_b<t_c$, and we let $W_g\left(t\right)$ be a COS of (1). Also, we let $C(i,t)=W_L\left(t\right)+i·W_{max}$ be the optimization problem constraints for $i\in \{0,1\}$. If

1.

$\exists [t_1,t_2]\subseteq [t_a,t_b],\exists i_1\in \{0,1\}$ such that $\forall t\in [t_1,t_2]:W_g\left(t\right)=C(i_1,t)$,

2.

$\exists [t_3,t_4]\subseteq [t_c,t_d],\exists i_2\in \{0,1\}$ such that $\forall t\in [t_3,t_4]:W_g\left(t\right)=C(i_2,t)$,

3.

$\forall t\in [t_2,t_3]:W_L\left(t\right)<W_g\left(t\right)<W_L\left(t\right)+W_{max}$,

then $t_2,t_3$ are unique.

Proof. 

Without loss of generality, we assume the COS $W_g\left(t\right)$ is discharging between an increasing interval $[t_a,t_b]$ and a decreasing interval $[t_c,t_d]$ of $P_L\left(t\right)$ (where $t_b<t_c$). We let $W_g^1\left(t\right),W_g^2\left(t\right)$ be two COS to the problem described in (1), such that

$$\begin{array}{cc}\hfill \exists [t_1^i,t_2^i]\subseteq [t_a,t_b]& \mathrm{such}\mathrm{that}\forall t\in [t_1^i,t_2^i]:W_g^i\left(t\right)=W_L\left(t\right)+W_{max},\hfill \\ \hfill \exists [t_2^i,t_3^i]\subseteq [t_c,t_d]& \mathrm{such}\mathrm{that}\forall t\in [t_3^i,t_4^i]:W_g^i\left(t\right)=W_L\left(t\right),\hfill \\ \hfill \forall t\in [t_2^i,t_3^i]& :W_L\left(t\right)<W_g\left(t\right)<W_L\left(t\right)+W_{max},\hfill \end{array}$$

(15)

where $i\in \{1,2\}$. We know from SF lemma that ${\displaystyle \frac{d}{\mathrm{d}t}}{W}_g^i\left(t\right)=P_g^i\left(t\right)$ are constant in $[t_2^i,t_3^i]$. Without loss of generality, we assume by negation $t_2^1<t_2^2$. Because we know $P_g^i\left(t_2^i\right)=P_g^i\left(t_3^i\right)$, and because $[t_a,t_b]$ is an increasing interval of $P_L\left(t\right)$ while $[t_c,t_d]$ is a decreasing interval, we determine that $t_3^1>t_3^2$ and

$$\forall t\in (t_2^1,t_3^1):P_g^1\left(t\right)<P_g^2\left(t\right).$$

(16)

This results in

$$\begin{array}{cc}\hfill -W_{max}& ={\int }_{t_2^1}^{t_3^1}(P_g^1-P_L)\left(t\right)\mathrm{d}t<{\int }_{t_2^1}^{t_3^1}(P_g^2-P_L)\left(t\right)\mathrm{d}t\hfill \\ \hfill & ={\int }_{t_2^2}^{t_3^2}(P_g^2-P_L)\left(t\right)\mathrm{d}t=-W_{max}\hfill \\ \hfill & \iff -W_{max}<-W_{max}.\hfill \end{array}$$

(17)

This contradicts the fact that both COS discharge in $[t_2^i,t_3^i]$, $t_2^1=t_2^2$ and $t_3^1=t_3^2$, accordingly. An illustration of the proof can be seen in Figure 8. □

2.3. The Fifth Lemma

Lemma 5

(Most Distant Reachable Interval (MDRI)). We let $[t_a,t_b]$ and $[t_c,t_d]$ be monotone intervals of $P_L\left(t\right)$, where $t_b<t_c$. Also, we let $C(i,t)=W_L\left(t\right)+i·W_{max}$ be the optimization problem constraints for $i\in \{0,1\}$ and let $W_g\left(t\right)$ be an optimal solution to (1). If $W_g\left(t\right)$ is routed between $[t_a,t_b]$ and $[t_c,t_d]$, then $[t_c,t_d]$ is unique, and it is either the most distant increasing or most distant decreasing reachable interval.

Proof. 

The proof of the lemma consists of three parts.

1.

The first part formulates the connection between the slopes of generated energies routed between monotone intervals of $P_L\left(t\right)$ and the distance between the different monotone intervals.

2.

The second part utilizes the conclusion regarding those connections and proves that an optimal solution cannot be routed to a monotone interval which is not either the most distant reachable increasing interval of $P_L\left(t\right)$ possible or the most distant reachable decreasing interval of $P_L\left(t\right)$ possible.

3.

The last part proves that only one of those monotone intervals is a valid solution.

We let $[t_a,t_b]$ be a monotone interval and let $[t_c,t_d]$ and $[t_e,t_f]$ be another monotone interval reachable from $[t_a,t_b]$ (where $t_b<t_c$, $t_d<t_e$). Let us show the following characteristics are valid and denote them as “Fan Behavior” of the COS:

We let $W_g^{<1>}\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_c,t_d]$ and let $W_g^{<2>}\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_e,t_f]$. We let $P_g^{<i>}\left(t\right)$ be the corresponding generated power for $i\in \{1,2\}$. We denote the separation point of $W_g^{<i>}\left(t\right)$ and the constraint at $[t_a,t_b]$ as $t_{sep}^{<i>}$, and the union point of $W_g^{<i>}\left(t\right)$ and the constraint at $[t_c,t_d]$ as $t_{meet}^{<i>}$. Then,

1.

if $[t_c,t_d]$ is an increasing interval, $P_g^{<1>}\left(t_{meet}^{<1>}\right)=P_g^{<1>}\left(t_{sep}^{<1>}\right)<P_g^{<2>}\left(t_{sep}^{<2>}\right)=P_g^{<2>}\left(t_{meet}^{<2>}\right)$;

2.

if $[t_c,t_d]$ is a decreasing interval, $P_g^{<1>}\left(t_{meet}^{<1>}\right)=P_g^{<1>}\left(t_{sep}^{<1>}\right)>P_g^{<2>}\left(t_{sep}^{<2>}\right)=P_g^{<2>}\left(t_{meet}^{<2>}\right)$.

To prove it, without loss of generality, we assume $[t_a,t_b]$ and $[t_c,t_d]$ are increasing intervals. We assume by negation that $P_g^{<2>}\left(t_{sep}^{<2>}\right)>P_g^{<1>}\left(t_{sep}^{<1>}\right)$. Because $[t_a,t_b]$ is an increasing interval, we know $t_{sep}^{<1>}<t_{sep}^{<2>}$. Therefore,

$$\begin{array}{cc}\hfill W_g^{<2>}\left(t_{meet}^{<1>}\right)& =W_g^{<2>}\left(t_{sep}^{<1>}\right)+{\int }_{t_{sep}^{<1>}}^{t_{sep}^{<2>}}{P}_g^{<2>}\left(t\right)\mathrm{d}t+{\int }_{t_{sep}^{<2>}}^{t_{meet}^{<1>}}{P}_g^{<2>}\left(t\right)\mathrm{d}t\hfill \\ \hfill & =W_L\left(t_{sep}^{<1>}\right)+W_{max}+{\int }_{t_{sep}^{<1>}}^{t_{sep}^{<2>}}{P}_L\left(t\right)\mathrm{d}t+P_L\left(t_{sep}^{<2>}\right)·(t_{meet}^{<1>}-t_{sep}^{<2>})\hfill \\ \hfill & >W_L\left(t_{sep}^{<1>}\right)+W_{max}+P_L\left(t_{sep}^{<1>}\right)·(t_{sep}^{<2>}-t_{sep}^{<1>})+P_L\left(t_{sep}^{<1>}\right)·(t_{meet}^{<1>}-t_{sep}^{<2>})\hfill \\ \hfill & =W_g^{<1>}\left(t_{sep}^{<1>}\right)+{\int }_{t_{sep}^{<1>}}^{t_{sep}^{<2>}}{P}_g^{<1>}\left(t\right)\mathrm{d}t+{\int }_{t_{sep}^{<2>}}^{t_{meet}^{<1>}}{P}_g^{<1>}\left(t\right)\mathrm{d}t=W_g^{<1>}\left(t_{meet}^{<1>}\right)\hfill \\ \hfill & =W_L\left(t_{meet}^{<2>}\right)+W_{max}.\hfill \end{array}$$

(18)

In this case, the meaning is that $W_g^{<2>}\left(t\right)$ violates the upper constraint at $t_{meet}^{<2>}$, which results in a contradiction. Therefore, the “Fan Behavior” holds. An illustration of this behavior can be seen in Figure 9 and Figure 10.

Now, consider again $[t_a,t_b]$, $[t_c,t_d]$ and $[t_e,t_f]$ monotone intervals where $t_b<t_c, t_d<t_e$ and $[t_c,t_d]$, $[t_e,t_f]$ are reachable from $[t_a,t_b]$. We prove that

1.

If $[t_c,t_d]$ and $[t_e,t_f]$ are both increasing intervals, $\nexists W_g\left(t\right)$ a COS on $[t_a,t_f]$ that is routed from $[t_a,t_b]$ to $[t_c,t_d]$.

2.

If $[t_c,t_d]$ and $[t_e,t_f]$ are both decreasing intervals, $\nexists W_g\left(t\right)$ a COS on $[t_a,t_f]$ that is routed from $[t_a,t_b]$ to $[t_c,t_d]$.

Without loss of generality, we assume $[t_a,t_b]$, $[t_c,t_d]$ and $[t_e,t_f]$ are all increasing intervals of $P_L\left(t\right)$. We let $W_g^{<1>}\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_c,t_d]$ and $W_g^2\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_e,t_f]$. We let $t_2^1\in [t_c,t_d]$ be the union point of $W_g^1\left(t\right)$ and $W_L\left(t\right)+W_{max}$ in $[t_c,t_d]$ and $t_2^2\in [t_e,t_f]$ be the union point of $W_g^2\left(t\right)$ and $W_L\left(t\right)+W_{max}$ in $[t_e,t_f]$. Also, we let $t_1^1\in [t_a,t_b]$ be the separation point of $W_g^1\left(t\right)$ and $W_L\left(t\right)+W_{max}$ in $[t_a,t_b]$ and $t_1^2\in [t_a,t_b]$ separation point of $W_g^2\left(t\right)$ and $W_L\left(t\right)+W_{max}$ in $[t_a,t_b]$.

From the “Fan Behavior” property, we know $P_g^1\left(t_1^1\right)>P_g^2\left(t_1^2\right)$.

Because $[t_a,t_b]$ is increasing, $t_1^1>t_1^2$. We let $[t_g,t_h]$ be a decreasing interval of $P_L\left(t\right)$ with $t_g>t_d$, $t_h<t_e$. If $[t_g,t_h]$ is reachable from $[t_a,t_b]$ and let $W_g^3\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_g,t_h]$. We know from the “Fan Behavior” that $P_g^3\left(t_1^3\right)<P_g^2\left(t_1^2\right)$, and also $P_g^2\left(t_1^2\right)<P_g^1\left(t_1^1\right)$. Hence, $P_g^3\left(t_1^3\right)<P_g^1\left(t_1^1\right)$. This means $W_g^1\left(t\right)$ cannot meet any decreasing interval before time $t_2^2$, and therefore $\forall t\in [t_1^2,t_2^2]:P_g^2\left(t\right)>P_g^1\left(t\right)$.

Therefore, we know

$$\begin{array}{cc}\hfill W_g^1\left(t_2^2\right)& =W_g^1\left(t_1^2\right)+{\int }_{t_1^2}^{t_2^2}{P}_g^1\left(t\right)\mathrm{d}t=\hfill \\ \hfill & =W_L\left(t_1^2\right)+W_{max}+{\int }_{t_1^2}^{t_2^2}{P}_g^1\left(t\right)\mathrm{d}t>\hfill \\ \hfill & >W_L\left(t_1^2\right)+W_{max}+{\int }_{t_1^2}^{t_2^2}{P}_g^2\left(t\right)\mathrm{d}t=\hfill \\ \hfill & =W_g^2\left(t_1^2\right)+{\int }_{t_1^2}^{t_2^2}{P}_g^2\left(t\right)\mathrm{d}t=W_g^1\left(t_2^2\right)=\hfill \\ \hfill & =W_L\left(t_2^2\right)+W_{max}.\hfill \end{array}$$

(19)

The resulting conclusion is that $W_g^1\left(t_2^2\right)$ violates the upper bound in $t_2^2$, and it is therefore not a valid COS. Consequently, the second claim of MDRI lemma holds; thus, an optimal solution must be routed to the most distant increasing or most distant decreasing reachable interval.

As a final step, let us prove that the first claim of MDRI lemma holds, which means that at least one of the most distant reachable monotone intervals is not valid since a COS that is routed to it violates a constraint. As a final step, let us prove that the first claim of MDRI lemma holds, which means that at least one of the most distant reachable monotone intervals is not valid since a COS that is routed to it violates a previous lemma or a boundary constraint. We let $[t_a,t_b]$, $[t_c,t_d]$ and $[t_e,t_f]$ be monotone intervals where $t_b<t_c,t_d<t_e$ and $[t_c,t_d]$, $[t_e,t_f]$ are the most distant increasing and decreasing intervals reachable from $[t_a,t_b]$. We let $W_g^1\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_c,t_d]$ and $W_g^2\left(t\right)$ be a COS routed from $[t_a,t_b]$ to $[t_e,t_f]$. Without loss of generality, we assume $[t_a,t_b],[t_e,t_f]$ are increasing intervals and $[t_c,t_d]$ is a decreasing interval. We let $t_1^1\in [t_a,t_b]$ be the separation point of $W_g^1\left(t\right)$ and $W_L\left(t\right)+W_{max}$ at $[t_a,t_b]$ and $t_1^2\in [t_a,t_b]$ be the separation point of $W_g^2\left(t\right)$ and $W_L\left(t\right)+W_{max}$ at $[t_a,t_b]$. Because $[t_a,t_b]$ is increasing, $t_1^2>t_1^1$. Also, we let $t_2^1$ be the union point of $W_g^1\left(t\right)$ and the $W_L\left(t\right)$ at $[t_c,t_d]$ and let $t_2^2$ be the union point of $W_g^2\left(t\right)$ and the $W_L\left(t\right)+W_{max}$ at $[t_e,t_f]$.

We know $W_g^1\left(T\right)=W_g^2\left(T\right)$ because both solutions must satisfy an empty battery at time $t=T$. Using the following fact,

1.

$[t_a,t_b]$ is increasing; hence, $\forall t\in (t_1^1,t_2^1]:P_g^1\left(t\right)=P_L\left(t_1^1\right)<P_L\left(t\right)=P_g^2\left(t\right)$;

2.

Utilizing “Fan Behavior”, one may conclude that $\forall t\in [t_1^2,t_2^1]:P_g^1\left(t\right)=P_g^1\left(t_1^1\right)<P_g^2\left(t_1^2\right)=P_g^2\left(t\right)$;

3.

$[t_c,t_d]$ is decreasing; thus, $\forall t\in ][t_2^1,t_d]:P_g^1\left(t\right)=P_L\left(t\right)<P_L\left(t_2^1\right)=P_L\left(t_1^1\right)<P_g^2\left(t_1^2\right)=P_g^2\left(t\right)$.

Therefore, for $t\in [t_c,t_d]$, we know $W_g^2\left(t\right)>W_g^1\left(t\right)$ and $P_g^2\left(t\right)>P_g^1\left(t\right)$. Moreover, $P_g^1\left(t\right)$ is only decreasing as long as $W_g^1\left(t\right)$ does not meet $W_L\left(t\right)+W_{max}$. The same is true in reverse for $W_g^2\left(t\right)$, $P_g^2\left(t\right)$ is only increasing as long as $W_g^1\left(t\right)$ does not meet $W_L\left(t\right)$. Because if both of them are valid, they must have a union point (at least at time $t=T$), at least one of the solutions must meet the other’s constraint: $W_g^2\left(t\right)$ must have a union point with $W_L\left(t\right)$ or the other way around. Without loss of generality, we assume $W_g^2\left(t\right)$ has a union point $t_0>t_f$ with $W_L\left(t\right)$ and assume $t_0$ is the first such union point. Then,

$$\begin{array}{cc}\hfill W_L\left(t_0\right)& =W_g^2\left(t_0\right)=W_g^2\left(t_1^1\right)+{\int }_{t_1^1}^{t_0}{P}_g^2\left(t\right)\mathrm{d}t>\hfill \\ \hfill & >W_g^2\left(t_1^1\right)+{\int }_{t_1^1}^{t_0}{P}_g^1\left(t\right)\mathrm{d}t=\hfill \\ \hfill & =W_g^1\left(t_1^1\right)+{\int }_{t_1^1}^{t_0}{P}_g^1\left(t\right)\mathrm{d}t=W_g^1\left(t_0\right).\hfill \end{array}$$

(20)

This means $W_g^1\left(t\right)$ violates the lower constraint at $t=t_0$, and is therefore not a valid COS on $[t_a,T]$. Therefore, only one of the COS $W_g^1\left(t\right),W_g^2\left(t\right)$ is valid, and there is only a single valid COS. □

As the final stage of the proof, we let $W_g^1\left(t\right),W_g^2\left(t\right)$ be two optimal solutions to the optimiation problem described in (1). We let $A=\{t\in [0,T]\mid W_g^1\left(t\right)\ne W_g^2\left(t\right)\}$. We assume by negation $A\ne \varnothing$, and let $t_0=inf\left(A\right)$. We can now check for the following cases:

1.

If $t_0=0$, then we have a contradiction to the starting condition.

2.

The case where $i\in \{1,2\}\Rightarrow W_L\left(t_0\right)<W_g^i\left(t_0\right)<W_L\left(t_0\right)+W_{max}$ is not possible because both $W_g^1\left(t\right)$ and $W_g^2\left(t\right)$ must be straight lines in the neighborhood of $t_0$ (SF lemma). Therefore, there must be $\delta >0$ such that $\forall t\in [t_0-\delta ,t_0):W_g^1\left(t\right)\ne W_g^2\left(t\right)$, which contradicts the definition of $t_0$.

3.

If $\exists i\in \{1,2\}:W_g^i\left(t_0\right)=W_L\left(t_0\right)$, then we know $t_0\in [a,b]$ such that $[a,b]$ is a decreasing interval of $P_L\left(t\right)$. We now have a contradiction, because

(a)

If both $W_g^1\left(t\right),W_g^2\left(t\right)$ are routed from $[a,b]$ to the same reachable monotone interval, we have a contradiction to UBMI lemma.

(b)

If $W_g^1\left(t\right)$ and $W_g^2\left(t\right)$ are routed from $[a,b]$ to different monotone intervals, we have a contradiction to MDRI lemma.

4.

The case where $\exists i\in \{1,2\}:W_g^i\left(t_0\right)=W_L\left(t_0\right)+W_{max}$ is similar to the last one.

5.

Lastly, if $t_0=T$, we have a contradiction to the ending condition.

In conclusion, it may be deduced that $A=\{\varnothing \}$, and the solution to (1) is unique. □

3. Comparative Analysis

3.1. Energy Balancing with Transients

We consider a power system comprising a grid-connected storage device and a photovoltaic (PV) unit, as illustrated in Figure 11. The storage device is charged from both the grid and the PV unit, supplying an aggregated load characterized by its active power consumption. The load’s active power demand is modeled as a continuous positive function, $P_a:\mathbb{R}\ge 0\to \mathbb{R}$, defined over a finite time interval $[0,T]$ for a given parameter T. The power supplied by the PV is represented by a piecewise continuous function, $Ppv:\mathbb{R}\ge 0\to \mathbb{R}$, which partially offsets the load’s power demand. Consequently, the net power consumption of the load is expressed as $P_L\left(t\right)=P_a\left(t\right)-Ppv\left(t\right)$. The power flowing into or from the storage device is given by $P_s\left(t\right)=P_g\left(t\right)-P_L\left(t\right)$, where $P_g:{\mathbb{R}}_{\ge 0}\to \mathbb{R}$ represents the power injected from the grid. Furthermore, $W_g\left(t\right)$, $W_L\left(t\right)$, and $W\left(t\right)$ denote the generated energy, load energy, and stored energy, respectively. These energy terms relate to the power functions via the integral equation $E\left(t\right)={\int }_0^{T}P\left(\tau \right)\mathrm{d}\tau$. The maximum storage capacity, denoted as $E_{max}$, is varied in simulations. The tested values are $E_{max}\in 0, 2, 6, 8,15, 27$ p.u. The power consumption of the load and PV generation data (both in p.u.) are synthesized to reflect real-world conditions. The simulation results for $E_{max}=2, 6$ p.u. are presented in Figure 12. Regarding the generation sources, we use a standard synchronous generator model where the generator operates with frequency and voltage regulation, providing stable power output while following load fluctuations. The photovoltaic plant model assumes a typical irradiance-dependent power output, where solar generation varies over time but follows a predefined profile based on environmental conditions.

To explore the behavior of this model in reinforcement learning (RL) settings, we first examine its ideal case performance before assessing how transient effects influence algorithmic efficiency. In the RL formulation, the deterministic load function is replaced with a generative model. The environment’s state is represented by a continuous state space, $\mathcal{S}=s=(E_s,H,P_L)$, where W denotes the state of charge (SOC), H indicates the current hour, and $P_L$ represents the current load. At each time step t, the agent selects an action $a\in \mathcal{A}$, determining the energy allocated to charge the battery. The action space aligns with the original search problem, defined as $\mathcal{A}=a=\Delta E,0\le \Delta E\le W_L+E_{max}$, where $\Delta E\in \mathbb{R}$ signifies the energy added to the battery’s current level. The SOC and battery capacity establish the permissible action bounds. After executing an action, the agent observes the new state and receives a scalar reward r, transitioning accordingly. The simplest reward function follows a quadratic cost structure [20], expressed as $r=-f\left(P_g\right)$ with $f\left(P_g\right)=P_g^2\left(t\right)$. The transition function deterministically maps the battery state to the prescribed SOC, ensuring consistent decision-making.

To determine the optimal control policy, we implement the analytical method in MATLAB and trained three model-free RL algorithms in Python 3.12. The policy specifies the generation and charge/discharge actions over 48 time steps spanning 24 h. Our study focuses on model-free approaches, which forego explicit environmental dynamics modeling in favor of direct policy optimization. These methods are widely used, particularly for storage control, due to their adaptability across domains. Nevertheless, model-based approaches present an intriguing avenue for future research. The RL methods analyzed include Soft Actor–Critic (SAC), Proximal Policy Optimization (PPO), and Twin Delayed DDPG (TD3), implemented via Stable Baselines. These methods were chosen due to their effectiveness in continuous action spaces, making them suitable for energy storage control tasks. The RL agents were trained in a simulated environment, where they learned optimal charging and discharging policies through trial-and-error interactions with the system. The training objective was to minimize the total energy cost while adhering to storage constraints.

The hyperparameter settings followed standard configurations for stability and efficiency. The discount factor $\gamma$ was set to 0.99 to prioritize long-term rewards, and the learning rate was adjusted based on the algorithm, $3\times {10}^{-4}$ for PPO and SAC, ${10}^{-3}$ for TD3. The policy and value networks consisted of two hidden layers with 256 neurons each, using ReLU activations. To improve exploration, we incorporated Gaussian noise for SAC and TD3 and used a clipped surrogate objective for PPO. Training was conducted over one million time steps, with batch sizes of 256 and an Adam optimizer for gradient updates. The final policy was evaluated over a 100-day test set, comparing RL-based energy management decisions to classical optimization methods.

The results shown in Figure 13 reveal that none of the RL methods succeeded in matching the optimal policy identified by the classical algorithm. Notably, PPO and TD3 yielded similar policies, despite TD3 employing a deterministic policy while PPO relies on a stochastic one. A few key insights emerge.

1.

When examining consecutive days with similar demand and PV production patterns, policies tend to be similar due to the Cauchy-bounded nature of the value function.

2.

The system dynamics remain consistent over time, ensuring stable relationships between storage, PV generation, grid interaction, and load.

3.

The reward function exhibits convexity, preventing local minima.

4.

The reward function exhibits symmetry concerning certain states and actions, guiding both algorithms toward similar policies. For example, we consider state $s_0$, where high PV generation satisfies demand but additional charging ($\Delta P_s=1$ p.u.) is necessary due to anticipated future demand, versus state $s_1$, where low PV production requires purchasing 1 p.u. from the grid. Despite differing conditions, identical rewards may cause algorithms to converge to similar policies.

Figure 14 presents, for each algorithm, the daily cost of energy production over a test set of 100 days. The mean and variance for each algorithm are presented in

Table 1. Mean and variance calculation; The method exhibiting the minimal mean and variance is presented in bold typeface.

Algorithm	Mean	Var
SP	711.85	70,854.35
PPO	2428.24	271,321.64
SAC	1935.02	154,239.84
TD3	2787.36	342,948.40

. The deviation from the mean of the daily costs of each algorithm over these 100 test days is depicted in Figure 15.

To extend the comparative analysis, we now introduce a more complex model that incorporates a lossy storage device [21]. In this formulation, storage losses are explicitly introduced, meaning that energy stored at one time step is no longer perfectly available in subsequent time steps. This modification affects the optimal trajectory construction because the system must now compensate for energy dissipation over time, requiring a more conservative approach to energy allocation. When losses are taken into account, the shortest path solution is affected because the system must optimize not only for immediate energy balance but also for minimizing accumulated losses over time. As a result, the optimal trajectory smooths out power fluctuations to avoid unnecessary energy cycling, which would otherwise exacerbate losses. The addition of losses reduces the feasibility of high-frequency energy exchanges and forces a more structured, long-term planning approach in the optimization. This additional complexity heightens the uncertainty faced by both traditional and reinforcement learning algorithms. The RL formulation remains largely consistent with the ideal storage case, with the key distinction being in the definition of the MDP’s transition function. Instead of a direct mapping between charge and discharge actions to a specified SOC, transmission losses must be considered. Thus, action $a=\Delta E$ constitutes a valid transition from state $s=W$ to $s^{\prime }=(W+\Delta E)\eta$ provided the transition adheres to the battery’s capacity constraint, i.e., $W+\Delta E<E_{max}$. To account for energy dissipation, if $\Delta E\ne 0$, the decay factor is given by $\eta ={\eta }_{\mathrm{dis}}{\eta }_{\mathrm{decay}}$. Conversely, when no charge or discharge occurs, the decay factor simplifies to $\eta ={\eta }_{\mathrm{decay}}$.

The corresponding results are illustrated in Figure 16. Once again, none of the RL methods managed to learn an effective policy when compared to the optimal path found by the classical algorithm. Again, the policy of both PPO and TD3 methods is similar. The mean and variance for each algorithm are presented in

Table 2. Mean and variance calculation; The method exhibiting the minimal mean and variance is presented in bold typeface.

Algorithm	Mean	Var
PMP	705.82	69,024.48
PPO	748.17	88,234.85
SAC	2280.81	207,902.61
TD3	748.17	88,234.85

.

From the table, it can be observed that the mean and variance of the classical algorithm decreased, indicating improved battery operation compared to the initial case. This highlights the characteristics of each classical approach: “SP”, which employs “Dijkstra” search, appears to be more suited for discrete-state dynamics, whereas “PMP” demonstrates better performance in continuous environments. Furthermore, the mean values for both PPO and TD3 also declined, possibly due to their capacity to manage large and complex state spaces, enabling them to capture environmental subtleties more effectively. However, the increased model complexity led to a rise in the variance across all RL algorithms. Figure 17 presents, for each algorithm, the daily cost of energy production over a test set of 100 days. The deviation from the mean of the daily costs of each algorithm over these 100 test days is depicted in Figure 18.

Following, we remove the assumption of idealized transmission of power between sources, storage, and loads. In previous paragraphs, we assumed that only storage losses existed, but now we also consider losses occurring between different components in the system and propose a model for analyzing losses over the transmission lines between the power source and the consumer, including the storage device. The system under consideration consists of a source, modeled as a synchronous generator, and a non-linear load. This non-linear load encompasses the photovoltaic unit, the storage device with capacity $E_{max}$, and the load consuming active power $P_L:{\mathbb{R}}_{\ge 0}\to \mathbb{R}$. We assume that the components within the non-linear load are closely situated, ensuring lossless transmission among them. Based on this, we conduct the analysis using a distributed circuit model, incorporating a short-length transmission line. The power approximation is given by

$$P_g\approx P+{\left({\displaystyle \frac{P}{V_g}}\right)}^{2}R\approx P+\alpha P^2,$$

(21)

where $P_g$ represents the generated power, P denotes the power flowing into the storage device, and $\alpha ={\displaystyle \frac{R}{|V_g{|}^2}}$ is a constant dependent on the line voltage $V_g$ and line resistance R. This quadratic relation is well documented in the literature, as discussed in [22].

For the RL solution, we incorporate the transmission line loss model. Consequently, the transition from state $s=W$ after taking an action $a=\Delta E$ leads to a new state $s^{\prime }=(W+\Delta E{\eta }_{\mathrm{tr}})\eta$, where ${\eta }_{\mathrm{tr}}$ accounts for transmission line losses and $\eta$ remains consistent with previous case studies.

The corresponding results are illustrated in Figure 19. In this scenario, PPO and TD3 algorithms yield distinct policies. Notably, the policy generated by the PPO algorithm, which employs a stochastic policy, appears smoother compared to that of the TD3 algorithm, which follows a deterministic policy.

Figure 20 presents, for each algorithm, the daily cost of energy production over a test set of 100 days. The mean and variance for each algorithm are presented in

Table 3. Mean and variance calculation; TThe method exhibiting the minimal mean and variance is presented in bold typeface.

Algorithm	Mean	Var
DP	773.10	89,220.93
PPO	1165.14	146,485.45
SAC	1293.96	109,379.59
TD3	2779.89	342,000.82

.

The results in the table reveal several trends: (a) The mean and variance of the classical algorithm increase due to the added uncertainty in the dynamic model. (b) Performance improvements are observed across all RL algorithms, highlighting their effectiveness in highly uncertain environments with stochastic transitions. (c) The PPO algorithm outperforms TD3, underscoring the benefits of stochastic policies in handling unpredictable model dynamics. The deviation from the mean daily costs of each algorithm over 100 test days is depicted in Figure 21.

3.2. Hybrid Electrical Vehicle Simulation

In this section, we present a series of simulations, showcasing numerical simulations on electrical vehicles, to demonstrate the validity of the algorithm whose properties were studied in the paper. The data set used for the experiments can be found in the Argonne National Laboratory. The analysis focuses on the real-time energy management of a hybrid electric vehicle, where the car’s battery is used alongside a fuel cell to meet its power demands during driving. In a hybrid vehicle, energy management involves optimally distributing power between the battery and the fuel cell to achieve efficiency and longevity. The optimization does not involve an external power system, but rather ensures that battery discharge is managed effectively to minimize fuel consumption while maintaining performance. The decision variables include when to draw energy from the battery versus when to rely on the fuel cell, taking into account state-of-charge constraints, efficiency losses, and future energy needs. We consider the Volkswagen e-Golf electric vehicle with an 85 kW, 279 Nm permanent magnet synchronous AC motor and a 24.2 kWh, 323 V rated lithium-ion battery. In the simulations, we define $E_{max}=300$ kJ and select the simulation time as $T=45$ s, with a sample time of $\Delta T=0.1$. The results are presented in Figure 22, in which the resulting optimal generated power and the load profile $P_L\left(t\right)$ are presented. The ripples seen in the PMP solution are likely due to the transients observed in the load and the properties of the numerical solver. It is important to note, however, that the shortest path method provides an intuitive reference solution that allows for a benchmarking framework when evaluating other optimization techniques. While it is not directly applicable in real-world scenarios due to its reliance on a graphical interpretation, it may be used as an analytical tool. By offering a qualitative baseline, it enables researchers and practitioners to assess the effectiveness of more advanced control strategies, such as those based on Pontryagin’s principle or reinforcement learning. Given that it is an approximation-based method, the shortest path approach naturally produces smooth trajectories, which may not fully capture system transients or high-frequency variations in load demand, and thus the fluctuations that are observed in the other methods are not apparent. Despite these limitations, the shortest path solution can assist in planning and algorithmic design. Its ability to highlight structural properties of the optimal solution makes it useful for generating heuristic insights into problem formulations. Additionally, by comparing new algorithms against this benchmark, it is possible to identify areas where more sophisticated numerical methods are required to account for system dynamics, constraints, and uncertainties. In this sense, the shortest path method acts as a guiding principle rather than an operational tool, shaping the development of more practical and computationally feasible optimization strategies for real-world applications.

The MSE between the shortest path method and the minimum principle method is $0.6803$, and the MSE between the shortest path method and the dynamic programming method is $1.6211$.

Moreover, another application that was tested is the peak of the generated power.

Table 4. Generated peak-power for different vehicle categories.

Model Name	Nominal	Shortes-Path	No
Mercedes-Benz	13.1000	2.4811	79
Nissan Leaf SV	20.0757	3.0375	144
Mitsubishi I-MiEV	6.5887	1.1180	100
Chevrolet Spark EV	7.8028	1.9977	21
Volkswagen e-Golf	38.6643	19.2181	207
Smart EV	16.1382	8.9181	188
BMW i3BEV	21.7496	5.6388	1
Ford Focus	8.8122	1.7193	42
Kia Soul	82.2908	30.2048	64

shows the peak power calculated by the shortest path algorithm for other vehicles available in the data set, and the optimal power flows are computed as well. The database contains various car data samples. Therefore, the last column in the table indicates the record serial number.

4. Discussion

The results presented in this work highlight that the shortest path method serves as a useful graphical benchmark tool for evaluating optimization techniques for energy storage management. While it is not directly applicable in real-world implementations, it provides a qualitative baseline that enables researchers to assess the effectiveness of more advanced control strategies, such as Pontryagin’s principle or reinforcement learning. Given its approximate and inherently smooth trajectories, the shortest path method may overlook fast system transients but remains valuable for identifying structural properties of optimal solutions. By comparing new algorithms against this benchmark, it becomes possible to pinpoint areas where more sophisticated methods are needed to better capture system dynamics, constraints, and uncertainties. In this sense, the shortest path method guides the design of more practical optimization strategies, contributing to the development of computationally efficient and real-world-applicable control solutions. In particular, the comparison between Pontryagin’s Maximum Principle (PMP), dynamic programming (DP) method, the shortest path method, and reinforcement learning (RL) algorithms reveals important insights into the performance of each method and their practical applicability.

When examining the trajectories produced by either the PMP or DP methods, it is apparent that these produce more fluctuations than then the shortest path solution. In other words, the shortest path method appears to perform better than the other two algorithms. However, this result should be interpreted with caution. The shortest path method serves primarily as a reference signal, offering an idealized solution that may not fully capture real-world transients and constraints. Since it arises from a graphical solution, it naturally smooths out power variations and provides an approximated, but interpretable, optimal trajectory. This smoothing effect can make the shortest path solution appear superior in terms of cost and feasibility. However, this does not necessarily mean that it is applicable in real-world scenarios.

Furthermore, reinforcement learning algorithms struggled to match the analytical solutions, as evident in Figure 13. This is not necessarily due to insufficient training but rather an inherent limitation of model-free RL methods in high-variability environments. Specifically, the RL algorithms used in this study are model-free, meaning they do not have access to a mathematical model of the environment. Instead, they have to learn the behavior of the system through exploration and interaction. This learning process is particularly difficult due to the stochastic nature of load variations and the complexity of the energy constraints. This is in contrast to the analytical methods that leverage full knowledge of the system dynamics, since RL algorithms must learn optimal policies purely from experience, without access to the explicit equations describing the system. This makes it particularly challenging to capture long-term dependencies and constraints, such as energy storage limits and future load variations, especially when the observation window is short. In this analysis, the RL agents could only observe a limited time window, preventing them from fully anticipating future constraints, such as battery state-of-charge limitations.

Additionally, RL solutions in Figure 19 suggest that storage capacity constraints are not fully respected, likely due to the difficulty in learning accurate representations of the system boundaries. Unlike the shortest path, the PMP and the DP approaches, which explicitly incorporate constraints in their optimization framework, RL algorithms must infer them from observed rewards. If the training process does not include enough penalties or incentives to learn boundary conditions, the resulting policy may fail to enforce them effectively.

Another key factor is the impact of problem complexity on RL generalization. Interestingly, in the more complex model with lossy storage, RL algorithms performed closer to the ideal solution. This seemingly counterintuitive result can be explained by the fact that energy losses naturally regularize the problem, discouraging rapid charge–discharge cycles and promoting smoother control policies. With losses, RL algorithms are forced to learn long-term energy conservation strategies, which align more closely with the analytical approaches. This highlights an important interaction between problem complexity and RL generalization, where certain real-world constraints can help RL agents learn better policies by discouraging unrealistic strategies. These insights are summarized in

Table 5. Summary of experimental results and analysis.

Experiment	Key Observations	Role of SP as a Benchmarking Tool
Baseline Case	SP exhibits the lowest mean cost (711.85) and variance (70,854.35). RL algorithms perform significantly worse, with higher variance, indicating unstable policies.	SP serves as an interpretable reference, revealing structural properties of the optimal energy trajectory. It highlights RL inefficiencies in capturing the long-term dynamics of the system.
Lossy Storage Model	PMP achieves the lowest cost (705.82) and variance (69,024.48), demonstrating the effect of incorporating physical constraints explicitly. RL methods improve but still exhibit performance gaps.	SP provides a qualitative baseline for assessing the effect of adding realistic losses. By comparing RL outputs to SP and PMP, it is evident that RL struggles with long-term energy planning.
Lossy Transmission Line Model	DP achieves the lowest mean cost (773.10) and variance (89,220.93), outperforming RL methods. RL variance remains high, showing unstable learning behavior.	SP acts as an initial structural guide, allowing researchers to interpret whether more advanced algorithms are following expected energy trajectories. The graphical approach aids in evaluating solution smoothness and feasibility.

.

To conclude, the shortest path method, while not a direct solution for practical applications, offers a comparative standard for energy storage optimization techniques. Its smooth trajectory provides a basis for understanding solution structures, allowing researchers to identify where more complex methods are necessary to address system dynamics and constraints. Therefore, this method aids in the development of optimization strategies that are both computationally efficient and relevant to real-world scenarios.

5. Conclusions

The current paper belongs to a class of research works that aim to find analytic and semi-analytic solutions to optimal power flow problems that involve storage systems. Following is a summary of the most important points stemming from the above analysis.

• The main contribution of this work is a rigorous proof of the central result provided in [19], which is one of the first papers in this group. This proof justifies the “shortest path” graphical design method, and assures that the optimal solution obtained is indeed unique, thus allowing to avoid possible conflicts between different competing optimal solutions.
• Most importantly, the uniqueness proof presented in this paper has practical implications, since a guarantee that the solution is unique allows for more confident decision-making in real-world applications, such as grid management, and energy dispatch. Furthermore, the analytical nature of the solution, in contrast to purely numerical approaches, offers potential advantages in terms of computational efficiency and interpretability. The graphical design procedure, coupled with a guarantee of a unique solution, facilitates a deeper understanding of the system’s behavior and can aid in the design and optimization of storage systems.
• The validation of the proposed solution through two distinct comparative studies further strengthens its credibility. The comparison with reinforcement learning algorithms on synthetic data highlights the potential advantages of the proposed method in terms of convergence speed and solution quality. The data analysis, using an electrical vehicle storage device, demonstrates the practical applicability and effectiveness of the proposed solution in a realistic scenario.

Future research directions could explore extending this approach to more complex scenarios, such as systems with multiple storage devices, time-varying constraints, or uncertainties in the system parameters. Investigating the applicability of this method to other types of energy storage technologies, beyond electrical vehicle batteries, is also a promising area for future work.

As a final remark, we must acknowledge that the comparison between the shortest path and reinforcement learning (RL) methods in this study is problematic. This is because the selected optimization problem inherently favors the shortest path approach, neglecting the uncertainty-rich environments where RL thrives. Furthermore, the problem’s low dimensionality and constrained solution space severely limit RL’s potential, and consequently the observed superiority of the shortest path method may be misleading, as RL’s performance was significantly degraded. While a comprehensive analysis of RL’s capabilities in this context falls outside the scope of this paper, which primarily focuses on the theoretical proof of the shortest path method, we recognize this as another valuable avenue for future research.