Supercapacitor Electro-Mathematical and Machine Learning Modelling for Low Power Applications

Abstract

Low power electronic systems, whenever feasible, use supercapacitors to store energy instead of batteries due to their fast charging capability, low maintenance and low environmental footprint. To decide if supercapacitors are feasible requires characterising their behaviour and performance for the load profiles and conditions of the target. Traditional supercapacitor models are electromechanical, require complex equations and knowledge of the physics and chemical processes involved. Models based on equivalent circuits and mathematical equations are less complex and could provide enough accuracy. The present work uses the latter techniques to characterize supercapacitors. The data required to parametrize the mathematical model is obtained through tests that provide the capacitors charge and discharge profiles under different conditions. The parameters identified are life cycle, voltage, time, temperature, moisture, Equivalent Series Resistance (ESR) and leakage resistance. The accuracy of this electro-mathematical model is improved with a remodelling based on artificial neuronal networks. The experimental data and the results obtained with both models are compared to verify and weigh their accuracy. Results show that the models presented determine the behaviour of supercapacitors with similar accuracy and less complexity than electromechanical ones, thus, helping scaling low power systems for given conditions.

1. Introduction

Portable and autonomous electronic devices are becoming increasingly popular. One of their major drawbacks is their dependence on batteries [1,2]. Thus, battery life is a critical design factor, especially when wireless capabilities are required, because it involves a high percentage of total consumption by the WSN [3,4]. One technique to store the harvested energy and increase the amount of energy available is using supercapacitors or Electric Double Layer Capacitors (EDLC). They are attractive because they have higher power density than batteries, do not require special charging circuitry, and have a long operational lifetime. In order to minimize or optimize the power consumption, supercapacitors on energy harvesting systems must act as a primary storage device with a battery backup whenever it is necessary [5]. The benefits of using supercapacitors are many, like extended battery life, less electromagnetic interferences and better operation of analogic electronics. They are explained at length in the literature [6,7].

The first step to select a supercapacitor is based on the dimension the capacitance and form factor for the target application, which requires a complete characterization of the power consumption pattern of the target application. This information determines the size of the supercapacitor and whether if primary or secondary battery backup is needed. However, supercapacitors can not be analysed based only on their capacitance. Load current drain patterns, work cycle and temperature conditions change the characteristics of charge and discharge profiles [8]. The previous statement is especially true regarding low power applications because changes occur more often. Therefore, when the application must optimize the last drop of energy harvested, even in extreme conditions, a simple model of supercapacitors is not enough to estimate the real energy store capacity of the supercapacitor, i.e., the capacitance and its dynamic behaviour. The energy consumption pattern is a variable that depends on the application and the amount of available harvested energy, and it can be estimated statistically (windmills, solar panels indoors, etc.) [9]. This information and an accurate enough supercapacitor model will allow determining the required capacitance value for different conditions.

Electrochemical models can be found in [10,11]; these studies and models use complex equations and rely on electrochemical theory based on physical principles shown in Figure 1. Moreover, the parameters used in these equations are difficult to obtain because they require complex, expensive measurement equipment and long measurement times. Despite this complexity, up to date, these models usually provide the best accuracy.

Figure 1. Supercapacitor physical composition and internal work mode.

The purpose of this paper is to define a complete model to analyse the charge and discharge behaviour of a supercapacitor with an accuracy equivalent to that of the electromechanical models. Two different techniques have been used to reach the aforementioned objective, one electro-mathematical equations based and another with machine learning algorithms. Besides, these models could provide information of the supercapacitors for low power systems without using complex electrochemical models.

2. Related Work at Different Techniques

Dimension of the power system requirements of a low power electronic system involves characterising the nonlinear behaviour of the supercapacitor [12]. Supercapacitor charging and discharging does not present the same behaviour as standard capacitors because the leakage currents are a function of the charge and discharge dynamics [13]. Unless characterization of capacitors is done best, theoretically, with electromechanical models, nonetheless, there are another two lines of research and modelling of supercapacitor, mathematical integro-differential equation-based and IA. There is also a third approach that uses an approximate equivalent electrical model.

2.1. Equivalent Electrical Model

Capacitor ideal and simplified models of Figure 2a,b do not suffice for supercapacitor characterization [14]. Thus, as with standard capacitors like tantalum or electrolytic [15], it could use an equivalent electrical model of distributed components, Figure 2c, based on the physic morphology of the supercapacitor [16,17].

Figure 2. Circuit-based supercapacitor models: (a) an ideal capacitor. (b) Simplified model including a series and parallel resistance. (c) RC ladder circuit with a voltage-dependent capacitance in its first branch, which may be extended to n branches.

The electrical model represented in Figure 2c uses a distributed capacitance architecture in which each capacitance branch has a capacitance equation that emulates a particular physical behaviour. The coefficients of these equations are obtained empirically. The dynamic characteristic of supercapacitor and its nonlinear leakage current behaviour limit the advantages of this model. There are studies that have tried to improve this model with more complex and elaborate RC nets [18].

2.2. Mathematical Model

The physical structure, chemistry reactions and current distribution inside the capacitor could be modelled through mathematical expressions. These models characterise the non-linear behaviour of the leakage current, which is a function of the discharge dynamic pattern and the internal chemistry of the device.

A full electrochemical model of self-discharge studies the reactions on the electrode surface, through which electrons cross its double layer, as per the rule of the leakage and mathematical model of [19]. The model combines quantum mechanical and electrochemical phenomena that occur in the self-discharge process. Once the model is defined, it employs a computational method to simulate the discharge profile and energy storage for blocking layers of different thickness. Another alternative to obtain a mathematical model is presented in [20]. This research uses a different approach; it applies Lie derivate and Kalman filter to analyse the ion mobility and, thus, develops a supercapacitor electrochemical model with an observer.

Mathematical models take several critical parameters and show how this concrete parameter affect the operation of the device. Models of these types are less complex to implement than models based on electrochemical equations. e.g., at [21] the state of charge estimation is realized considering charge redistribution in real-time systems as EHWSN. Implementation of such a model without complex electrochemical equations provides good accuracy because it requires low memory usage and has low computational cost.

2.3. Machine Learning Algorithms

Machine learning as defined in [22] is a subfield of artificial intelligence closely related to data mining. Its aim is to develop algorithms that allow the machines to learn from data, i.e., to create programs able to induce models that improve their performance with the incoming data flow over time. It mixes mathematical elements with statistics and computational sciences such as classification trees, induction rules, neural networks, Bayesian networks, regression algorithms, supported vector machines or clustering [23].

Artificial Neural Network (ANN) is a mathematical computational model, which processes the information like biological neural systems such as the brain. It consists of a group of interconnected artificial neurons, which solves problems working in unison and processing information by means of a connectionist approach to computation. An ANN is an adaptive system that changes its structure based on information that flows through the neural network over the learning or training phase. The aforementioned phase involves adjusting the existing synaptic connections between neurons, thus, updating the network architecture and the connection weights.

ANN is one of the most widespread learning algorithms used within machine learning to model complex nonlinear relationships between inputs and outputs and find patterns in data. Its increasing popularity is due to its seemingly low dependence on any domain-specific knowledge and the availability of efficient learning algorithms, computational power, ability to model any given function, convergence speed and statistics. For this reason, ANN has been successfully applied to a broad spectrum of data-intensive applications.

In literature, there are some examples that use machine learning models to characterize some parameters of capacitor and energy storage devices. Work in [23] develops an algorithm for automotive applications that has one-layer feed forward ANN. This ANN is trained using a back-propagation algorithm that takes into account temperature, chemistry and life cycle. The work presented in [24] implements a current impedance test using machine learning. This test set-up presents an equivalent circuit base on the theory of electrochemical impedance spectroscopy that defines a first-phase linear model. In a second step, a multi-layer of Artificial Neuronal Network (ANN) establishes relationships among non-linear parameters of the capacitances and resistances. The aforementioned ANN is trained with experimental data banks using a back-propagation algorithm. Combining ANN and the equivalent circuit, the author can create an online nonlinear dynamic model. The research of [25] models a hybrid energy storage system for a Toyota Rav4EV with a stochastic nonlinear predictive control. Power demand is modelled as a Markov chain that had been trained with real-world driving cycles and simulation results given with a model in the loop.

3. Experimental Tests

Both models are verified and debugged through the experimental test set-ups: Furthermore, the tests also provide the charge current profile for the mathematical model. Then, obtained results are used later for the training and verification processes of the machine learning algorithms.

As described and referenced in section I, the behaviour of a supercapacitor is quite sensitive to several parameters. The following steps were used to certify and guarantee that the measurements are valid.

• Define the number of tests and their conditions and the measurement procedure.
• Design and build an evaluation fixture for testing different supercapacitors simultaneously.
• Implement the tests and obtain results.

3.1. Test Definition

The number and type of test conditions are applied to variables or parameters are shown in Table 1.

Table 1. List of variables and test conditions.

Variable	Temperature (°C)	Charge Voltage (V)	Charge Time (s)	Supercapacitor Values (F)	Discharge Time (s)
Test conditions	10, 25, 40, 60	2.5, 5	60, 600, 3600	1 F, 5 F	Infinity load, fixed value load, variable load steps

With this set of variables, 15 charge tests and 21 discharge tests have been implemented for each supercapacitor. Each test condition is reproduced in a climatic chamber. The test starts when the conditions are stabilized, and the power supply is turned on.

Once the power supply is turned on, the supercapacitor starts its charge. The charge process is isolated from the load to prevent energy losses. When the charge finishes, the power supply is disconnected from the supercapacitor. At this point, if there is no load, the self-discharge through the supercapacitor internal resistance starts. Different discharge profiles are obtained connecting an electronic load to the capacitor. The load discharge profile chosen ranges from a single constant discharge step up to five different constant discharge steps. This number is considered necessary to verify the mathematical and machine learning model, which may be of n stages.

3.2. Evaluation-Board

Figure 3b shows the evaluation board which is made of four test fixtures. The schematic of each text fixture is the same. The test circuit of Figure 3a has two switches and uses the following three states:

Figure 3. Evaluation board set-up. (a) Schematic of a single capacitor test fixture. (b) Complete test evaluation board.

• Charging: first switch on and second off.
• Self-discharge process: with the capacitor in charge state, the first switch is turned off.
• Load dependent discharge process: with the capacitor in charge state, the first switch is turned off and the second on.

This switched architecture emulates an energy-harvesting environment and prevents instantaneous charge of the supercapacitor. The circuit has a current limiter resistor of 100 Ω at the input that limits the maximum charging current to 50 mA. The load is modelled with a variable resistance, and the voltage is regulated with a low dropout voltage regulator of 2 V minimum input operation voltage (TPS79730-EP).

Unless the evaluation board has four capacitors, the measurement data obtained with two capacitors is enough to build, verify and validate the mathematical and machine learning model, the remain two are used to further verify the test. Therefore, the data present in this work comes from the models PM-5R0H105-R of 1 F and PHV-5R4H505-R of 5 F.

3.3. Test Results

The measurements of the charge current, charge and discharge voltage are obtained with a data logger switch unit, model 34970A and data acquisition periods of 1 s. The test set-ups results are summarised in Figure 4 and Figure 5. Traces of Figure 4 show the charge and discharge profiles of the two aforementioned supercapacitors under test for a fix charging time of 600 s and voltage of 5 Volts, at four operation temperatures of 10, 25, 40 and 60 °C. The results show that the temperature impact on the supercapacitor charge and discharge profiles increases as its capacitance value decreases. Electron movement is slower as the temperature decreases, which reduces the discharge rate. As the temperature builds up, the movement of internal electrons are faster, thus the number of the electrochemical reactions is higher, and therefore, the discharge rate increases.

Figure 4. Charge and discharge profiles at 10, 25, 40 and 60 °C for supercapacitors of 1 and 5 F.

Figure 5. Charge and discharge profiles of 1 and 5 F supercapacitor at different test conditions. (a) T = 25 °C, t_charge = 600 s, V_charge = 5 V, discharge profile (5 steps of I_leak 30 s, 3 mA 90 s, 0.058 mA 600 s, I_leak 1200 s and 0.3 mA until discharge). (b) T = 40 °C, t_charge = 3600 s, V_charge = 5 V, discharge profile (five steps: I_leak 2400 s, 3 mA 1800 s, 0.025 mA 2400 s, 1.5 mA 1200 s, I_leak until discharge). (c) T = 60 °C, t_charge = 600 s, V_charge = 5 V, discharge profile (3 steps: 0.3 mA 180 s, 0.1 mA 600 s, 0.058 mA until discharge). (d) T = 10 °C, t_charge = 600 s, V_charge = 5 V, discharge profile (single step: 0.02 mA until discharge).

Another test results are completed with the traces of Figure 5 that provide the charge and discharge profiles of the two supercapacitors tested when the temperature conditions, the charging time, the load and the discharge pattern change. The profiles selected are quite different in order to validate the mathematical model for different ambient conditions and ranges. The results show that:

• The supercapacitor discharge curves are a function of the load and the discharge time.
• Changes of the ambient conditions and load produce significant deviation of the discharge characteristic.
• The accuracy of the machine learning model is a direct function of the amount data.

4. Electro-Mathematical Model

The mathematical model selected is based in integro-differential equations. The electro-mathematical model presented combines the equations of [26,27,28,29,30,31,32]. As has been aforementioned in section II, the existing electro-mathematical models [26,27,28,29,30,31,32] implement only one or two parameters, which produces a limited model. The present work increases the number of parameters (capacitance life cycle, voltage, time, temperature, moisture, ESR, and leakage resistance) to characterize the supercapacitor behaviour and integrates them in a single model. Combining different parameters at the same time may produce unexpected or unwanted effects with an unknown root cause, therefore, the parameters are inserted and analysed one at a time to isolate and determine the origin of these issues whenever they occur.

4.1. Equations

Electro-mathematical model employs two equations [26], one for charge (Equation (1)) and one for discharge (Equation (2)). These equations create an exponential increase/decrease curve for each case, which are principles of the operation mode.

$$V_{charge}=I_{charge}\times R_{esr}+I_{charge}\times R_{leak}\times \left(1-e^{\left(\frac{-t}{R_{leak}\times C_{complet}}\right)}\right)$$

(1)

$$\begin{gathered} V_{discharge}=V_{charge}-I_{discharge}\times R_{esr}-(V_{discharge-1}+I_{discharge}\times R_{leak}) \\ \times \left(1-e^{\left(\frac{-t}{R_{leak}\times C_{complet}}\right)}\right) \end{gathered}$$

(2)

where V_charge/V_discharge is the charge/discharge value of the supercapacitor in Volts, I_charge/I_discharge is the profile of charge/discharge current in Amperes, R_esr is the equivalent series resistance in ohms, R_leak is the internal high resistance (leakage resistance) in ohms, C_complet de capacitance of the supercapacitor in Farads and T the time in seconds. C_complet, R_esr and R_leak values are a function of the ambient conditions, lifetime, applied voltage, moisture, etc.

4.1.1. Capacity C_complet

Supercapacitor full capacity is obtained multiplying the percent of change that life cycle introduces in theoretical nominal capacitance,

$$C_{complet}=C_0 \left(F\right)\times C_{cycle-life}(\%)$$

(3)

Capacitor lifetime decreases with increasing temperature, applied voltage, humidity, current and backup time requirements [27,28]. Therefore, operation at high temperature reduces the life of a device. According to the “10-degrees-rule”, lifetime of supercapacitor will be double for each 10 °C reduction in operating temperature. This rule employs the Arrhenius equation, a formula for the temperature dependence of reaction rates. Likewise, lifetime diminishes as operation time increases. In a supercapacitor, the internal capacity decreases by the root of cycles [29]. Moreover, this type of device has a single voltage range, therefore, if this level is exceeded, life cycle decreases, and eventually the device may break. Finally, the last parameter included in the present work life cycle equation is moisture. Unless this environmental condition has less affection than the other parameters, it is included in the equation to improve the accuracy of the algorithm for a real low power application. Thus, the capacitor lifetime is:

$$C_{cycle-life}(\%)=100-{\alpha }^{\left(\frac{T-T_{ref}}{10}\right)}\times \sqrt{N}\times \left(\frac{V-Vc}{V}\right)\times H$$

(4)

where, α is acceleration factor, (usually α equal 2 for this type of applications), T is the actual operation temperature in Kelvin degrees, T_ref is the reference operation temperature (usually 298 K), N is the number of cycles used, V is the voltage range, V_c is the applied voltage, both in Volts, and H is the percent of ambient moisture.

4.1.2. Equivalent Series Resistance

Equivalent Series Resistance (ESR) of supercapacitor is a function of temperature and device size [30]. The ESR increases as the temperature decreases. Consequently, when supercapacitor is operating at low temperature, its discharge efficiency diminishes. The ESR equation employed is:

$$R_{esr}=b_1\times R_{esr0}\times \left(1+\gamma \times \left(T-T_{ref}\right)\right)+b_2\times R_{esr0}\times e^{\left(\left(\frac{-K_t}{2}\right)\times \left(T-T_{ref}\right)\right)}$$

(5)

where T₀ = 293 K, γ = 0.007 (K⁻¹) the temperature coefficient of the aggregated aluminium and carbon constituents and kT = 0.045 (K⁻¹) the aggregate ionic activation energy constituent. The electronic (b₁) and ionic (b₂) weight factors are function of the supercapacitor cell size. Smaller cells have higher imbalance in the electronic to ionic ratio, in this case b₁ is a larger fraction of b₁ + b₂ = 1 [31]. R_esr0, is the theoretical series resistance and must be measured if the manufacturer does not provide the value.

4.1.3. Leakage Resistance

A supercapacitor usually has a high internal resistance, which implies a low leakage current, i.e., the current needed to keep the capacitor charged is low. When the capacitor is disconnected from the source, it begins to lose charge through the internal or leakage resistance. Manufacturers do not usually provide internal resistance information. Moreover, it is a parameter that is hard to measure. This work uses equation 6 [32], to obtain the instantaneous theoretical or approximate leakage resistance value, R_leak.

$$R_{leak}=\frac{{\tau }_{leak}}{C_{complet}+k\times u\left(t\right)}=\frac{{\tau }_{leak}}{C_{complet}+\left(\frac{2\left(I\Delta t_t-C_{complet}\Delta U_t\right)}{\Delta U_t^2}\right)\times U_0{e}^{\left(-\frac{t}{{\tau }_{leak}}\right)}}$$

(6)

where τ_leak is the time constant of leakage current, k is the linear voltage-dependent capacitance and u(t) is voltage across the capacitor over time and is an exponential function where U₀ is the initial voltage. The slope k is defined by the evolution of voltage ΔU_t with time and the stored charge on the supercapacitor.

4.2. Model Architecture

The aforementioned Equations (1) and (2) require post processing, Figure 6, to obtain their coefficients and define the capacitor equivalent curves. In order to simulate the desired operation conditions, the variables and the charge current profile, i.e., a data file, must be introduced manually in the equations.

Figure 6. Model architecture represented as a block diagram, including the program inputs and outputs.

The first step defines the capacitor charging process. Temperature, moisture, life cycle state, ESR theoretical resistance are introduced and the configuration charge voltage and time are inserted on Equation (1) and the program provides the charge curve.

The second step defines the discharge profile. There are two alternatives to define the discharge profile. The self-discharge one only uses the internal losses of supercapacitors. It describes how long the self-discharge discharge process takes, (usually a large period of time), and which is the amount of energy lost under different ambient conditions; thus, it does not require additional configuration parameters. The second option employs different loads to create discharge profiles, (those found in EHWSN applications as reference of low power system), therefore, for each discharge profile test performed, load and time step must be defined.

The complete mathematical model was implemented in MATLAB^®. It has a different algorithm for each case based on the aforementioned principles. All these algorithms have been integrated in a single program where the user can select the desired test set-up.

4.3. Results

This new electro-mathematical model is post-processed at different conditions of voltage, current, power and energy levels to analyse its performance and verify its accuracy. Figure 7 and Figure 8 summarizes the results of test set-ups devised through the charge and discharge profiles of the two reference supercapacitors under test. Figure 7 provides the effects for different load consumption discharge profiles on the behaviour of the same and different supercapacitor at the same temperature. Figure 8 presents how changing the temperature affects the supercapacitor’s storage capacity having the same load consumption discharge profile and shows which supercapacitor value goes into critical working level.

Figure 7. Mathematical model results with 1 F and 5 F supercapacitors, 600 s charge with 5 V at 40 °C temperature and different profiles of discharge. Yellow discharge profile has five steps all with 200 s of duration; First 1 mA, second self-discharge, third 0.5 mA, fourth 7 mA and fifth self-discharge. Purple discharge profile has also 5 steps; First 1.5 mA for 250 s, second self-discharge for 75 s, third 6 mA for 50 s, fourth 3.5 mA for 325 s and fifth self-discharge for 300 s.

Figure 8. Charge and discharge results obtained with the mathematical model for 1 F and 5 F supercapacitor values at different temperatures (10, 25, 40 and 60 °C), 600 s charge with 5 V and same outer profile consumption composed by three steps; First step 0.5 mA for 2000 s, second step 1 mA during also 2000 s and third step as first one.

5. Machine Learning Model

Artificial Neural Networks (ANN) algorithms is one algorithm to improve the accuracy of mathematical models. The machine learning model defined in this work is used to predict the charge and discharge curves of supercapacitors for the low power application field based on the electro-mathematical model and test measurements described in the previous section.

5.1. Artificial Neural Network Setup

One of the most commonly used families of Artificial Neural Networks is the feed-forward network, where connections between the neurons do not form a directed cycle. In this type of network, the information moves only in one direction, forward, from the input nodes, through the hidden nodes (if any) to the output nodes. Cycles or loops in the aforementioned ANN network do not exist.

The present work shows a multilayer perceptron (MLP), which is a feed-forward ANN model. It consists of multiple layers of nodes in directed graphs, with each layer fully connected to the next one. Each node is a neuron (or processing element), but the input one, with nonlinear activation function, usually shows a sigmoid function. Nonetheless, the output, in this case, is linear, as it is described in the numerical regression task. This type of network uses a variety of learning techniques, but back-propagation is the most popular one. This aforementioned technique is divided into two phases: propagation and weight update. Both phases are repeated until the performance of the network is good enough. In the present work, the output values are compared with the correct answer to compute the value of some predefined error-function. Then, the error is fed back through the network employing various techniques. With his information, the algorithm adjusts the weights of each connection until the error function reaches a target value. The previous process is repeated for a large enough number of training cycles until the ANN converges to a state were the computation error is low. In this case, one would say that the network has learned a certain target function and, thus, the ANN has learnt the target function.

In this research, we use an MLP with back-propagation learning technique to learn the charge and discharge curves of a supercapacitor. Thereafter, it predicts the aforementioned characteristics curves of new supercapacitors with the function previously obtained. The settings of the ANN configuration parameters are shown in Table 2.

Table 2. Setting configuration for the MLP network.

Parameter	Description	Value
hidden_layers	Name and size of the hidden layers.	(number of attributes + number of classes)/2 + 1
training_cylces	Number of training cycles used for the training phase.	500
learning_rate	Weight rate change of each step	0.3
momentum	It adds a fraction of the previous weight update to the current one.	0.2
decay	Flag to decrease or not the learning rate	False
shuffle	Flag to shuffle or not the data before learning	Yes
normalize	The Neural Net operator uses a usual sigmoid function as the activation function. Therefore, the value range of the attributes should be scaled to −1 and +1. This can be done through the normalize parameter. Normalization is performed before learning. Although it increases runtime, but it is necessary in most cases	Yes
error_epsilon	optimization training error target value.	1 × 10−5
use_local_random_seed	Flag to use a random seed for randomization	No
local_random_seed	Local random seed. It is only used if use_local_random_seed is set to TRUE	-

5.2. Objectives and Experimental Setup

The development of the present work has been performed using Rapid Miner software [33], which is a code-free data science platform with a collection of machine learning algorithms.

As mentioned in section III, the modelling was performed with data gathered from a set of 36 tests under different operating conditions. For each test, the set of variables given in Table 3 was gathered, which represents the structure of the final data set. The variable to be predicted was ‘VDC—real’.

Table 3. Variables used for the development of the prediction models.

Variable	Data Type	Values
Time	Integer
Temperature (°C)	Integer	10, 25, 40, 60
Voltage (V)	Integer	2.5, 5
Charge Time (s)	Integer	60, 600, 3600
Capacitance	Integer	1 F, 5 F
Charge current	Real	[0–45] mA

5.3. Evaluation Method

The objectives of the evaluation model are the following:

• Estimate and compare the goodness of the model regarding future outcomes predictions.
• Select the model amongst two or more models.

Ten-fold cross-validation model technique has been used for calculating RMSE measures in order to estimate the goodness of the model based on the difference between real and predicted values. Then, the best model (a ranking of models has been selected) is established based on the estimation of how accurately the predictive model will perform in practice.

5.4. Experimental Results

The modelling of the charge and discharge curves requires different approaches to achieve a valid algorithm. The 1st approach uses a Neural-Net algorithm because of the extent of its application and the performance it provides. This algorithm trains two models individually, one predicts charge curve and another one the discharge curve. The modelling of charge curve gives good results, meanwhile the modelling of the discharge curve deviates from the real one. The predictions show that the discharge curve is different depending on the charge time parameter. The second approach deals with the deviation issue developing a single charging curve model and three models for the discharge curve. The latter discharge models are obtained from datasets with different charge times.

In the 2nd approach, a single model for charging curve and three models for the discharge curve were developed, separating the dataset of the discharge curve according to the charge time used during the charge phase. The results for Ct60 and Ct3600 were improved (0.21 and 0.11 RMSE reductions, respectively) while for CT600, RMSE remains similar.

The 3rd and last approach was addressed to verify the initial hypothesis that charge and discharge curves should be modelled separately. This is done through the implementation and verification of three different models that include both the charge and discharge curves, with the charging time (t_charge) as a parameter. The results of Table 4 show that modelling the charge and discharge individually provides better results since, in all the cases, those models that train the curves independently present lower RMSE values.

Table 4. Obtained statistical results in different approaches and at different conditions.

Variable	Data Type	Values	RMSE	Sd
1st	Charge	For all conditions of the test	0.007	0.005
	Discharge	For all conditions of the test	0.222	0.035
2nd	Discharge	Charge-Time (60)	0.007	0.002
		Charge-Time (600)	0.232	0.091
		Charge-Time (3600)	0.111	0.024
3rd	Discharge and charge curve at the same model	Charge-Time (60)	0.021	0.002
		Charge-Time (600)	0.213	0.071
		Charge-Time (3600)	0.244	0.098

The 3rd approach results are displayed in Figure 9. Figure 9a shows the charge modelling and Figure 9b the discharge one. Figure 9a presents the charge curves for CT60, CT600 and CT3600 of 1 Farad and 5 Farad. These examples have 2.5 V or 5 V as charge value and 10 °C, 25 °C or 40 °C temperature conditions. Finally, Figure 9b provides the self-discharge curves for the tested supercapacitor with the charge conditions of Figure 9a.

Figure 9. Obtained results with 3rd approach. (a) Charge curves at different conditions and their (b) self-discharge profiles.

Figure 10 shows two examples of neuronal-trained network topologies generated during the experimental analysis. Figure 10a net represents the neural network model generated by applying the NeuralNet algorithm with the complete charging dataset, meanwhile Figure 10b network has been obtained using the same algorithm but with the discharging dataset reduced to 60 s of charging time. The weight of each neuronal connections is represented trough the track scale of shade; the darker the more important a connection is.

Figure 10. Neuronal networks topologies generated during the experimental analysis. (a) Charge neuronal network topology for all conditions. (b) Discharge neuronal network topology when charge time is 60 s.

The experiments show that the Neural Network algorithm provides results that are better or similar to those of the mathematical model, as explained in Section 6. Future work should develop and consider more machine learning to compare their strengths and drawbacks, considering the nature of the problem and the application.

6. Validation of Models with Experimental Data

The charge and discharge patterns obtained with the electro-mathematical model and machine learning model are compared with the measured data from the experimental tests. As has been reported through this work, all tests are oriented to low power applications takins as reference Energy Harvesting Wireless Network Sensor system. Consequently, the charge profiles are of low voltage levels and last short periods. The thirty-six implemented tests have been analysed with two methods, graphic visualization and data statistical techniques, using exported data from the different sources mentioned. In spite of this, there are techniques more accurate to validate the results; graphic tools where chosen because they give a quick validation. Four graphics comparing results achieved with experimental tests, electro-mathematical and machine learning models are shown in Figure 11. It must be mentioned that these examples have a 50% value of ambient moisture. Figure 11a represents a charge through 10 min with 5 Volt of input at 40 °C in a 1 Farad supercapacitor. Then, a unique consumption profile of 1 mA is applied for discharge. As is shown in this example, the discharge curve descends quickly because of reproduced constant consumption of a low power system. Figure 11b represents a 2.5 V of charge for 10 min at 25 °C in a 5 Farad supercapacitor.

Figure 11. Graphic results and comparisons of described -four different tests. (a) 1 Farad supercapacitor charged through 10 min with 5 V at 40 °C and 1 mA consumption discharge, (b) 5 Farad supercapacitor charged for 10 min with 2.5 V at 25 °C and self-discharge procedure, (c) 1 Farad supercapacitor charge during 1 h with 5 Volt at 10 °C and self-discharge procedure, (d) 5-Farad supercapacitor charge through 10 min with 5 V at 40 °C temperature and three discharge steps: 0.3 mA consumption through 3 min, 0.1 mA consumption through 10 min and self-discharge.

After charge time, the represented profile belongs to the self-discharge curve. Then, Figure 11c represents a 10 °C temperature, 1 Farad supercapacitor charge during 1 h with 5 Volt of input. In this case, the discharge procedure is also based on self-discharge technique. In the last displayed test, Figure 11d represents a 5-Farad supercapacitor charge with 5 V at 40 °C temperature for 10 min. Discharge has three different steps. The first one has 0.3 mA consumption profile through 3 min, the second one has 0.1 mA consumption profile through 10 min and represents the self-discharge process to the end.

Visual check is enough to certify the result and final approval is complemented with statistical techniques. Therefore, the results are verified and validated through the root mean square deviation (RMSE), Mean Absolute Error (MAE), Mean Square error (MSE) and Mean Absolute Percentage Error (MAPE). The value of the RMSE predicts the error for each time slot in a single data profile. With this value, the accuracy of each profile is known and the behaviour of models in different work conditions are analysed. MAE measures and validates differences between the profile of the models generated. MAE provides the difference between profiles at specific time slots and in a complete profile. In addition, MSE is used to know the error introduced in each profile. This value gives the error produced when parameters or information during the construction of the algorithm are not considered. The last statistical method used is MAPE; it measures prediction accuracy between forecast (math or prediction) and experimental data. This value is expressed in percentage error. This error will be lower in most cases because machine learning models learn from experimental data instead of approximate data, as electro-mathematical models do. Table 5 shows the statistical validation results obtained with RMSE, MAE, MSE and MAPE techniques. The last column contains statistical average results of all tests performed in the present work. This is the key column to weigh the accuracy, precision and reliability of each model and the differences between them.

Table 5. Obtained statistical errors from prediction and electro-mathematical models on three described tests and the average of 36 tests.

	Statistical Technique	(a) Example		(b) Example		(c) Example		(d) Example		Average All
		Pred.	Math	Pred.	Math	Pred.	Math	Pred.	Math	Pred.	Math.
Charge	RMSE	0.137	0.204	0.113	0.178	0.007	0.145	0.005	0.134	0.052	0.146
	MSE	0.019	0.042	0.013	0.032	0.0001	0.021	0.00002	0.018	0.005	0.023
	MAE	0.123	0.125	0.103	0.167	0.006	0.114	0.004	0.118	0.047	0.101
	MAPE	3.606	5.354	4.689	11.642	0.655	4.387	1.514	8.607	3.549	5.712
Discharge	RMSE	0.119	0.123	0.071	0.083	0.157	0.048	0.074	0.086	0.099	0.134
	MSE	0.014	0.015	0.005	0.007	0.025	0.002	0.005	0.007	0.017	0.031
	MAE	0.079	0.111	0.064	0.078	0.143	0.047	0.069	0.081	0.090	0.128
	MAPE	2.013	3.210	2.211	2.653	3.659	1.230	2.272	2.685	2.819	3.792

Table 6 provides a comparison of the error between the reference models and the models of the present research work. The error figures of Table 6 must be used with care because the reference models have specific target applications and operating conditions, and the research models are devoted for low power electronic systems and generic conditions of temperature, humidity, capacitance, load and charge and discharge time, thus, some deviations are expected.

Table 6. Comparison between this work and other model.

Model Type	Ref. Numb.	Statistical Error	Ref. Result	This Work Result
Electro-mathematical	[12]	Deviation	3–7%	2.35%
Electro-mathematical	[34]	Deviation	2.56%	2.35%
Electro-mathematical	[35]	RMSE	0.65	0.071–0.204
M.L. → ANN	[23]	MSE	0.089	0.005–0.017
M.L. → Kalman Filtering	[36]	MAE	0.50	0.047–0.090
		RMSE	0.63	0.052–0.099

7. Conclusions

The mathematical model and machine-learning algorithm developed in the present work is devoted to low power electronic devices in general, but the consumption patterns are those of Energy Harvesting Wireless Sensor Network applications. It constitutes an improvement in the state of supercapacitor electro-mathematical models for low power applications because it uses more characteristic parameters. Moreover, the simulation and tests results show that it provides the same accuracy of electro-chemical and higher accuracy than the state of the art mathematical models. Furthermore, the improvement of the electro-mechanical model with machine-learning algorithms needs less experimental data than electrochemical models to achieve the same accuracy. Besides, the machine-learning algorithm does not require any electrochemical parameters to implement the model, and the aforementioned parameters, if compared with the electromechanical ones, are not hard to obtain empirically.

The experimental tests show that the algorithms presented are able to model all types of supercapacitors for low power applications and any type of environment conditions. On the other hand, unless the model of machine learning increases the accuracy of the electro-mathematical model, it is less generic as it depends on predefined test conditions and requires more processing resources, i.e., time. Therefore, the accuracy requirements and the simulation time available determines which model should be used: the electro-mechanical or the machine learning model.