Satisfaction-Based Energy Allocation with Energy Constraint Applying Cooperative Game Theory

: There has been an effort for a few decades to keep energy consumption at a minimum or at least within a low-level range. This effort is more meaningful and complex by including a customer’s satisfaction variable to ensure that customers can achieve the best quality of life that could be derived from how energy is used by different devices. We use the concept of Shapley Value from cooperative game theory to solve the multi-objective optimization problem (MOO) to responsibly fulﬁll user’s satisfaction by maximizing satisfaction while minimizing the power consumption, with energy constrains since highly limited resources scenarios are studied. The novel method uses the concept of a quantiﬁable user satisfaction, along the concepts of power satisfaction (PS) and energy satisfaction (ES). The model is being validated by representing a single house (with a small PV system) that is connected to the utility grid. The main objectives are to (i) present the innovative energy satisfaction model based on responsible wellbeing, (ii) demonstrate its implementation using a Shapley-value-based algorithm, and (iii) include the impact of a solar photovoltaic (PV) system in the energy satisfaction model. The proposed technique determines in which hours the energy should be allocated to maximize the ES for each scenario, and then it is compared to cases in which devices are usually operated. Through the proposed technique, the energy consumption was reduced 75% and the ES increased 40% under the energy constraints.


Introduction
Energy is the backbone of modern society. It provides the means to support everyday infrastructure, such as hospitals, schools, and homes. In the case of homes, for most of the 90 percent of the global population with access, it is difficult to imagine living without electric energy [1]; it powers our essential needs, such as water pumping, lighting, cooling, and very often also cooking, among others. Additionally, electric energy provides comfort and entertainment, and although these are not essential needs, they can also improve quality of life.
Throughout the years, with an increasing energy demand, managing energy consumption has become important. Demand Side Management (DSM) encapsulates those strategies that change the main power consumption to better match the power supply. Through DSM methodologies, one of the purposes is to create an energy demand scheduling to benefit a household.
Many DSM studies focus on minimizing energy cost, achieving utility stability, and shifting peak demand. Wu et al. [2] proposed a mixed-integer linear programming (MILP) model for the energy system optimization to reduce the annual cost in a building distributed heating network. Similarly, Tang et al. [3] proposed a game theoretic method to maximize net profit and reduced demand fluctuation using real data of building on a campus in considering responsible wellbeing in terms of energy consumption. The energy threshold hypothesis is defined. It was previously presented by Max-Neef [24] in terms of economic growth and quality of life. The proposed MOO consists of responsibly fulfilling user's needs by maximizing satisfaction while minimizing the power consumption. A novel model is proposed to include customer's satisfaction in an optimization problem to minimize the energy consumption. To summarize, the contributions of this paper can be highlighted as follows: • Energy satisfaction (ES) is proposed to capture the benefit of energy uses and to model the optimization problem.

•
The Shapley Value algorithm is implemented to maximize ES and minimize energy consumption. • The proposed model also integrated solar-based renewable-energy resources (RESs). Real data from [25] was used for validation.

Overview of Game Theory
Game theory provides a series of analytical tools, which allows us to understand what is observed in decision-making interactions. The foundation of the theory is formed by two basic assumptions: decision-makers are rational, and reason strategically by considering the expectations of other decision-makers' behaviors. Real-life situations are modeled by game theory through highly abstract representations, thus, allowing their use to study problems in many fields [26].

Types of Games
There are noncooperative games and cooperative (or coalitional) games. In the former, each action is taken by a single player in response to the other players [27]. In cooperative games, the model consists of the set of joint actions that each group of players (or coalition) can take in response to the other players. Cooperative games are concerned with the interactions among players, the value of each coalition and how the value can be distributed to the participating players.

Shapley Value
The Shapley Value [28] is a solution concept for coalitional games along with the core, nucleolus and Pareto optimal, among others. Given a coalitional game (N , v), there is a unique feasible payoff division x(v) = ϕ (N , v) that divides the full payoff of the grand coalition. The Shapley Value can be defined as [29], where R is the set of all N! orderings of N , P i (R) is the set of players preceding i in the ordering R and ϕ i (N , v) is the expected marginal contribution over all orders of player i to the set of players who are preceding it [26]. The Shapley Value also satisfies the following axioms [30]: • Symmetry: The symmetry axiom states that interchangeable players i and j should receive the same payments: . Two agents are interchangeable if they contribute the same amount to every coalition with the other agents. • Dummy Axiom: The dummy player i contributes to any coalition the same amount that i can achieve alone. Thus, for any v, if i is a dummy player, then ϕ i (N , v) = v(i). • Additivity: The additivity axiom states that for any two coalitional game problems, defined by v 1 and v 2 , we have for any player i that • Efficiency: The efficiency axioms states that the entire payoff is divided among the players, so ∑ N 1 ϕ i = v(N ), where ϕ i is the Shapley Value of player i.

Satisfaction Concept
The human experience is complex to describe since it depends on the reality of each person. Satisfaction, on the other hand, is a more general concept and describes the user's perception and how its expectation can be fulfilled [7]. This research proposes a model to quantify satisfaction in the context of a household with different devices. The use of each device provides different levels of satisfaction at different times of the day. Daily moments are captured and summed up to build the energy satisfaction concept. As part of this model, we also consider excessive and poor consumption.

User Input Satisfaction
The proposed algorithm will be finding the hours of the day for which energy should be allocated to achieve the maximum satisfaction at a minimum energy consumption. For the user satisfaction, it is assumed: • Satisfaction among devices can be compared at two levels: time-based, satisfaction Ω, and device-based satisfaction ∆, as defined by Ogunjuyigbe [8]. The former implies that if there is a device no. 1, then the satisfaction it is providing at time t 1 , (Ω 1 (t 1 )), can be compared with the satisfaction the same device is providing at different time t 2 , (Ω 1 (t 2 )). For the latter, if the two devices need to be used at the same time, then there is a satisfaction derived from using device no. 1 (∆ 1 (t 1 )), which can be compared with the satisfaction derived from using device no. 2 (∆ 2 (t 1 )) at that hour. • Time-based satisfaction Ω has an integer numerical value from zero (0) to six (6), where six (6) means completely satisfied, three (3) means neutral, and small satisfaction values, such zero (0), one (1) and two (2) will denote dissatisfaction. • Three (3) levels of time-based satisfaction are identified according with these seven (7) scores, see Table 1.

Power Satisfaction
The power satisfaction will not only depend on the user input satisfaction set by the household's head. PS also depends on the energy consumption patterns at each hour of the day. Thus, PS depends on the number of hours of continuous usage (CLoU) and the total length of use in a day (LoU). To find the PS at time t, we need to analyze the previous 24 h, i.e., from time t − 24 until time t − 1. It is suggested that PS cannot be tested based on how satisfied a person is at t but it will be affected by its perception of the last 24 h.

Equation of Power Satisfaction
Power satisfaction at a given time/hour can be expressed as the function in Equation (2) below: where, where, α is related to the customer's answer, β is related to CLoU and γ is the related to LoU. The main idea is to penalize the initial satisfaction, α according to an excess or poor consumption. Variables β and γ depends on α and convert t u and t t , respectively, into a value that can be deducted from α and, where Ω i is the time-based input satisfaction for device i and it is divided by the responsible LoU, t t i , hence α will reach its maximum if it used the expected time t t i hours during a 24-h period. To find PS, 24 h of experiment is needed. For each device i, the array u i = {u n : n = 1, 2, . . . 24} where u n is 0 or 1. It is the input vector for each device with the operational status of the devices, it will be one (1) when 'ON' and (0) if it is 'OFF'. For further details, see previous work reference [19].

Energy Satisfaction
Each user has a subset {i : i = 1, 2, . . . , N} of N participant devices. Following the well-known concept of electric energy, to find the energy satisfaction, ES, in time t, the previous 24 (5).
where N is the number of participant devices, k is the initial time of the experiment, t = k + 23 and ∆ i,t is the device-based satisfaction. Equation (5) is modified to include the concept of device-based satisfaction ∆. ES becomes a weighted summation and reflects the specific needs at that current time t. Subsequently, the average is computed to obtain the ES value at time t,

Electric Energy Function
The energy consumption in an hour from time t until time t + 1 is defined as in Equation (7), where e i,t represents an usual energy consumption of device i in one hour and u i is the input vector for each device i with its operational status. It will be one (1) when 'ON' and zero (0) if it is 'OFF'.

Optimization Problem
The problem of jointly maximizing the satisfaction while minimizing the power consumption f 2 (u) can be formulated as a multi-objective (MOO) problem: where,

Constraint
The algorithm is subjected to the constraint that the total energy consumption of the user is less or equal to a pre-defined energy budget, E con ,

Cooperative Game Model Implementation
A mathematical model based on cooperative game theory is developed to capture the complex interactions among the different devices. The Shapley Value algorithm based on the cooperative game implementation is applied to all participating devices to obtain the u i vector. Such that energy can be allocated to simultaneously maximize the ES and minimize the energy consumption L for a corresponding energy reference. The flow chart for the proposed algorithm is shown in Figure 1. Time-based and device-based satisfaction tables and power consumption for each device are established. The proposed approach is to design a consumption scheduling for a selected number of hours n. Additionally, we should decide time t to start the experiment and the number N of participating devices. Considering the experiment should collect energy data for the 48 h before this time t, then the status matrix is of size 48 × N. The number of possible action profiles A, is found (2 N ). Since each possible action profile contains the possible devices status, each of these possible profiles are multiplied for the correspondent Device-based satisfaction ∆ according to the time of the day and the device. PS and ES are computed using Equations (2) and (6) at each time slot, respectively. Next, the worth of the coalition v(S k ) can be found for each time slot for each profile. Lastly, the group of actions with maximum SV can be selected.
To implement this algorithm the following considerations are followed: • Players: Devices-agents i = 1, 2, . . . , N are considered the players.
• Actions: Devices can be ON or OFF, so the actions would be turning ON or OFF the participant devices (players.) The u i array will be zeros or one depending on the device status.

•
Payoffs: Consumption and ES are combined and modeled as optimization function using the characteristic function.

•
Value of coalition: Agents form coalitions and every coalition and corresponding actions have different value. For K ⊆ N, S K is the group of K devices that agreed to make a coalition to maximize the total expected payoff v(S K ) that they can achieve together. The value of a coalition for this cooperative game is, where f is a factor to penalize level of consumption among the energy reference and solar energy less than cases when less then solar energy and P PV [t] is the minimum among the set of power output of generation(P pv [t]) and the power inverter , for the cases with just utility and P PV [t] = min P pv [t] + P bat , P inv [t] for the cases with batteries and utility, where P bat is the set of power output from batteries. There are three levels of consumption: L ≤ P PV , P PV < L ≤ E re f and L > E re f . Where E re f would be a consumption that is acceptable, there will be penalty if it is less than P PV or greater than E re f . To implement this algorithm the following considerations are followed: • Players: Devices-agents = 1,2, . . . , are considered the players.

Case Study: House with Photovolatic System (PV) and Utility or House with PV, Batteries and Utility
The proposed model is being validated by representing a single rural residential house that may contain a small photovoltaic (PV) system that is composed of one or more solar panels combined, a DC/AC inverter, and it is grid-connected. Figure 2 describes a generalized version of the system. Energy data to simulate the case study is obtained in two ways:

•
Reference Energy Disaggregation Data Set (REDD) [31] for power ratings and status vector (u i ) for refrigerator, microwave, lighting, stove, and water heater. • Power ratings for TV, AC, Radio and phone, are obtained from Ogunjuyigbe [8]. Status vectors for these four devices are randomly generated.

Household's Load
For the load data, a single house's devices (four of them) are analyzed, the REDD data [31] is used to represent a part of house's loads. Data that is shown in Table 2 contains average power reading for the individual circuits of the house. The REDD data was sampled every three seconds. Hence, there are 20 data points per minute. The array is an hourly vector. For this vector, it needs to be decided how much time within an hour a certain device must be ON to assign a one (1) in that time slot.

Household's Load
For the load data, a single house's devices (four of them) are analyzed, the REDD data [31] is used to represent a part of house's loads. Data that is shown in Table 2 contains average power reading for the individual circuits of the house. The REDD data was sampled every three seconds. Hence, there are 20 data points per minute. The u i array is an hourly vector. For this vector, it needs to be decided how much time within an hour a certain device must be ON to assign a one (1) in that time slot. Satisfaction matrices were generated according to author's personal experience and they are described in Tables 3 and 4. This first stage of results is simply used as a sanity check. In the Section 5.3, actual load description (see Table 5) is used for each device based on REDD database and Ogunjuyigbe et al.'s [8] work. Besides, actual values of satisfaction were used to generate Tables 6 and 7. The authors personal experience reflects and summarizes how devices were prioritized by most of the population in extreme conditions such as those experienced after hurricane Maria in 2017 in Puerto Rico. However, future work will include field data to build the time-based satisfaction and device-based satisfaction tables. Figure 3 depicts the amount of energy used by each of the four devices under the proposed SV allocation algorithm and the actual energy usage according to the REDD database [31]. The proposed optimization problem has been constrained. Thus, showing how with the same amount or less energy than in the actual REDD scenario. ES (see Figure 4) is higher at each time slot.

Results
Energies 2021, 14, x FOR PEER REVIEW 10 of 18 work will include field data to build the time-based satisfaction and device-based satisfaction tables. Figure 3 depicts the amount of energy used by each of the four devices under the proposed SV allocation algorithm and the actual energy usage according to the REDD database [31]. The proposed optimization problem has been constrained. Thus, showing how with the same amount or less energy than in the actual REDD scenario. ES (see Figure  4) is higher at each time slot.  Refrigerator 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1  3 Stove

Results
Microwave 4 Figure 6 shows the resulting ES from that energy usage. Lighting is suggested to be on early in the morning and in the afternoon when the satisfaction derived from them is the highest. Therefore, the refrigerator needs to be off for one hour because it is a priority (see Table 4 to have lighting on since its contribution to the total ES is higher. However, analyzing ES brought by refrigerator in Figure 6 is slightly lower for the proposed algorithm since it was turned off. According to the REDD database [31], the stove is in on status during one hour but it is on at a time that is not bringing any satisfaction, hence the proposed algorithm recommends to not turn it on under the energy restrictions. For the microwave, in the REDD database [31], it is on at different times of the day however it is not on when the satisfaction is the highest. For example, in the morning it is preferable to have lighting on instead of the microwave, since the energy usage of lighting is less than the energy usage of the microwave.
For the calibration part, we made sure the algorithm was suggesting those profiles where the Energy Satisfaction was highest at a minimum energy usage. In the following section, we will be testing the algorithm for a different set of devices.     Figure 5 depicts how power is used in both scenarios. Figure 6 shows the resulting ES from that energy usage. Lighting is suggested to be on early in the morning and in the afternoon when the satisfaction derived from them is the highest. Therefore, the refrigerator needs to be off for one hour because it is a priority (see Table 4 to have lighting on since its contribution to the total ES is higher. However, analyzing ES brought by refrigerator in Figure 6 is slightly lower for the proposed algorithm since it was turned off. According to the REDD database [31], the stove is in on status during one hour but it is on at a time that is not bringing any satisfaction, hence the proposed algorithm recommends to not turn it on under the energy restrictions. For the microwave, in the REDD database [31], it is on at different times of the day however it is not on when the satisfaction is the highest. For example, in the morning it is preferable to have lighting on instead of the microwave, since the energy usage of lighting is less than the energy usage of the microwave.
Energies 2021, 14, x FOR PEER REVIEW of the microwave, since the energy usage of lighting is less than the energy usa microwave.
For the calibration part, we made sure the algorithm was suggesting those where the Energy Satisfaction was highest at a minimum energy usage. In the f section, we will be testing the algorithm for a different set of devices.      Table 7. Device-based satisfaction. Hours   1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22

Data Characterization for the Algorithm Testing.
Eight devices were selected according to the ones used in Ogunjuyigbe et al. [8] for testing purposes. For the algorithm simplicity, all lights were considered as a single appliance. Ogunjuyigbe et al. [8] used some loads for which there are not data available in the REDD database [31], such as TV, AC, Radio, and phone. Hence, for these ones, Unit Wattage data from [8] was used. Additionally, was randomly generated for those devices, such that a comparison can be made with the proposed algorithm's output. Table 5 describes the electrical appliances used by a user, their rating, their optimum CLoU, and their optimum LoU, for a responsible consumption. For the calibration part, we made sure the algorithm was suggesting those profiles where the Energy Satisfaction was highest at a minimum energy usage. In the following section, we will be testing the algorithm for a different set of devices.

Data Characterization for the Algorithm Testing.
Eight devices were selected according to the ones used in Ogunjuyigbe et al. [8] for testing purposes. For the algorithm simplicity, all lights were considered as a single appliance. Ogunjuyigbe et al. [8] used some loads for which there are not data available in the REDD database [31], such as TV, AC, Radio, and phone. Hence, for these ones, Unit Wattage data from [8] was used. Additionally, u i was randomly generated for those devices, such that a comparison can be made with the proposed algorithm's u i output. Table 5 describes the electrical appliances used by a user, their rating, their optimum CLoU, t u and their optimum LoU, t t for a responsible consumption.

House Head's Satisfaction
The algorithm also needs practical input data for satisfaction to create the model. Data from Ogunjuyigbe et al. [8] (σ t and σ d ) are being used for this purpose. Data from time-based satisfaction was mapped into satisfaction levels described in Table 1 in the following fashion, Dissatisfaction and 'indifference' values (0, 1, 2 and 3) are mapped into negative values and zero (−3, −2, −1 and 0, respectively). The proposed model introduced negative values when low satisfaction, thus making it preferable to have them 'OFF', representing by itself, before the optimization problem, a more accurate satisfaction model, which allows making decisions not only for energy and economic savings but also responsibly fulfilling the customer's satisfaction. Table 6 shows complete resulting time-based satisfaction table and Table 7 shows device-based satisfaction after mapping d-domain satisfaction found in Ogunjuyigbe et al. [8] (σ d ) by using Equation (16).

Results
One of the main results to report is that the implementation of the SV optimization provided a consumption pattern that represents an energy consumption less or equal than the initial actual use and a higher energy satisfaction for almost all hours in all devices. Figure 7 provides the graphical comparison of the power consumption at each hour for the actual based case and the case with the SV optimization. Figure 8 presents the energy satisfaction at each hour showing how the SV optimization outperforms the base case, particularly increasing its advantage in the early morning hours and the late evening hours.      Figure 9 depicts a comparison of the hourly power consumption of all devices between the proposed SV allocation algorithm and the actual power consumption. In Figure 9 (left side), the algorithm attempts to meet desired time-based and device-based satisfaction tables (See Tables 6 and 7) while consuming equal or less hourly power than the one shown in 9 (right side). The energy reduction was of approximately 75%, from 32.6 KWh to 7.35 KWh. Equally important is the energy satisfaction increase of 40% with the SV algorithm, from 5500 to 7825. A consumption plan is scheduled for the user by managing devices based on the SV game theory approach. Next, a reliability signal or economic signal will be sent to a Human-Machine Interface (HMI). A reliability signal will ensure that the electric system keeps operating when a house is not connected to the grid, while an economic signal ensures the same purpose when connected to the grid. This way, the customer is aware of the situation and can make a final decision based on the available information.     Figure 10 includes the ES for each device and it seems that the better SV performance is due mainly to the microwave use in the morning and the TV at night. Since operational status vectors, u for TV, AC, Radio, and phone, were randomly generated, Figure 9 shows an atypical consumption pattern. Figure 10 shows a comparison between the hourly ES obtained through the SV allocation algorithm and the ES obtained in the actual case representation, for each of the devices. When attempting to meet the energy constraints imposed by the actual case scenario, with the proposed SV algorithm, the ES obtained at each hour is equal or higher for almost each of the hours for every device. This is one of the most important results of this study. Only in the case of the laptop, the ES is higher at the last hour of the day in the actual case scenario.
Similarly, Ogunjuyigbe et al. [8] presented their results in 24 h plots for three different daily budget constraints to provide a maximum satisfaction at those predefined budgets. On the other hand, the present research used energy constraints rather than budget, and thus including a key component of the research when penalizing excessive and low consumption, because of its detrimental impact in the quality of life. He compared a 'desired satisfaction' with an 'achieved satisfaction'. The 'achieved satisfaction' was the output of their load-satisfaction algorithm, which is analogous to the present 'SV load allocation' algorithm. We did not choose to compare the output results with the 'desired satisfaction' (as Seen in Tables 6 and 7). Instead, we compare it with an 'actual' scenario represented by using the REDD database. Ogunjuyigbe et al. [8] implemented a genetic algorithm (GA) approach. While the GA approach may have a good convergence speed and good efficiency, the present approach does not have to deal with convergence times and offers a more intuitive optimization framework.

Conclusions
A novel model to include customer's satisfaction in an optimization problem was introduced. A quantifiable user satisfaction was developed. The satisfaction concept through the novel concepts of power satisfaction (PS) and energy satisfaction (ES) included the detrimental impact that excess consumption could have in the quality of life. The algorithm provided the hours of the day for which the energy should be allocated to achieve maximum satisfaction at an energy constraint imposed by the energy consumption in actual case scenarios. Actual case scenarios were represented by using the REDD database. The Shapley Value (SV) concept from the game theory framework was implemented to obtain a recommendation on how energy should be allocated. The results showed how the algorithm maximized user s ES at a minimum energy consumption. The proposed approach reduced energy consumption 75%, while increasing ES 40%.
SV-based optimization successfully achieved to maximize satisfaction at a minimum energy consumption although it also has high computational complexity. Further work can be done to decrease computational complexity and thus the required reduction processing times. This could be achieved by using more powerful computers or through the derivation recursive and/or parallel implementation of the proposed algorithm. The proposed methodology is validated by simulating a rural single house with limited resources connected to the grid. It should be pointed out that the satisfaction model can be readily applied in a real case scenario of rural communities.