A Non-Intrusive Load Monitoring Method Based on Feature Fusion and SE-ResNet

Abstract

In the study of non-intrusive load monitoring, using a single feature for identification can lead to insignificant differentiation of similar loads; however, multi-feature fusion can pool the advantages of different features to improve identification accuracy. Based on this, this paper proposes a recognition method based on feature fusion and matrix heat maps, using V-I traces, phase and amplitude of odd harmonics, and fundamental amplitude. These are converted into matrix heat maps, which can retain both large and small eigenvalues of the same feature for different loads and can retain different features. The matrix heat map is recognized by using SE-ResNet18, which avoids the problem of the classical CNN depth being too deep, causing network degradation and being difficult to train, and achieves trauma-free monitoring of home loads. Finally, the model is validated using the PLAID and REDD datasets, and the average recognition accuracy is 96.24% and 96.4%, respectively, with significant recognition effects for loads with similar V-I trajectories and multi-state loads.

1. Introduction

The two main approaches used in existing load monitoring systems can be categorized under non-intrusive load monitoring (NILM) and intrusive load monitoring (ILM). ILM involves installing smart power plugs directly at each electrical socket. NILM does not intrude upon the occupants during data collection and only requires the installation of a single power meter for the entire floor or building [1]. NILM does not disturb the occupants during the data collection process and only one meter has to be installed. Therefore, NILM is one of the important directions for future smart grid development. It can not only help users to adjust their own electricity consumption behavior and save electricity, but also help the grid to realize fine-grained load sensing, accurately profile user behavior, and offer differentiated and precise services [2]. For the moment, load-monitoring has been applied to power system diagnostics [3], smart energy management [4], appliance recognition [5], and energy dashboards [6].

Early NILM used single features for identification, and the common features include active power, reactive power, current waveform, steady-state current harmonics, power harmonics [7,8,9,10], and V-I trajectory [11], among which V-I trajectory has better identification [12]. However, with V-I trajectory it is difficult to distinguish loads with similar trajectories. Power features are difficult to distinguish with low-power loads and harmonic currents cannot distinguish loads with less current distortion. Therefore, multi-feature fusion has become a current research hotspot. Yi Sun et al. proved that harmonic features can effectively distinguish low-power loads and fuse the third harmonic with power [13]. Z Zheng et al. verified that harmonic current features are additive, and a multilayer perceptron classifier is constructed using harmonic current features to accurately identify typical nonlinear and resistive loads [14]. Shouxiang Wang et al. used convolutional neural networks and BP neural networks to convert V-I trajectories and power into vectors before fusion, but the overall training time of the model is long, and it is prone to the problem of local minima [15].

Some scholars transformed the signal data into image data and used image recognition techniques for recognition. De Baets L et al. encoded the signal data into images and used V-I trajectories to generate binary images, but some features of V-I trajectories disappeared after conversion to binary images [16]. Junfeng Chen et al. constructed load identification by Gram matrix color coding [17]. Xiang Y et al. first converted the power and current magnitude of the load into color information by color coding, and then fused the features with the V-I trajectory. However, the fused image had a single color, the power and current differed greatly between loads and determining the color by power or amplitude will lead to most of the loads having lower power or current concentrated in the same color interval, which will affect the recognition accuracy [18]. Finally, this literature used convolutional neural networks (CNN) for recognition, and as the number of layers of the network increased, the model degenerated and the gradient exploded or disappeared [19].

Based on this, a non-intrusive load monitoring method based on multi-feature fusion and image recognition is proposed in this paper. The voltage and current of the V-I trajectory, the amplitude and phase of the odd harmonics, and the fundamental amplitude are converted into numerical values. Then, a feature matrix is built, in which the features are superimposed according to the number of times they appear at each position, and then the superimposed matrix is color-coded according to the size of the values and converted into a matrix heat map. Finally, a residual network18 (ResNet18) with squeeze and excitation (SE) is used for image recognition. This paper effectively solves the similarity load identification problem and the masking problem of features generated when performing feature fusion. Unlike most related solutions [20,21], the proposed NILM approach does not use a dedicated one-to-one model for each load but is implemented on the basis of a single multi-class classifier. That is, one model is used to identify multiple loads, i.e., one model is used to identify several loads. Therefore, it is more computationally efficient [22]. The average recognition accuracy of the proposed method is 96.24% and 96.4% for loads with similar V-I trajectories, 96% and 94.9% for loads with similar V-I trajectories, and 97.79% and 96.5% for different states of the same load, respectively, when tested with PLAID and REDD datasets, and compared with other recognition methods. The results show that the proposed method has higher accuracy.

2. Feature Construction

2.1. Single-Class Feature Processing

V-I traces are extracted by event detection. Detecting the load start-stop event, the voltage before the load is turned on is $v_{off}$, and the voltage after it is turned on is $v_{on}$. Accordingly, the current before it is turned on is $i_{off}$, and the current after it is turned on is $i_{on}$. Before and after the cut the voltage does not change much and, considering the possibility of certain fluctuations, the average voltage before and after the event is calculated as follows:

$$V=\frac{v_{off}+v_{on}}{2}.$$

(1)

Using a differential extraction method that uses the current after the event minus the current before the event, it is possible to obtain the load data for the start-up:

$$I=i_{on}-i_{off}.$$

(2)

The V-I trajectory can be formed by taking the voltage and current in one cycle as the horizontal and vertical coordinates, respectively [11].

Figure 1 shows the V-I trajectories of several loads with large differences, and Figure 2 shows the V-I trajectories with large similarities, in which the voltage and current data have been normalized.

Decomposition of the steady-state current signal using the fast Fourier transform (FFT) is as follows:

$$I=A_1\sin \left(wt+{\theta }_1\right)+A_2\sin \left(wt+{\theta }_2\right)+\dots +A_k\sin \left(wt+{\theta }_k\right),$$

(3)

where $A_1$, $A_2$, $A_k$ is the amplitude of each harmonic, and ${\theta }_1$, ${\theta }_2$, ${\theta }_k$ is the phase angle of each harmonic.

The load of Figure 2 was harmonic decomposed, and it was compared and found that the fundamental of the load has a large variability with the 3rd, 5th, 7th, and 9th harmonics, see Figure 3. Fusing it with the V-I traces can improve the resolution.

Harmonic amplitude and phase are numerically large and fusing them with V-I traces requires data processing for voltage, current, and amplitude phase. To normalize the selected data:

$$K^{\prime }=\frac{K-K_{\min }}{K_{\max }-K_{\min }},$$

(4)

where: $K$ denotes the original value and $K_{\min }$ denotes the minimum value of a period. $K_{\max }$ denotes the maximum value of a period. $K^{\prime }$ denotes the normalized value.

Rounding of the normalized data is as follows:

$$K_{sure}=Floor\left(K^{\prime }\times \left(N-1\right)\right)$$

(5)

$$A^{\prime }{}_f=Floor\left(A_f\right)÷N=x_f\cdots y_f,$$

(6)

where: $K_{sure}$ and $A^{\prime }{}_f$ denote the finalized eigenvalue. $A_f$ denotes fundamental amplitude. $x_f$ denotes quotient. $y_f$ denotes remainder. $Floor$ denotes rounding down. $N$ denotes the order of the matrix.

2.2. Matrix Heat Map Construction

Create a 32 × 32 matrix and draw grid lines. Determine the position of the features in the matrix in the form of values in ($I_{sure}$,(31-$U_{sure}$)), ($i$,$A_{sure}$), ($i$,${\theta }_{sure}$), ($x_f$,$y_f$). Write the corresponding values at (m, n) of the matrix according to the number of occurrences of the features to form the feature matrix, where: i denotes the number of harmonics (i = 3, 5, 7, 9) and (m, n) denotes the position of the mth row and nth column in the matrix.

The feature matrix is converted to RGB images according to the size of the values, and according to the definition of the matrix heat map, the larger the value, the darker the color. The opposite is lighter. A sample set consisting of matrix heat maps with multiple features is constructed. The matrix heat map has a significant improvement in feature resolution for similar loadings after adding color features. Taking the coffee machine and the kettle in the similarity load of Figure 2 as an example, binarizing the matrix heat map shows that the feature locations in the matrix are largely similar and not very distinguishable. However, as shown in the Figure 4, when comparing this with the matrix heat map, the feature differentiation between the two loads increases after adding the color features.

3. Identification Network Construction

CNN is widely used in the field of image classification. The edges and contours of the images are extracted by convolutional computation, which reduces the complexity of the model by reducing a large number of parameters into a small number of parameters. When the image is rotated, inverted, or deformed, the image can be recognized [19]. However, the problem of model degradation and gradient explosion or disappearance occurs as the depth of the network increases. Based on this, SE-ResNet adds a residual network with squeeze and excitation (SE) to the CNN to improve the accuracy of the model effectively by showing the interdependence between modeling channels and adaptively calibrating the features of the channel directions accordingly.

3.1. ResNet Network Structure

ResNet adds shortcut connection branches outside the convolutional layer to perform a simple constant mapping and to form the basic residual learning units, and then solves the problem of network degradation that is difficult to train when the CNN network is too deep by sequentially stacking the residual learning units, making it possible to train deep convolutional neural networks.

Figure 5 shows the standard network structure and the ResNet structure. The basic residual learning unit neither introduces new parameters nor increases the computational complexity.

The principle of ResNet is:

$${\mathrm{x}}_l=h(x_l)+F(x_l,W_L)$$

(7)

$${\mathrm{x}}_{l+1}=f(y_1)$$

(8)

where: $h()$ is the direct mapping and $f()$ is the activation function.

The residual block can be expressed as:

$$x_{l+1}=x_l+F(x_l,W_1).$$

(9)

The relationship between the deeper L layer and the l layer is:

$$x_L=x_l+{\displaystyle \sum _{i=1}^{L-1}F(x_i,W_i)}.$$

(10)

According to the chain rule for derivatives used in backward propagation, the gradient of the loss function $\epsilon$ with respect to $x_l$ is

$$\frac{\partial \epsilon }{\partial x_l}=\frac{\partial \epsilon }{\partial x_L}\frac{\partial x_L}{\partial x_l}=\frac{\partial \epsilon }{\partial x_L}(1+\frac{\partial }{\partial x_l}{\displaystyle \sum _{i=1}^{L-1}F(x_i,W_i)})=\frac{\partial \epsilon }{\partial x_L}+\frac{\partial \epsilon }{\partial x_L}\frac{\partial }{\partial x_l}{\displaystyle \sum _{i=1}^{L-1}F(x_i,W_i)}.$$

(11)

Throughout the training process, $\frac{\partial }{\partial x_l}{\displaystyle \sum _{i=1}^{L-1}F(x_i,W_i)}$ cannot be −1 all the time, so there is no problem of gradient disappearance in the residual network, and $\frac{\partial \epsilon }{\partial x_l}$ means that the gradient of L layer can be directly passed to any l layer that is shallower than it.

After converting numerical features to image features, the number of samples in the dataset decreases and does not require too much network depth, so ResNet18 is used in this paper, and the structure diagram is shown in Figure 6.

The features extracted by CNN through convolutional layer stacking are high-dimensional features. Some of them are lost, while the residual block of ResNet is to skip the extraction of features by some convolutional layers and fuse the features before n layers, with the convolutional features after n layers, so that both high-dimensional features and low-dimensional features are retained, and the network performance is improved. Global average pooling is also used to replace the fully connected layer (fc) in the classical CNN. The global average pooling strengthens the correspondence between feature maps and categories on the fully connected layer, which is more suitable for convolutional structures. In addition, there are no parameters to be optimized in the global average pooling, which avoids overfitting. Furthermore, global average pooling aggregates spatial information and is more robust with regard to spatial transformation of the input.

3.2. SE-ResNet

SE-ResNet focuses on the interdependencies between the convolved feature channels using 1D convolution. The SE block is completed by a squeeze operation, which summarizes the overall information of each feature mapping, and an excitation operation, which scales the importance of each feature mapping. In this way, the squeeze operation extracts only the significant information of each channel, and the excitation operation computes the dependencies between channels using a fully connected layer with a nonlinear function.

The difference between the SE-ResNet and ResNet network structures is shown in Figure 7.

3.3. Identification Process

The recognition process in this paper is shown in Figure 8, which contains data acquisition, data preprocessing, feature construction, and training and recognition using SE-ResNet.

In the identification process, firstly, the data acquisition equipment collects voltage and the current data of each type of load from the home, normalizes the voltage and current data, and also extracts harmonic features from the current waveform by FFT, including the amplitude and phase. The three types of feature data are processed and written into a matrix to form a feature matrix, which is converted into a matrix heat map. The matrix heat map of each type of load is made into a sample set and divided into a training set and a test set, which is put into SE-ResNet for training and testing.

4. Experimental Results of REDD and PLAID

In the actual arithmetic example, the deep learning framework of tensorflow 2.0 is used to construct a sample set consisting of a matrix heat map from the PLAID dataset as well as the REDD dataset in the way described in this paper. After the sample set is produced, 80% of each class is selected as the training set and 20% as the test set. The confusion matrix is used to evaluate the recognition accuracy of the method proposed in the article. Each column of the confusion matrix represents the predicted category, and the total number of each column represents the number of data predicted to be in that category. Each row represents the true attribution category of the data, and the total number of each row represents the number of data instances in that category. The value of each column represents the number of real data predicted as that category. From the confusion matrix, TP, FP, FN, and TN can be calculated, where TP means that the predicted result is true and the actual result is also true, FP means that the predicted result is true and the actual result is false, FN means that the predicted result is false and the actual result is true, and TN means that the predicted result is false and the actual result is also false. The precision rate (P), recall rate (R), summed average of precision and recall (F1), and accuracy rate (A) can be calculated from TP, FP, FN, and TN. P and R can measure the correctness of a positive sample, and the higher the index result, the better. The formulas for calculating each indicator are as follows:

$$\mathrm{P}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(12)

$$\mathrm{R}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(13)

$$\mathrm{F}1=\frac{2\times \mathrm{P}\times \mathrm{R}}{\mathrm{P}+\mathrm{R}}$$

(14)

$$\mathrm{A}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FN}+\mathrm{FP}+\mathrm{TN}}.$$

(15)

4.1. REDD

The nine loads in the REDD dataset are represented using the numbers shown in

Table 1. Symbols representing the load of the REDD dataset.

Load	Symbol	Load	Symbol
fridge 1-1	0	microwave 1-2	7
fridge 1-2	1	microwave 1-3	8
light 1-1	2	CE appliance	9
light 1-2	3	sockets 1-1	10
light 2	4	sockets 1-2	11
light 3	5	washer dryer	12
microwave 1-1	6	electric furnace	13

below: 1-1, 1-2 denote different states of the same device and electric light 2 and electric light 3 denote two different light loads.

The load identification results for the REDD dataset are shown in Figure 9 and Figure 10: green represents the number and percentage of correctly identified samples and yellow represents the number and percentage of incorrectly identified samples.

From the results, it can be seen that the correct identification rate of each type of load is above 91.7%, among which the light 1 and light 2 reach 100%, with an average correct rate of 96.4%. Similarly, for the load socket, washer dryer, and electric furnace, the correct identification rate is more than 94.4%, and the average correct rate is 94.9%. The correct recognition rate of multi-state devices such as the fridge, light, microwave, and socket is above 92.3%, and the average correct rate reaches 96.5%.

Table 2. Evaluation indexes of confusion matrix.

	P	R	F1	A
0–1	0.994	0.987	0.990	0.992
2–3	1.000	0.943	0.971	0.995
4	0.963	1.000	0.981	0.997
5	1.000	1.000	1.000	1.000
6–8	1.000	0.981	0.990	0.997
9	0.917	0.917	0.917	0.995
10–11	0.929	0.945	0.937	0.982
12	0.957	0.957	0.957	0.995
13	0.850	0.944	0.895	0.990
average	0.956	0.964	0.969	0.993

is evaluated by P, R, F1, and A. As can be seen from Table 2, most of the P, R, F1, and A of various types of loads were kept at 0.85 and above, and light 3 reached 1, with the average values of 0.95, 0.96, 0.96, and 0.99, respectively.

4.2. PLAID

The 15 loads in PLAID were identified. The load representation is shown in

Table 3. Symbols representing the load of the PLAID dataset.

Load	Symbol	Load	Symbol
air conditioning	0	laptop	11
blender	1	microwave 1-1	12
coffee maker	2	microwave 1-2	13
compact fluorescent lamp	3	microwave 1-3	14
fan	4	soldering iron 1-1	15
fridge	5	soldering iron 1-2	16
hair curler	6	vacuum 1-1	17
hair dryer	7	vacuum 1-2	18
heater	8	vacuum 1-3	19
incandescent light bulb	9	washing machine	20
kettle	10

, and the multi-state loads are represented in the same way as in the previous example.

The results of load identification are shown in Figure 11 and Figure 12. The correct identification rate of each type of load is above 90.8%, and the correct identification rate of the hair curler, laptop and vacuum is 100%, and the average correct rate is 96.24%. Among these, although the kettle is still affected by the similarity load resulting in a slightly lower recognition correct rate than the other loads, the overall correct recognition rate reaches 92%, which is higher than the correct rate generated using the original V-I trajectory.

Meanwhile, the correct recognition rate of the soldering iron is affected by the number of samples and lower than the other loads. However, in terms of the number of incorrectly identified samples, the recognition of the soldering iron is not worse than that of other loads. The multi-state loads such as the microwave, soldering iron, and vacuum have a correct recognition rate of 86.7% or more for each state, with a 100% correct recognition rate for each type of state for vacuum, and an average correct rate of 97.79% for each type of state for multi-state loads. The results of the identification of the four similar loads described previously are shown in Figure 11. As can be seen from Figure 11, the average correct identification rate is 96%.

Table 4. Evaluation indexes of confusion matrix.

	P	R	F1	A
0	0.990	0.929	0.959	0.996
1	0.974	0.974	0.974	0.997
2	0.925	0.974	0.949	0.993
3	1.000	0.925	0.961	0.996
4	0.935	0.923	0.929	0.993
5	1.000	0.983	0.991	0.999
6	1.000	1.000	1.000	1.000
7	0.925	0.940	0.932	0.994
8	0.953	0.934	0.943	0.997
9	0.820	0.984	0.895	0.990
10	0.900	0.920	0.910	0.990
11	1.000	1.000	1.000	1.000
12–14	0.986	0.968	0.977	0.993
15–16	0.979	0.933	0.955	0.990
17–19	1.000	1.000	1.000	1.000
20	0.955	0.969	0.962	0.997

evaluates by P, R, F1, and A. Most of the index values are kept at 0.82 and above, among which the hair curler, laptop, and vacuum reach 1.00, and the average values are 0.96, 0.96, 0.96, and 0.99, respectively.

4.3. Comparative Analysis of Results

Using the same dataset, sample sets made of different features are selected and compared using different recognition models to evaluate the effectiveness of different methods in terms of recognition correctness or F1 values.

As shown in

Table 5. Comparison of results of different methods based on REDD data sets.

Characteristic	REDD
	Algorithm	F1
V-I trajectories	SoftMax	0.880
	SE-ResNet	0.855
V-I binary graph	CNN	0.826
	SE-ResNet	0.83
(V-I + harmonics) binary diagram	ResNet	0.872
	SE-ResNet	0.895
(V-I + power) vector characteristic	ResNet	0.879
	SE-ResNet	0.964

, the literature that identifies the REDD dataset quantifies the V-I trajectory and uses the SoftMax classifier for identification with an F1 value of 0.88 and quantifies only a single feature V-I trajectory. The similarity load will be difficult to identify with similar quantified values. CNN or SE-ResNet networks are used to identify the V-I trajectory binary map, the binary map after fusion of V-I trajectory and harmonics, and matrix heat map. The feature class, fusion method, and identification network are changed to form a control group, and the average F1 value of the proposed method in this paper is 0.99, which is a better result compared with other control groups.

As shown in

Table 6. Comparison of results of different methods based on PLAID data sets.

Characteristic	PLAID
	Algorithm	Accuracy Rate
V-I trajectories	Random forest [15]	77%
	Random forest [7]	81.75%
Amplitude, phase angle	Multi-layer perceptron [16]	73%
V-I binary graph	CNN [17]	75.3%
	ResNet	76.1%
(V-I + harmonics) binary diagram	CNN	80.25%
	ResNet	84.37%
(V-I + power) vector characteristic	BP neural network [10]	83.7%
	SE-ResNet	85.3%
(V-I + power) vector characteristic	CNN	89.35%
	SE-ResNet	96.24%
True color feature map	CNN [13]	82.87%

, in the literature on identifying PLAID datasets, Gao J et al. [12] and De Baets Leen et al. [16] used random forests for the identification of V-I302 trajectories; however, with a single feature, random forests have a long training time and high data requirements, otherwise they are prone to overfitting. Shouxiang Wang et al. proposed a feature fusion of V-I trajectories with power, using different networks for feature extraction and, after feature input to the network, the output vectors of the hidden layers of the two neural networks were combined together to form a composite vector feature, which was recognized using BP neural networks with a correct rate of 83.7% [15]. Xiang Y et al. uses power and current magnitudes incorporated into V-I trajectories to form true color feature images for recognition by CNN, but the true color image of each type of load is a single color, while the color is determined by the power and current magnitudes, which can lead to the difficulty in distinguishing the true color image because the power or current values of individual loads are too large, making most of the loads concentrated in the same color interval, such as microwave ovens and washing machines. In this paper, the data intervals are evenly distributed in a 32 × 32 matrix, making the loads in the same range of value intervals show more significant color variability [18]. Z Zheng et al. used amplitude and phase angle composition features and a multilayer perceptron for recognition [14]. De Baets L et al. used CNN for recognition with a correct rate of 77.60% [23]. Meanwhile, for matrix heat map recognition using CNN and SE-ResNet network respectively, the correct rate was 89.35% and 96.24%, which proves that the SE-ResNet network is better than CNN under the same conditions.

After the above comparison, it was found that the studies of Wang Shouxiang et al. [15] and Xiang Y et al. [18] were based on the same dataset, PLAID, and the overall recognition results were better, but some of the loads were not recognized with high accuracy, such as: air condition, fan, fridge, heater, and washing machine. As shown in Figure 13, the accuracy of this paper’s method for identifying such loads is higher than the above literature. For the PLAID dataset, the current study generally selects 11 load types to be carried out, and this paper also identifies four other types of other loads: blender, coffee maker, hair curler, and soldering iron, and the identification accuracy is higher than 91%.

Instead of using a dedicated one-to-one model for each device, the method proposed in the article is implemented on the basis of a single multiclass classifier, where one model is used for multiple devices with higher computational efficiency. SE-ResNet adds residual connectivity and an SE block completed by squeeze and excitation operations to the CNN, and although it still needs to be retrained when new devices are added, the overall training time of the model is greatly reduced. The proposed method is trained on the REDD dataset with six houses and the PLAID dataset with 56 houses, which indicates that the method is scalable and generalizable, and the addition of more house data is to be continued to be tested on new datasets. Follow-up work should investigate how the proposed method in this paper can reduce the training volume and improve the computational efficiency when identifying new devices.

5. Conclusions

In this paper, a non-intrusive load monitoring method based on multi-feature fusion with image recognition technology is proposed. When performing feature fusion, the V-I trajectory, amplitude and phase angle of odd harmonics, and fundamental wave amplitude are fused into a feature matrix. The matrix heat map is constructed by color according to the magnitude of the values, which effectively solves the feature masking problem arising from feature fusion. In addition, the single feature similar load is well distinguished from the multi-state load. In terms of image recognition, the SE-ResNet network recognition model is built to avoid the gradient descent problem of CNN, and also to realize the use of one classifier model to recognize multiple loads, which shortens the training time and improves the computational efficiency. Two datasets, the PLAID dataset and the REDD dataset, were used for validation, and the results prove that the proposed method has better accuracy compared with other methods.

In the existing research, the features used in the methods with high recognition rates are mostly transient features, which have low computational efficiency and high data collection and storage costs. In this paper, a one-to-many algorithm model is used for recognition, which improves the computational efficiency to some extent, but adding new loadings requires retraining. In future research, the balance degree between accuracy and computational efficiency should be considered. For example, migration learning might be used to reduce the training time. Furthermore, the correlation could be fused between loads with steady-state features to improve the accuracy of recognition, using steady-state features.