Identification of TV Channel Watching from Smart Meter Data Using Energy Disaggregation

Smart meters are used to measure the energy consumption of households. Specifically, within the energy consumption task smart meter have been used for load forecasting, reduction of consumer bills as well as reduction of grid distortions. Except energy consumption smart meters can be used to disaggregate energy consumption on device level. In this paper we investigate the potential of identifying the multimedia content played by a TV or monitor device using the central house's smart meter measuring the aggregated energy consumption from all working appliances of the household. The proposed architecture is based on elastic matching of aggregated energy signal frames with 20 reference TV channel signals. Different elastic matching algorithms were used with the best achieved video content identification accuracy being 93.6% using the MVM algorithm.


Identification of TV Channel Watching from Smart Meter Data Using Energy Disaggregation Pascal A. Schirmer, Iosif Mporas and Akbar Sheikh-Akbari
Abstract-Smart meters are used to measure the energy consumption of households. Specifically, within the energy consumption task smart meter have been used for load forecasting, reduction of consumer bills as well as reduction of grid distortions. Except energy consumption smart meters can be used to disaggregate energy consumption on device level. In this paper we investigate the potential of identifying the multimedia content played by a TV or monitor device using the central house's smart meter measuring the aggregated energy consumption from all working appliances of the household. The proposed architecture is based on elastic matching of aggregated energy signal frames with 20 reference TV channel signals. Different elastic matching algorithms were used with the best achieved video content identification accuracy being 93.6% using the MVM algorithm.
Index Terms-Video content identification, smart meters, load disaggregation.

I. INTRODUCTION
O VER the last decades there has been an extensive use of smart meters in residential buildings, with 60% of the houses in USA [1] and 50% of the houses in Europe [2] having smart meters installed. Smart meters provide residents/consumers with information about their daily energy consumption and based on this information residents can manage or reschedule the usage of their devices to reduce electricity bills, e.g. by using some appliances like washing machines at night time during which electricity costs are usually lower [3].
Apart from measuring household's energy consumption smart meters can also be used to provide more detailed information, as in the case of energy disaggregation where from one smart meter installed at the main inlet of the household, the usage and energy consumption on device level is extracted using Non-Intrusive Load Monitoring (NILM) methods [4]. In NILM, the aggregated signal is split into device signals using source separation methods [5]- [7] or is processed by machine learning based device models to detect the existence of devices within time sliding frames [8]- [13]. Specifically, variants of HMMs [14], CNNs [11] and LSTM [15] architectures have been utilized in order to achieve accurate disaggregation. Furthermore, also elastic matching algorithms have been proven to work successfully [16], [17]. By breaking down the energy consumption information on device level, the consumers can be informed about the distribution of energy consumption P. A. Schirmer and I. Mporas are with the school of Engineering and Computer Science, University of Hertfordshire, AL10 9AB Hatfield, UK (email: {p.schirmer/i.mporas}@herts.ac.uk).
Akbar Sheikh-Akbari is with the School of Built Environment, En-gineering and Computing, Leeds Beckett University (e-mail: a.sheikh-ak-bari@leedsbeckett.ac.uk). across home appliances and manage them, or rearrange the schedule of their operation in a more efficient way [18], [19].
Furthermore, smart meters have been utilized for other energy related tasks, e.g. load forecasting, for reduction of consumer bills [20] or reduction of grid distortions [21]. Moreover, additional information, e.g. weather condition [22] or socio-economic information [23], has been used and combined with the measurements of the smart meters. Based on NILM algorithms, smart meters can be used as non-intrusive sensors, unlike cameras and microphones, that further to energy consumption can also monitor consumers behaviour, device usage preferences and daily routine habits [24]. Therefore, in more exotic scenarios, smart meters can be used to detect or even to predict abnormal behaviour of residents, especially in the case of elders or mental disorders [25].
However, the usage of smart meters for various tasks in consumers households raises the question of home data security and privacy [26], [27]. Specifically, smart meters providing high frequency energy consumption data have raised security issues even before their major implementation in consumer households [28], [29]. Studies have shown that even nonintrusive smart-meters enable accurate tracking of a person's location within the house, e.g. by detecting changes of lighting or other frequently used devices, or enable estimation of working routines and number of people living in a household [30], [31]. As these information are very personal and could even be related to security, e.g. working routines could be observed and used by criminals to plan burglaries, studies on encryption of energy data have been presented in [32]- [34].
Multimedia content identification from energy consumption has been presented in [35] where the study was limited to measurements of correlations of energy predictors in isolated monitors (intrusive load monitoring) on specific video chunks. The application of NILM techniques, for identifying the use of appliances during a time sliding frame in a household, has been reported in literature utilizing high [36], [37] and low frequency features [38], [39]. However, to the best of the authors knowledge, the application of NILM in recognising which TV channel is being viewed, by just observing the aggregated energy consumption signal, has not been investigated. In this paper, we investigate the potential of using smart meter's measurement data to identify the viewed TV channels. Specifically, given that the smart meter is measuring the aggregated energy consumption from all operating appliances of a household, we investigate the possibility of identifying the TV channel a resident is watching from the aggregated energy consumption signal.
The remainder of this paper is organized as follows: In Section II the proposed architecture for detection of TV channel watching from the central household's smart meter is presented. In Sections III and IV the experimental setup and the evaluation results are presented, respectively. Finally, the paper is concluded in Section V.

II. TV CHANNEL WATCHING IDENTIFICATION FROM SMART METER DATA ARCHITECTURE
The presented architecture deems to investigate the potential of identifying the TV channel watching preferences of residents using the aggregated energy consumption signal acquired outside the house from a smart meter installed at the main inlet of the household. The conceptual diagram of the architecture for identification of watched TV channels using explicitly smart meters energy data is illustrated in Fig. 1. Specifically, the architecture in Fig. 1 underlies the following five assumptions: 1) The number of TV channels is of medium size (∼ 20 different channels).
2) The noise of the other devices is simulated through multiple different scenarios, with different noise levels each. 3) There is no time lag between the recordings in the household and the server base station. 4) In the considered household a maximum of one TV device is turned on at the same time. 5) The TV operates in real-time watching mode not in video on demand mode. As can be seen in Fig. 1 a smart meter is measuring the aggregated energy consumption, P agg (t), of a household. The aggregated signal is the sum of the energy consumption of all the devices of the house and in the present setup we consider the TV device (or a monitor) as the target device with energy consumption p(t) and all other home appliances having energy consumption N (t), i.e.
where N is the number of all appliances of the household, including the TV device, e.g. fridge, washing machine, etc., operating in the considered household.
Subsequently, the aggregated signal, P agg (t), is frame blocked in frames of constant length equal to W samples and from every energy frame, P w agg (t) ∈ R w : P agg (t − W : t), the DC offset is removed, resulting to P w res . The reason for the DC offset removal is the fact that most of the common home appliances like fridges, refrigerators, boilers, electric heating bodies, electric ovens etc., consume energy at the order of 200-2000 Watts while the average energy consumption of a TV device or monitor is at the order of 25-250 Watts. Therefore, the main part (DC part) of the energy consumption signal within each frame will come from devices with high energy consumption and by removing it in the remaining residual signal, P w res (t) ∈ R W , the contour shape characteristics of the energy signal of devices with lower energy consumption like the TV device will be shown more clearly.
Except the household aggregated energy consumption measurements, we consider a server base station where the broadband signals from M TV channels are received, assuming that M are all available TV channels. Each of the received signals is played by M reference TV devices of the same brand and model and the corresponding energy consumption signals, r m (t), with 1 ≤ m ≤ M , are measured by smart meters. It is worth mentioning that the TV devices used at the server base station are not the same with the TV device of the household. Next, each of the M reference signals, r m (t), is frame blocked in frames of constant length equal to W samples, i.e. r w m (t) ∈ R W .
Assuming no significant time-lag at the reception of the TV broadband signal between the house and the server base station template matching (elastic matching) is performed between the unknown residual signal, P w res (t), and each of the M reference signals, r m (t), and the reference TV channel energy consumption signal with the lowest distance from the residual households signal is considered as the detected TV channel Except the household aggregated energy consumption measurements, we consider a server base station where the broadband signals from TV channels are received. Assuming that are all available TV channels, which are played by reference TV devices of the same brand and model and their corresponding energy consumption signals can be written as: ( ), with 1 ≤ ≤ , are measured by smart meters. It is worth mentioning that the TV devices used at the server base station are not the same with the TV device of the household. Next, each of the reference signals, ( ), is frame blocked in frames of constant length equal to samples, i.e. (t) ∈ ℝ .
Assuming no significant time-lag at the reception of the TV broadband signal between the house and the server base station template matching (elastic matching) is performed between the unknown residual signal, (t), and each of the reference signals, ( ), and the reference TV channel energy consumption signal with the lowest distance from the residual household's signal is considered as the detected TV channel ℎ watched at time frame , i.e.
where () is a template (elastic) matching algorithm and ℎ( ) ∈ ℕ: [1, M] is the detected TV channel watched at time frame .
In order to provide more distinctive information, the above described architecture can be expanded with the elastic matching algorithm () not only using the aggregated active power samples, but a set of features (t) ∈ ℝ × and ′ (t) ∈ ℝ × . The features are computed from the raw current and voltage samples, ( ) and ( ), providing set of most common electrical features, e.g. reactive power or harmonics, and include the active power samples. Thus Eq. 3 can be rewritten as: In the presented architecture the case of not watching TV can be considered either by applying threshold values to the elastic matching algorithms of the detected TV channels or by considering not watching TV as an additional class, i.e. the ( + 1)-th one. Also, the architecture can be expanded to the case of more than two TV devices to play different channels simultaneously by considering one of them as belonging to the other devices.
the 'other devices' or by selecting all elastic matching scores below a threshold as detected devices

III. EXPERIMENTAL SETUP
The video classification architecture presented in Section II was evaluated using the dataset, energy features, classification models and experimental protocols presented below.

A. Dataset
As there is no benchmark dataset with energy consumption measurements from TV or monitor devices we have recorded a dataset for video classification using the publicly available video footage from the THUMOS challenge 2014 [36]. In detail, we selected the first 26 videos of THUMOS 2014 challenge, excluding the videos 11, 14, 20, 23, 24 and 25 which have duration less than 1 minute, resulting in a set of 20 videos. Energy consumption data measurements were taken from two different monitors, namely an Acer P235H, which was used as the test screen at the house as shown in Fig. 1 and an Iiyama Prolite B2483HS, which was used as reference pattern at the servers' station. The technical characteristics of the two monitors are tabulated in Table I.   TABLE I. TECHNICAL CHARACTERISTICS OF THE MONITORS USED IN ENERGY DATA COLLECTION.

Acer P235H
Iiyama B2483HS Ch watched at time frame t, i.e.
where f () is a template (elastic) matching algorithm and In order to provide more distinctive information, the above described architecture can be expanded with the elastic matching algorithm f () not only considering the aggregated active power samples, but a set of F features X w res (t) ∈ R F xW and r m (t) ∈ R F xW . The F features are computed from the raw current and voltage samples, i agg (t) and v agg (t), providing set of most common electrical features, e.g. reactive power or harmonics, and include the active power samples. Thus Eq. 3 can be rewritten as: In the presented architecture the case of not watching TV can be considered either by applying threshold values to the elastic matching algorithms of the detected TV channels or by considering not watching TV as an additional class, i.e. the (M +1) th one. Also, the architecture can be expanded to the case of more than two TV devices to play different channels simultaneously by considering one of them as belonging to the other devices or by selecting all elastic matching scores below a threshold as detected devices.

III. EXPERIMENTAL SETUP
The video classification architecture presented in Section II was evaluated using the dataset, energy features, classification models and experimental protocols presented below.

A. Evaluation Data
As there is no benchmark dataset with energy consumption measurements from TV or monitor devices we recorded a dataset for video classification using the publicly available video footage from the THUMOS challenge 2014 [40]. In detail, we selected the first 26 videos from the background data, excluding videos 11, 14, 20, 23, 24 and 25 as their duration was less than 1 min, resulting in a set of 20 videos. Energy consumption data measurements were taken from two different monitors, namely an Acer P235H which was used as the test screen at the house as shown in Fig. 1 and an Iiyama Prolite B2483HS which was used as reference pattern at the servers station. The technical characteristics of the two monitors are tabulated in Table I. For measuring the electrical energy consumption of the monitors, a hardware prototype smart meter developed by MEAZON (https://meazon.com/) was used for the first 1 minute of each video. The smart meter is a energy circuitlevel meter, measuring current, voltage, line frequency, active and reactive power and energy as well as harmonics and crest factor. It was designed for monitoring loads in an electrical board in commercial or industrial buildings or homes with an internal control capability up to 16 Amperes that can be extended further by driving an external relay. The USB port and the ARM Cortex M3 CPU run at 32 MHz Clock Speed with 512 Kbyte of In-System Programmable Flash and 32 Kbytes of Memory. All the above parameters, e.g. active power or reactive power, were outputted once per electrical cycle by the smart meter, thus at a rate of 50 times per second. Specifically, the internal sampling rate for calculating each feature is 8000 Hz, thus for each output sample 160 samples have been used for the calculation. The prototype of the used smart meter is shown in Fig. 2.
As regards the UK-DALE database [41], it was used to generate the energy consumption signal of N -1 other devices from a real house. In particular, UK-DALE was chosen among other online available databases, e.g. REDD [42], ECO [43] or AMPds [44], since it provides high frequency measurements (16 kHz) of households combined with a large number of appliances operating in parallel. In specific, we used one hour of the first house of UK-DALE (maximum of 52 appliances) from the 2 nd of December 2014 between 4pm-5pm due to the presence of 1 up to 26 electrical appliances within this time window, with none of them being a TV device. In order to have the same sampling rate as measured by the smart meter the data were down-sampled to 50 Hz.

B. Feature Extraction and Feature Ranking
The raw samples of the aggregated current and voltage i agg and v agg was frame blocked in frames of 160 samples. For every frame a feature vector was calculated consisting of 4 statistical features (peak voltage (V ), rms current/voltage (iFRMS, vFRMS), crest factor of current (cF)) and 15 electrical features (current (I), voltage (V), active power (P), frequency (f), reactive power (Q), apparent power (S), load angle (ϕ), total harmonic distortion current/voltage (iTHD, vTHD), 3 rd /5 th /7 th harmonic current/voltage (iHD3/5/7, vHD3/5/7) resulting to feature vectors of dimensionality equal to F = 19. In order to calculate the statistical importance of the 19 features the ReliefF feature ranking algorithm [45] was used by averaging the ranking scores across the 20 measured video signals. The results are illustrated in Fig. 3. As can be seen in Fig. 3 most of the electrical features in general outperform the statistical features with the most dominant features being the frequency, apparent power, raw current as well as of set current and voltage harmonics. This is in line with previous publications reporting high importance of electrical features [46], [47]. Regarding current and voltage harmonics, they carry significant amount of the video playing energy signal's information, e.g. both iHD5 and iHD7 have high feature ranking scores. Active power P , reactive power Q and apparent power S carry similar information as they can be computed by the relation S = P 2 + Q 2 , thus Q has a relatively low feature ranking score as it can be computed by S and P . Furthermore, as it is not clear how discriminative electric measurements of monitors are when being used for TV channel classification, e.g. electrical measurements might be filtered through a large capacitor at the monitor input, the time domain signals of eight different features are illustrated for two different video sequences in Fig. 4. Further to feature ranking measurements, examples of the monitors energy consumption information carried by the active power, raw current and the 7 th current harmonic are illustrated in Fig. 5 -Fig. 7, respectively, including the aggregated signals before and after frame's DC offset removal.

C. Elastic Matching Algorithms
For the classification, four different well known and widely used elastic matching algorithms were employed, namely the Dynamic Time Warping (DTW), soft Dynamic Time Warping (sDTW), Multi Variance Matching (MVM), and Global Alignment Kernel (GAK) [48]- [50]. The mathematical descriptions of these four template matching algorithms are presented below.
Lets considering the aggregated power consumption signal P agg (t)∀t : t ∈ {1, · · · , T } as measured by a smart meter. For simplicity let the w th frame of P w res be denoted by P a , where P a = [p(i), p(i + 1), · · · , p(i + K)] be a sequence of length K where p(i) is the i th sample of P agg . Furthermore, let P b = [p(j), p(j + 1), · · · , p(j + L)] be a second sequence of length L where p(j) is the j th sample of P agg and K < L. Furthermore, let ∆(P a , P b ) = [δ(p k a , p l b )] i,j ∈ R KxL be an arbitrary cost matrix, where δ(·) is a distance metric e.g., Euclidean distance, Manhattan distance or Kullback-Leibler (KL) distance and A, ∆(P a , P b ) being the inner product of matrix A with the cost matrix ∆(P a , P b ), where A is an alignment matrix with A k,l giving the scores of A.
Based on the above the generalized min γ operator, with the smoothing parameter γ ≥ 0 can be written as in Eq. 4 and is referred to as soft dynamic time warping dtw γ .
where the original DTW score is recovered by setting γ = 0.
Extending the previous definition of sDTW the Global Alignment (GA) kernel is defined as the exponentiated softminimum of all alignments distances and can be written as in Eq. 6 where γ > 0 is the smoothing parameter of the kernel. Compared to DTW, k γ GA incorporates the whole spectrum of costs A, ∆(P a , P b ) and thus provides a richer representation than the absolute minimum of set A, as considered by DTW.
In contrast to DTW, sDTW and GAK, MVM tries not to find the optimal alignment between the two sequences P a and P b , but also considers the alignment of subsequences. Therefore, MVM tries to find a subsequence P a of length K such that P b best matches P a . To formally describe MVM the difference matrix r between the two sequences P a and P b and is defined as follows: Furthermore, r kl is treated as a directed graph with the following links [50]: r kl ↔ r rs with r − k = 1 and l + 1 ≤ l + K − L (8) Using Eq. 7 and Eq. 8 the least-value path in terms of the linkcost and pathcost can be written as described in [50]. The free parameters of each elastic matching algorithm were empirically optimized after grid search on a bootstrap training subset utilizing 50% of the evaluation data. The grid search results are shown in Table II and the best elastic matching classification accuracy corresponding to the optimal values of each elastic matching algorithm are shown in bold. As can be seen in Table II the optimal parameters of the elastic matching algorithms used are γ = 5 for sDT W , γ = 5 for GAK and v = 10 for M V M . The best classification accuracy on the bootstrap training data was achieved by M V M algorithm and was equal to 95.6% outperforming all other evaluated elastic matching algorithms.

D. Experimental Protocols
To evaluate the proposed architecture three different experimental protocols were utilized: (A) evaluation under a noiseless conditions, i.e. N (t) = 0, was carried out to determine whether or not different videos can be distinguished from their electrical energy signals recorded from the same monitors, thus in this protocol it will be P (t) = P (t); (B): evaluation with additional other devices was carried out, thus N (t) = N −1 i=1 n i (t), using the aggregated energy signal from the UK-DALE dataset and reference patterns from the same monitor, i.e. P (t) = P (t); (C): evaluation with additional other devices was carried out using different monitors, i.e. P (t) = P (t), using Acer P235H monitor at the target house and Iiyama B2483HS monitor at server station.
For the additional other devices 59 different sets of recordings were randomly selected from the 1 hour duration measurements of the UK-DALE dataset and added to the energy measurement of the respective video signal. It must be noted that 59 noise scenarios were chosen in order to avoid zero padding for the 60 th noise scenario, as the 1 hour UK-DALE datafile is slightly shorter than 60 minutes.

IV. EXPERIMENTAL RESULTS
The architecture presented in Section II for the identification of the TV channels watched using an outdoors smart meter was evaluated according to the experimental setup described in Section III. The performance of the three evaluated protocols was estimated in terms of accuracy (ACC) and in terms of F-score (F 1 ), i.e.

ACC =
T P + T N T P + T N + F P + F N (9) where T P are the true positives, T N are the true negatives, F P are the false positives and F N are the false negatives, respectively. For each of the three experimental protocols (A, B and C) 21 energy signals (from playing 20 videos and one experiment with no video played) were tested for 60 different noise scenarios and the averaged results are tabulated in Table  in terms of ACC and F 1 scores for the noiseless (A), noisy (B) and noisy using different monitors (C) experimental protocol. As can be seen in Table III all four elastic matching algorithms were able to identify the played videos with 100% accuracy when intrusive load monitoring was used (protocol A), thus under a noiseless scenario. When identification was performed using the aggregated signal (protocol B) and using the same monitor (Acer P235H), M V M outperformed all other elastic matching algorithms achieving accuracy of 94.7% and F 1 score 94.3%. In protocol C, elastic matching was performed on the signals from different monitors and M V M again achieved the highest performance among all evaluated algorithms (accuracy 93.8% and F 1 score 93.3%), which is in agreement with our previous study [16] where M V M was also found to perform well on the N ILM task.
In a further step the results of the best performing architecture (MVM) were evaluated using confusion matrices for the 21 signals and compared to the baseline system when utilizing DTW. First, when considering protocol A, there is no difference in terms of classification between DTW and MVM as all signals are perfectly classified for each of the 59 noise scenarios (Fig. 8a and Fig. 8d). Second, for protocol B, there is a significant drop in performance for DTW, which is mostly due to misclassification of TV signals with the case of not watching TV (Fig. 8b and Fig. 8e). In detail, there were 35 misclassifications when utilizing DTW, while there have been only 5 misclassifications when utilizing MVM. Third, in protocol C, a similar behaviour is observed as in protocol B with a high number of misclassifications and thus a relatively large performance decrease for DTW and only a small performance decrease for MVM ( Fig. 8c and Fig. 8f). In detail, there have been 81 misclassifications with the scenario of not watching TV when utilizing DTW as elastic matching algorithm and 20 misclassifications for MVM respectively. The results of protocol B and C are in line with the work presented in [51] where significantly better performances for elastic matching were reported in noisy scenarios when utilizing MVM. Furthermore, a similar behaviour was reported in our previous study where MVM has also been proven to enhance accuracy for the Energy disaggregation task [16].
V. CONCLUSION In this paper we investigated the potential of identifying the watched TV channel or multimedia content using a smart meter installed outside a house by comparing the households aggregated energy consumption signal with a closed set of reference signals acquired from the energy consumption of other monitor devices. The experimental results for a set of 20 possible TV channel options revealed that this is possible and the best identification performance, achieved by the M V M elastic matching algorithm, was equal to 93.8% in terms of accuracy and 93.3% in terms of F 1 score. ACKNOWLEDGMENT This work was supported by the UA Doctoral Training Alliance (https://www.unialliance.ac.uk/) for Energy in the United Kingdom. Furthermore, the authors thank Dr Stelios Koutroubinas and Meazon S.A. for providing them with a smart meter device.