NILM-Based Feedback for Demand Response: A Reproducible Binary State-Detection Algorithm Using Active Power

Abstract

Non-intrusive load monitoring (NILM) can provide actionable feedback for demand response (DR) when direct measurements of device states are unavailable. We propose a reproducible, engineering-oriented pipeline for detecting ON/OFF states of end-use load groups from an aggregated active power time series. The method uses robust hysteresis-based labeling with adaptive thresholds derived from the median and median absolute deviation, followed by compact feature engineering restricted to global active power (GAP). After removing collinear features (|r| > 0.98), permutation importance is used to retain informative predictors. Probabilistic binary classifiers (LGBM, Histogram-based Gradient Boosting, XGBoost, and CatBoost) are trained for each target load, and the decision threshold is optimized via F_β to balance missed events and false alarms. A post-processing stage stabilizes predictions by smoothing probabilities and suppressing spurious triggers. Model quality is assessed with both sample-wise metrics and event-based metrics that credit the correct detection of switching intervals within a time tolerance. Experiments on the open “Individual Household Electric Power Consumption” dataset (1-min resolution, 2007–2010) demonstrate that lightweight gradient boosting models, particularly LGBM, deliver reliable and interpretable state estimates suitable for practical DR integration and edge deployment.

1. Introduction

One of the ways to improve energy efficiency in power supply systems with low capital expenditure, provided that sufficient load flexibility is available, is demand response (DR) [1]. Demand response, taking into account the classification of electrical loads developed in the authors’ previous studies, enables the application of load-control algorithms depending on the load class; however, stable operation requires feedback on the state of the end-use device and on the load response to the demand-response control signal [2,3].

Such feedback can be provided by integrating the information streams of the SCADA system with the control system of the power supply system [4,5]. However, when this information is not available, recent machine learning and data-processing studies suggest that the state of an end-use device can be inferred using NILM models [6,7].

From a DR perspective, NILM is not only an energy disaggregation task but also a practical mechanism to infer device/group-level operating states when direct device telemetry is unavailable or costly. In such settings, the key requirement is often reliable ON/OFF state (event) detection from aggregated measurements, because actionable DR feedback depends on timely identification of switching events and stable state tracking under noise, multi-device overlap, and measurement constraints [8]. This motivates engineering-oriented pipelines that emphasize reproducibility, limited sensing requirements (e.g., GAP-only), and evaluation protocols that reflect event timing tolerance in addition to sample-wise accuracy.

Numerous studies address non-intrusive load monitoring (NILM) and related load monitoring methods. For example, [9] provides a review of state-of-the-art methods for online NILM in load monitoring applications based on aggregated data. Nevertheless, existing reviews do not sufficiently formalize the requirements for feature causality and for a reproducible protocol for feature engineering workflows.

Recent AI/ML research in NILM has increasingly explored deep sequence models (e.g., CNN/RNN/attention-based architectures), representation learning, and methods aimed at robustness, uncertainty awareness, and generalization under non-stationarity. A complementary line of work emphasizes “trustworthy” energy disaggregation—covering reliability, interpretability, privacy, and robustness considerations—highlighting that deployment constraints and real-world data drift remain major bottlenecks beyond pure predictive performance. In this respect, reference [10] provides a comprehensive review of challenges and perspectives toward trustworthy energy disaggregation and discusses why reproducible baselines and transparent decision logic remain valuable. At the broader smart-grid level, recent surveys summarize how AI/ML methods are used for grid monitoring, stability, and energy-management applications, reinforcing the need to balance model complexity with operational constraints in real deployments [11].

In line with these AI/ML trends, studies [12,13,14] explicitly report the high effectiveness of large language models and multi-task probabilistic network models. However, they also highlight extremely high computational complexity, which constrains implementation on devices with limited computational resources—an important requirement for target industrial and commercial deployments with resource-constrained edge computing [15]. Moreover, in the context of binary classification, the models discussed in those studies are likely to be unnecessarily complex [16].

Therefore, for DR feedback loops that require reproducible behavior and predictable runtime on resource-constrained edge/enterprise platforms, lightweight models paired with transparent feature pipelines remain practically attractive—especially when the goal is binary state/event detection rather than full-scale disaggregation.

Study [14] shows that NILM can effectively analyze energy consumption processes in industrial settings, yielding satisfactory results even with limited datasets. In turn, augmenting NILM with auxiliary methods for dimensionality reduction and clustering has achieved high accuracy (ACC = 95%) in identifying load types, which is required for effective demand response [17]. However, the considered studies do not provide an in-depth analysis of the results using event-based model quality metrics. Primarily, this is because the works do not address the binary classification problem; nevertheless, it should be noted that the direct use of accuracy (ACC) in NILM tasks is not always appropriate.

In [18,19], a reasonable boundary is drawn between the achieved gains in energy efficiency and the energy and computational costs of the developed algorithms. The results can be interpreted as motivation to offload computations associated with heavy machine learning models to remote servers. The classical machine learning models considered in this work (HGB, XGB, LGBM, and CAT) are not resource-intensive for online deployment on an enterprise server, although they may require substantial resources for training [20,21].

This work advances NILM research by proposing a reproducible, engineering-oriented algorithm for inferring end-use device states based on binary classification of an aggregated electricity consumption time series. Unlike most studies focused on pointwise metrics, we introduce and employ event-oriented quality assessment, which is more consistent with demand-response tasks and with monitoring the load reaction to control actions. We show that machine learning methods, with appropriate post-processing and threshold selection, provide stable and interpretable results suitable for practical integration into DR systems.

The study aims to detect the states of individual loads (ON/OFF) from an aggregated electricity consumption signal as a component of the feedback loop in demand-response systems. The aggregate time series is used as the source of features, and the modeling procedure includes data preparation, labeling, feature construction, and training of binary classifiers for each target load.

It should be noted that experimental validation in this work is performed using an open residential electricity consumption dataset. This dataset was selected due to its representativeness, its time resolution relevant to demand response, and the availability of labeled end-use load groups, enabling reproducible comparison of load state-detection methods.

2. Materials and Methods

2.1. Dataset Description

This study uses the open dataset “Individual Household Electric Power Consumption” [22]. The dataset is provided as a CSV file containing 2,075,259 measurements collected in a private house located in a suburb in France over the period 2007–2010. The sampling interval is 1 min. The overall fraction of missing values is 1.25%.

The raw data comprise four main time series synchronized in time: the household-wide electricity consumption series, and three electricity consumption series corresponding to individual end-use load groups.

The first end-use load group (Sub_metering_1) exhibits no clear periodicity and occasional low-magnitude consumption spikes. The Sub_metering_1 measurements were obtained on the feeder supplying selected kitchen appliances.

The second end-use load group (Sub_metering_2) is characterized by periodic switching and relatively small magnitude compared with the household total (peak power of about 2 kW versus a peak aggregated household demand of about 32.5 kW). The Sub_metering_2 time series represents loads located in the laundry room.

The third end-use load group (Sub_metering_3) demonstrates stable periodicity and a high amplitude relative to the household total. This subgroup corresponds to the boiler room and includes several large consumers, such as a water heater and an air-conditioning system.

Notably, the metering system does not cover all consumers. According to summary statistics, the coverage ratio is CR = 48.8%. In this study, this limitation is beneficial because it introduces an additional unmetered component that better approximates real-world conditions. The coverage ratio was calculated as:

$$CR = {\displaystyle \frac{W_{s_{m_1}}+W_{s_{m_2}}+W_{s_{m_3}}}{W_{\mathsf{\Sigma }}}}$$

(1)

where $W_{s_{m_n}}$—is the total energy recorded by the n-th sub-meter over the period considered (kWh), and $W_{\mathsf{\Sigma }}$—is the total household electricity consumption over the same period (kWh).

2.2. Algorithm Description

Solving the binary classification problem using machine learning (ML) methods in the context of equipment-state monitoring can be reduced to the following five stages:

• Data loading and basic transformations (Step 1);
• State labeling and dataset construction (Step 2–5);
• Model training (Step 6);
• Threshold selection and repeated validation (Step 7–8);
• Final visualization and export of results (Step 9).

The methods used at each step are described below, starting from Step 1. At the first stage, the data are loaded and the time series corresponding to total household electricity consumption and to candidate predictors are identified. Basic transformations include harmonizing the scale of the electricity consumption data: sampling interval of 1 min and energy units initially expressed as $\mathrm{W}\cdot \min$, which were converted to the conventional form $\mathrm{kWh}$ for subsequent calculations. During preprocessing, missing values in the time series are imputed using time interpolation combined with limited value propagation, which provides a more physically meaningful signal reconstruction than linear interpolation or mean imputation.

After preprocessing, Step 2 forms additional predictors and, in particular, generates state labels for electrical loads. State labeling is based on the hysteresis principle. The hysteresis labeling method converts a power time series into a binary state sequence using hysteresis with adaptive thresholds computed from the median and the median absolute deviation (MAD), with filtering of short-duration false triggers. Hysteresis labeling built on robust statistics represents a hybrid of physical and statistical approaches and, in essence, constitutes an adaptive time series segmentation using non-linear thresholds. It does not rely on a priori device characteristics; instead, it exploits properties of the signal itself.

Let $x\left(t\right)$ denote the target sub-metering power time series used to construct proxy ON/OFF labels. We compute robust location and scale estimates on the calibration window as:

$$\tilde{x}=\mathrm{median}\left\{x\left(t\right)\right\}, \mathrm{MAD}=\mathrm{median}\{|x(t)-\tilde{x}|\}+\mathsf{\epsilon },$$

(2)

where $\mathsf{\epsilon }={10}^{-6}$ prevents division-by-zero. The adaptive hysteresis thresholds are then defined as:

$$T_{\mathrm{on}}=\tilde{x}+k_{\mathrm{on}}\cdot \mathrm{MAD}, T_{\mathrm{off}}=\tilde{x}+k_{\mathrm{off}}\cdot \mathrm{MAD},$$

(3)

with $k_{\mathrm{on}}>k_{\mathrm{off}}$. The state is updated causally: the label switches from OFF to ON when $x\left(t\right)\ge T_{\mathrm{on}}$ and switches from ON to OFF when $x\left(t\right)<T_{\mathrm{off}}$.

The results of hysteresis labeling for a data segment are shown in Figure 1.

Figure 1. Original electricity consumption time series and hysteresis labeling results.

Figure 1 shows that low-amplitude signal components (standby electricity consumption) are not marked as an “ON” state of the load. At the same time, the “ON” label is assigned throughout the entire operating cycle of the load, regardless of whether electricity consumption is constant or varies over time.

To suppress short-duration false triggers, we apply two causal duration rules. First, an OFF to ON transition is accepted only if the condition $x\left(t\right)\ge T_{\mathrm{on}}$ holds for at least $m_{\mathrm{on}}$ consecutive samples (minutes), where $m_{\mathrm{on}}$ is target-specific. Second, after probability thresholding, we apply a morphological post-processing to the predicted binary mask: ON segments shorter than $m_{\mathrm{on}}^{\mathrm{pp}}$ are removed (set to OFF), and OFF gaps between two ON segments are filled if the OFF gap length is $\le m_{\mathrm{off}}^{\mathrm{pp}}$.

At Step 3, after labeling, the original dataset is enriched with a set of statistical features. At this stage, a set of features suitable for machine learning is extracted from the DateTime time stamps. The purpose of this step is to transform the raw data into a feature space in which the difference between the “ON” and “OFF” states becomes linearly or quasi-linearly separable.

In the proposed algorithm, feature construction is centered on a single aggregated parameter—global active power (GAP). The derived predictors are referred to as GAP features according to their origin. The measured active power P(t) reflects the total electricity demand of all devices. Temporal, amplitude-based, and derivative characteristics are extracted from P(t) to describe load dynamics. The algorithm constructs three groups of features:

• Instantaneous statistical characteristics: power values, their logarithms, normalized deviations from the median, and local variance. These features characterize the current state of the system.
• Temporal characteristics: moving averages, exponential smoothing, differences between adjacent samples, and the ratio of the current value to a windowed mean. These features capture the rate of change and the transition patterns between states.
• Context features: time of day, day of week, and public holidays. These variables represent the cyclicity of household processes, improving model generalization.

A key feature of this study is the deliberate reduction of the feature space dimensionality. Features are not constructed from the full set of physical parameters available in the raw dataset (voltage, current, etc.) but exclusively from active power. This design choice is motivated by the hypothesis that, for state-detection tasks (as opposed to identification of specific appliances), excessive feature dimensionality does not improve classification performance but increases the risk of overfitting. The resulting feature vectors are augmented with the binary label vector that represents the load state and was obtained at the previous step.

After obtaining the complete feature table, Step 4 performs the removal of collinear features, i.e., predictors that exhibit strong linear dependence on each other. Collinearity implies that a subset of features effectively duplicates information already contained in other predictors [23]. This increases model instability (small perturbations in collinear features can lead to substantial changes in feature weights) and may reduce interpretability, because the model distributes importance across correlated variables in an arbitrary manner. In the proposed algorithm, collinearity is mitigated by computing the correlation matrix for all features and removing redundant predictors with high pairwise correlation (|r| > 0.98). Sensitivity at threshold |r| will be tested further in Section 2.4. From each strongly correlated group, one feature is retained, eliminating duplicated information and reducing dimensionality without sacrificing predictive power.

Unlike the dimensionality reduction performed at the previous stage, Step 5 selects features based on their predictive value. Feature importance is computed using repeated training of a Histogram-based Gradient Boosting (HGB) model, followed by permutation of each feature and evaluation of the performance drop measured by average precision (AP). This further reduction in feature dimensionality decreases the probability of overfitting, which is particularly important under class imbalance (e.g., Sub_metering_2 contributes far less than Sub_metering_3).

All experiments use strictly chronological (time-ordered) splits with no shuffling. For a given training duration $D_{\mathrm{tr}}$ and validation duration $D_{\mathrm{val}}$, the training window is defined as $\left[t_0, t_0+D_{\mathrm{tr}}\right]$, and the validation window is the immediately following contiguous segment $\left(t_0+D_{\mathrm{tr}}, t_0+D_{\mathrm{tr}}+D_{\mathrm{val}}\right]$. Feature selection, model fitting, probability calibration, threshold optimization, and all hyperparameter choices are performed using training data only (and, when applicable, a calibration sub-window carved from the end of the training period). Reported validation metrics are computed exclusively on the subsequent validation window.

Unless explicitly stated otherwise, the primary benchmark results (Table 1, Table 2 and Table 3, ROC/PR curves) use $D_{\mathrm{tr}}=336$ days and $D_{\mathrm{val}}=21$ days. The varying validation horizons (3/5/7 days up to one year) and the longer validation horizons (30/60/90 days up to three years) are used only in the training horizon sensitivity study shown in Section 3.2; all these splits remain strictly chronological and anchored at the same starting time $t_0$.

Table 1. Differences between metrics.

Target	Model	Sample-Wise Metrics				Event-Based Metrics
		TP	FP	FN	F1	TP	FP	FN	F1
y_boiler_ac	lgbm	14,196	2088	1028	0.901104	254	81	64	0.777948
	xgb	14,276	2260	948	0.898992	254	87	64	0.770865
	hgb	14,284	2428	940	0.894539	253	99	65	0.755224
	cat	14,159	2214	1065	0.896224	253	100	65	0.754098
y_kitchen	cat	2045	1757	1357	0.56774	52	140	54	0.348993
	lgbm	2020	1638	1382	0.572238	49	149	57	0.322368
	hgb	2117	1987	1285	0.564082	44	123	62	0.322344
	xgb	2093	1833	1309	0.571234	48	156	58	0.309677
y_laundry	hgb	6743	15628	2665	0.424368	218	925	178	0.283301
	lgbm	5338	9355	4070	0.442969	201	828	195	0.282105
	xgb	5759	11136	3649	0.437897	212	902	184	0.280795
	cat	5787	11185	3621	0.438741	202	876	194	0.274084

Table 2. Differences between metrics in different threshold.

Target	|r| Threshold	Number of Features After Correlation Filter	Number of Features After PI Filter (HGB)	AP	F1event
y_kitchen	0.98	18	14	0.6010	0.2267
	0.95	16	12	0.5884	0.2014
	0.90	16	12	0.5870	0.2082
y_laundry	0.98	18	14	0.4408	0.2814
	0.95	16	12	0.4399	0.2598
	0.90	16	12	0.4232	0.2785
y_boiler_ac	0.98	18	14	0.9781	0.7685
	0.95	16	12	0.9765	0.7616
	0.90	16	12	0.9769	0.7683

Table 3. Sensitivity to the permutation importance procedure.

Target	PI Estimator	Number of Features After PI Filter	AP	F1event
Kitchen	HGB	14	0.6010	0.2267
	LGBM	14	0.6023	0.2319
Laundry	HGB	14	0.4408	0.2814
	LGBM	14	0.4415	0.2780
Boiler_ac	HGB	14	0.9781	0.7685
	LGBM	14	0.9779	0.7504

At Step 6, the resulting feature table after primary filtering is used to train the classifier. The key objective of this stage is to build a statistical model capable of predicting, from the feature representation of the current power-network state, whether the device in a given load group (Sub_metering_1–3) is on. The problem is formulated as a binary classification with probabilistic output. Specifically, each candidate model receives a feature vector ${\mathbf{x}}_{\mathbf{t}}\mathrm{ }=\mathrm{ }\left(x_1,\mathrm{ }{x}_2,\mathrm{ }\dots ,\mathrm{ }{x}_n\right)$ and produces an estimate of the probability that the device is on, $p_t=P\left(y_t=1\right|{\mathbf{x}}_{\mathbf{t}}).$

An example output at this step is shown in Figure 2, which presents the predicted probability of the “on” state for the target load within Sub_metering_1 obtained using only the selected GAP features.

Figure 2. Probability of the “on” state for Sub_metering_1.

At Step 7, a binarization threshold is selected in relative units (or percent). The binarization threshold, denoted $\tau$, is required to convert probabilities to binary outputs (ON/OFF):

$${\widehat{y}}_t=\{\begin{array}{c}1, p_t\ge \tau \\ 0, p_t<\tau \end{array}.$$

(4)

Threshold selection is performed using precision–recall analysis, as shown later in the metrics subsection. To determine a rational threshold, the $F_{\beta }$ score is computed. Clarity, precision, and recall are defined as:

$$Precision={\displaystyle \frac{TP}{TP+FP}}=P,$$

(5)

which quantifies the fraction of predicted positive instances that are truly positive, and

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\left(TP+FN\right)}}=R,$$

(6)

which quantifies the fraction of true positive instances that are detected by the algorithm. TP, FP, FN, and TN denote the standard confusion-matrix counts. Let P and R denote precision and recall, respectively. The general form of the $F_{\beta }$ score is

$$F_{\beta }=\left(1+{\beta }^2\right)\cdot {\displaystyle \frac{P\cdot R}{\left({\beta }^2\cdot P\right)+R}},$$

(7)

where $\beta$ controls the relative weight of recall versus precision. For $\beta >1$ recall is prioritized (missing events are penalized more strongly, at the cost of more false positives), whereas for $\beta <1$ precision is prioritized (a more conservative model that may miss true ON intervals).

To mitigate class imbalance (especially for the laundry group), all classifiers are trained using class-balanced sample weights computed on the training window. In addition, the decision threshold is optimized using a target-specific $F_{\beta }$; for the most imbalanced target (laundry) we use $\beta <1$ to penalize false positives more strongly under rare-event conditions

The $F_{\beta }$ score is a weighted harmonic mean of precision and recall; values closer to 1 indicate better performance. The threshold $\tau$ is selected to maximize $F_{\beta }$ for the considered $\beta$ values [24].

Next, Step 8 performs post-processing of the resulting binary masks for each load group. A binary mask $\widehat{y}\left(t\right)$ is a discrete indicator function of the load state obtained by thresholding the probabilistic model output $p_t$, where $\widehat{y}\left(t\right)=1$ corresponds to the ON state and $\widehat{y}\left(t\right)=0$ to the OFF state.

Post-processing suppresses spurious triggers caused by sudden probability spikes and eliminates short OFF gaps within the probability time series. Here, $\widehat{p}\left(t\right)$ is the calibrated ON probability, $\tau$ is the decision threshold, and W is the smoothing window length (min). This stage also includes temporal stabilization of the probabilistic output. Probability calibration is performed using isotonic regression fitted on a dedicated calibration sub-window carved from the end of the training period (chronological split; no shuffling), and the resulting calibrated probabilities are then smoothed using a moving average window W prior to thresholding and post-processing. Specifically, a moving average with window width W minutes is applied to the calibrated probabilities (in the baseline case, W = 5 min). The smoothed probability series $p_t^{\left(W\right)}$ reduces the impact of noisy spikes and isolated false triggers, which are particularly typical for weak and short-duration loads.

The impact of the smoothing window length W on the trade-off between false alarms and missed events is quantified in Section 2.4.2 (Table 4). Since smoothing is implemented as a causal moving average, the added response latency is bounded by $W-1$ min, which we report together with event-based FP/FN counts.

Table 4. Sensitivity of event-based performance to the post-processing smoothing window W.

Target	W	Latency	$\mathit{\beta }$	${\mathit{\tau }}^{\mathit{*}}$	AP	ROC-AUC	Event-Based Metrics
							F1	Precision	Recall	TP	FP	FN
Boiler AC	1	0	0.70	0.63	0.97	0.99	0.69	0.57	0.87	276	206	42
	3	2	0.70	0.58	0.97	0.99	0.71	0.67	0.76	243	120	75
	5	4	0.70	0.59	0.97	0.99	0.64	0.67	0.61	193	94	125
	7	6	0.70	0.60	0.97	0.99	0.42	0.58	0.33	105	77	213
	10	9	0.70	0.61	0.97	0.99	0.40	0.59	0.30	95	66	223
Kitchen	1	0	0.80	0.26	0.58	0.92	0.31	0.27	0.37	83	223	140
	3	2	0.80	0.28	0.58	0.92	0.33	0.34	0.32	71	139	152
	5	4	0.80	0.25	0.58	0.92	0.30	0.33	0.26	59	118	164
	7	6	0.80	0.26	0.58	0.92	0.25	0.31	0.22	48	108	175
	10	9	0.80	0.25	0.58	0.92	0.21	0.28	0.17	38	99	185
Laundry	1	0	0.50	0.39	0.43	0.70	0.26	0.22	0.33	129	467	267
	3	2	0.50	0.43	0.43	0.70	0.29	0.28	0.30	119	310	277
	5	4	0.50	0.41	0.43	0.70	0.28	0.29	0.28	110	272	286
	7	6	0.50	0.43	0.43	0.70	0.25	0.29	0.22	88	216	308
	10	9	0.50	0.43	0.43	0.70	0.22	0.29	0.18	72	180	324

Finally, Step 9 performs a strict evaluation of model quality (Figure 3). Two groups of metrics are computed:

Figure 3. Output of the HGB model for three load groups.

• Classical (sample-wise) metrics: precision (P), recall (R), $F_1$, accuracy (ACC), and integral probabilistic measures—area under the precision–recall curve (AP) and area under the receiver operating characteristic curve (ROC-AUC). These metrics assess the model’s ability to assign the correct class at each minute of observation.
• Event-based metrics: a true positive event is counted if the start or end of a predicted “on” interval falls within a tolerance of $\pm \Delta t$ relative to the true interval. Event-based precision (P), recall (R), and $F_1$ are computed, along with counts of missed events and false alarms. This evaluation aligns with operational requirements, where the correct detection of complete ON/OFF episodes is more important than minute-level labels.

In Figure 3, the dotted line indicates the threshold, and the horizontal lines below (green, yellow, red, and black) refer to the results as TP, FP, TN, and FN, respectively.

Overall, the proposed algorithm enables the automatic extraction of operating states of individual end-use devices from an aggregated electricity consumption signal using machine learning methods based exclusively on active power data. By combining hysteresis-based labeling with GAP feature engineering and probabilistic classification, followed by calibration and binarization-threshold optimization, high state-detection accuracy is achieved without direct metering of each load. As a result, the method provides a foundation for intelligent power supply system control in which load states can be assessed in near real time with a controllable balance between sensitivity and recognition reliability.

2.3. Evaluation Metrics

During model development, a set of dedicated metrics was used to demonstrate the correctness and adequacy of the binary classifier. In total, 11 metrics were employed, grouped into three categories: (1) sample-wise; (2) probabilistic; and (3) event-based.

Each metric serves a specific purpose. Sample-wise metrics, partially introduced in Section 2.2, quantify classification quality at each time step (each minute is treated as an individual sample). This group includes the standard metrics ACC, precision, recall, and $F_1$.

The probabilistic-metric group includes ROC-AUC (area under the receiver operating characteristic curve) and AP (average precision). These measures do not assess the final binary decisions directly; instead, they evaluate the quality of the model output as a probability function $p_t=P\left(y_t=1\right|x_t)$.

ROC-AUC is the area under the ROC curve, which describes the relationship between the true positive rate (TPR) and the false positive rate (FPR). The closer ROC-AUC is to 1, the better the model separates classes irrespective of the chosen threshold.

To construct the ROC curve, the threshold $\tau$ is varied iteratively from 0 to 1 and, at each iteration, the following quantities are computed:

$$TPR\left(\mathsf{\tau }\right)={\displaystyle \frac{TP\left(\mathsf{\tau }\right)}{TP\left(\mathsf{\tau }\right)+FN\left(\mathsf{\tau }\right)}}=\mathit{Recall}$$

(8)

$$FPR\left(\mathsf{\tau }\right)={\displaystyle \frac{FP\left(\mathsf{\tau }\right)}{FP\left(\mathsf{\tau }\right)+TN\left(\mathsf{\tau }\right)}}.$$

(9)

Thus, the ROC curve is the locus of points in the (FPR, TPR) plane obtained for all possible $\tau$. ROC-AUC is then computed as:

$$\mathrm{AUC}={\int }_0^{1}TPR\left(FPR\right)\hspace{0.17em}d\left(FPR\right).$$

(10)

Therefore, ROC-AUC (Figure 4) is invariant with respect to the selected threshold $\mathsf{\tau }$ and reflects the model’s ability to discriminate between device states.

Figure 4. ROC curves for different models. The dash–dot line corresponds to a random classifier.

The AP metric corresponds to the area under the precision–recall (PR) curve and focuses on the model’s performance in predicting the positive class. PR curves were mentioned earlier; here, we provide the computation method and visualization in Figure 5. In general form, AP is defined as the integral:

$$AP={\int }_0^{1}P\left(R\right)\hspace{0.17em}dR.$$

(11)

Figure 5. Precision–recall curves (AP) for different models.

Event-based metrics, unlike the previous two groups, evaluate model performance not at the level of individual time steps (i.e., treating each minute as a separate object and counting TP/FP/FN/TN), but at the level of switching intervals—on and off episodes of the end-use device. This metric group better reflects the requirement for stable operation and is closer to human perception of “events”.

The event-based group includes: Precision_event, Recall_event, F_1event. The base counts TP/FP/FN/TN are computed using the rule that an event is considered correctly detected if the start or end of the predicted “ON” interval falls within a tolerance of $\pm \Delta t$ minutes relative to the ground-truth interval.

As an example, for the GAP-only model, both sets of metrics (sample-wise and event-based) were computed. The results are summarized in Table 1.

As shown in Table 1, the TP/FP/FN/TN counts become substantially smaller when moving from sample-wise to event-based evaluation. The metric values also decrease for loads that are weakly distinguishable. These results (Table 1 and the associated ROC/PR curves) were obtained using the primary chronological split with 336 training days followed by 21 validation days.

2.4. Sensitivity Analysis

2.4.1. Feature Selection Sensitivity

To assess the robustness of the proposed two-stage feature selection procedure (collinearity removal followed by permutation importance ranking), we conducted a sensitivity analysis along two dimensions. First, we varied the correlation threshold used to remove highly collinear predictors from the GAP-derived feature set, testing $\left|r\right|\in \{0.98, 0.95, 0.90\}$. Second, we evaluated whether the permutation importance ranking depends on the choice of the estimator by computing permutation importance using the baseline estimator (HGB) and the final selected model (LGBM). In all cases, the feature selection steps were fitted strictly on the training window (chronological split), while model quality was evaluated on the fixed chronological validation window, consistent with the main experiment.

Table 2 summarizes the results. When varying the correlation threshold, the number of retained features changed only slightly (14 features for |r| > 0.98 vs. 12 features for |r| > 0.95/0.90).

The resulting performance variations were limited: the largest absolute deviations across targets were $\Delta AP \le 0.018$ and $\Delta F_{1\mathrm{event}}\le 0.026$.

When switching the permutation importance estimator from HGB to LGBM (with |r|>0.98), AP remained effectively unchanged ($\Delta AP \le 0.0013$ across targets), while event-based F1 changed modestly ($\Delta F_{1\mathrm{event}}\le 0.018$). Overall, these results indicate that the feature selection stage is not brittle with respect to reasonable variations of the collinearity threshold and the choice of the permutation importance estimator.

Table 3 shows the results of calculating sensitivity to model selection when calculating permutation importance in the question of feature selection.

All feature selection steps were performed using training data only (chronological split); the reported metrics are computed on the fixed chronological validation window.

2.4.2. Sensitivity Analysis for the Post-Processing Window Length

To justify the post-processing window length W used in Step 8, we performed a sensitivity analysis of the causal moving average smoothing applied to calibrated probabilities. We evaluated $W \in \{1, 3, 5, 7, 10\}$ minutes using the same strictly chronological split and the same calibration protocol described in Section 2.2.

For each W, the decision threshold ${\tau }^{\ast }$ was re-optimized on the validation window using the target-specific $F_{\beta }$ objective (with $\beta$ as defined in Section 2.2), and both sample-wise and event-based metrics were computed. In addition, we report the worst-case additional latency induced by causal smoothing as $W-1$ min.

Table 4 shows that increasing W consistently reduces the number of false-alarm events (${FP}_{event}$) but increases missed events (${FN}_{event}$), i.e., a classical precision–recall trade-off. For example, when moving from $W=3$ to $W=5$, ${FP}_{event}$ decreases from 120 to 94 for the boiler/AC group, from 139 to 118 for the kitchen group, and from 310 to 272 for the laundry group, while $F{N}_{event}$ increases accordingly.

Although $W=3$ yields the highest event-based F1 for some targets, we retain $W=5$ as the baseline because it provides a more conservative operating point with fewer spurious triggers (false alarms) while keeping the causal smoothing latency within 4 min at $1-\mathit{min}$ sampling, which is acceptable for the targeted DR feedback loop.

3. Results

3.1. Computational Performance

Training the baseline machine learning models yielded the following observations. Depending on the model, training and validation times differed, which directly affects the suitability of specific approaches for online deployment and for integration into demand-response control systems.

LGBM demonstrated the best performance in terms of both training and inference speed. This can be attributed to implementation characteristics: the histogram-based algorithm and efficient parallelization enable high throughput while maintaining classification quality. HistGradientBoosting, despite relying on a similar principle, is somewhat slower, which can be explained by implementation constraints in the scikit-learn Python library (version 1.4.2, Python version—3.11.6).

XGBoost provides stable performance and strong predictive quality; however, under comparable training settings, it is slower than LGBM, particularly on CPU. This is related to less aggressive optimizations in memory handling and parallel execution.

CatBoost, in contrast, is particularly effective when categorical features are present; nevertheless, its computational performance is the lowest among the models considered. This can be attributed to additional computational overhead for handling categorical variables, as well as a more complex, multi-stage optimization procedure.

Overall, based on the computational-performance analysis, the models can be ranked in descending order of speed as follows: LGBM → HGB → XGB → CAT.

More complex neural network models—specifically LSTM-based architectures reported in [25,26] and neural networks of various configurations described in [27,28]—are not suitable for implementation on low-power computing hardware in online mode. Recent NILM studies in IEEE Transactions on Industrial Informatics demonstrate that deep-learning approaches can reach high disaggregation accuracy, including CNN-based trajectory and multi-feature representations and efficient architectures with local feature extraction. However, these methods typically assume richer measurement modalities and/or substantially higher computational budgets compared with the GAP-only, edge-oriented setting considered in this paper. Therefore, we focus on lightweight gradient boosting baselines and explicitly quantify the accuracy–latency trade-offs under strict deployability constraints [29,30]. This limitation becomes a practical barrier to deploying the demand-response programs considered in this paper when load state monitoring is included as part of the control loop, even when using relatively lightweight machine learning approaches.

At the same time, it should be noted that recent edge-oriented neural approaches attempt to mitigate the above limitation via model-compression techniques, including pruning, quantization, and knowledge distillation. These methods reduce the memory footprint and inference cost of deep models, potentially enabling deployment on constrained hardware. However, they address a different design point compared with the present work: compressed neural networks still require a dedicated neural runtime, and their effective latency and energy consumption may remain hardware-dependent, while calibration and decision-thresholding for event-level objectives can introduce additional complexity. In contrast, the proposed pipeline is lightweight by construction and provides predictable computational requirements without specialized neural network stacks. For the binary state-detection task considered here, this design choice yields an efficient baseline suitable for online DR feedback; nevertheless, compressed neural models remain a promising extension for hybrid schemes, in which a lightweight detector triggers a selectively invoked neural module to resolve ambiguous events under strong load overlap.

3.2. Accuracy

In addition to computational performance, model accuracy (according to the metrics described above) was evaluated as a function of the number of days used for training/validation. This analysis supports deployment-oriented assessment, including indicative timelines for data acquisition, commissioning, and routine operation.

Figure 6 presents the $F_1$ score for each model as a function of the number of training days. Note that, for each training set size, the validation interval was selected using physically and logically justified durations: 3/5/7 days when the training set size was up to one year, and 30/60/90 days when the training set size was up to three years. This approach allows validation of the resulting model on time intervals that capture relevant operating periodicities of equipment, shift patterns, and both working and non-working days (including holidays).

Figure 6. ${\mathit{F}}_1$ score as a function of training set size for different targets.

Based on Figure 6, it can be concluded that the dataset size does not affect model accuracy in any consistent manner. Figure 6 reports a sensitivity study where the training horizon is varied; the corresponding validation horizon is also varied to maintain a realistic evaluation period, while preserving the strictly chronological split defined in Section 2.2.

For load (b), represented as an aggregated load group associated with the kitchen, contrary to expectations, increasing the training set size had a negative impact on the final metrics.

For load (a), represented as an aggregated load group labeled “laundry”, a spike in model accuracy is observed, which may be interpreted as a data outlier without an interpretable physical meaning.

3.3. Cross-Seasonal Validation

Household load patterns are known to vary seasonally, especially for HVAC/boiler-related end-use consumption. To evaluate robustness under seasonal shifts, we performed an additional cross-seasonal experiment using strictly chronological, non-overlapping windows. In both cases, the baseline LGBM model was trained on a contiguous historical window, probability calibration (isotonic regression) was fitted only on a dedicated calibration sub-window taken from the end of the training period immediately preceding the validation window, and the final evaluation was conducted on a 21-day validation window from a different season (no information leakage).

We considered two representative season shifts: winter/spring→summer (validation 1 July 2007 to 21 July 2007) and summer/fall→winter (validation 10 January 2008 to 30 January 2008). Table 5 reports AP and event-based performance (±3 min tolerance) for the three targets at the baseline post-processing settings $W=5$ min and target-specific $\beta$. For the boiler/AC group, performance remains high in the winter/spring→summer case ($AP = 0.975$, $F_{1event}=0.759$, ${Recall}_{event}=0.988$) but degrades under the summer/fall→winter shift ($AP = 0.946$, $F_{1event}=0.458$), mainly due to an increase in false-alarm events (${FP}_{event}=133$) despite high recall (${Recall}_{event}=0.797$). For kitchen and laundry, event-based scores remain low in both season shifts ($F_{1event}\approx 0.21-0.22$ for kitchen; $0.152-0.227$ for laundry), indicating that these weaker/overlapping load groups are intrinsically harder to identify from GAP-only features and are more sensitive to behavioral variability.

Table 5. Cross-seasonal validation results for the baseline LGBM model.

Season Shift	Target	AP	ROC-AUC	Event-Based Metrics
				F1	Precision	Recall	TP	FP	FN
summer/fall → winter	Boiler/AC	0.946	0.969	0.458	0.321	0.797	63	133	16
	Kitchen	0.639	0.930	0.221	0.246	0.201	32	98	127
	Laundry	0.387	0.669	0.151	0.201	0.121	25	99	180
winter/spring → summer	Boiler/AC	0.975	0.994	0.758	0.615	0.988	85	53	1
	Kitchen	0.601	0.915	0.216	0.336	0.158	31	61	164
	Laundry	0.592	0.707	0.227	0.139	0.610	102	628	65

3.4. Brief Error Analysis

While the $F_1$ score provides a compact summary, it does not explain the dominant failure modes of GAP-only state inference. Visual inspection of validation windows indicates three typical scenarios.

• Load overlap and correlated switching: false positives appear when a non-target device produces a power step similar in magnitude and duration to the target group, particularly during periods of high household activity (evening hours), where multiple appliances switch in close succession.
• Short and weak events: false negatives are more frequent for short-duration operating intervals and low-power patterns (notably for the kitchen and laundry groups), where the ON interval is comparable to the smoothing window W and can be suppressed by post-processing rules designed to remove spurious spikes.
• Boundary/transition artefacts: errors cluster near state transitions when the probability trajectory crosses the decision threshold ${\tau }^{\ast }$ slowly due to gradual ramps or mixed operation; this produces fragmented events (FP/FN pairs) which pointwise metrics penalize more strongly than event-based metrics with time tolerance.

These observations reinforce the design trade-off of Step 8: increasing W reduces spike-driven false alarms but may miss short events and increases detection latency. Consequently, we report event-based metrics alongside sample-wise measures and include sensitivity results to guide parameter selection under different DR cost asymmetries.

3.5. Application of the Algorithm to Demand Response

Based on the comparative analysis of several machine learning models, we conclude that, as a first approximation, they exhibit similar accuracy in the binary classification setting. Therefore, a rational model choice is to prioritize computational performance, as indicated by the runtime analysis [31].

Given the deployment requirements for a subsystem that monitors load response to demand-response control signals, LGBM was selected as the baseline model. This choice is motivated by the combination of predictive performance and computational efficiency identified in the conducted evaluation. This conclusion is consistent with findings from related applied domains, where LGBM demonstrates a comparable advantage under the “accuracy–speed” trade-off criterion [32,33].

4. Discussion

From a scalability perspective, the results of this study should be evaluated on a broader electricity consumption database. The present work reports results for only three load groups, which also differ in their operational characteristics (as described in Section 2). One direction for further improvement is to expand the pool of end-use devices (or load groups) to be identified within the aggregated consumption profile.

The proposed emphasis on computationally efficient models, rather than heavier and potentially more accurate approaches, may raise questions regarding the sufficiency of the resulting load-activation probability estimates. However, confirming the validity of these models under field conditions requires additional investigation, including the formulation and evaluation of a cost-sensitive error function, as performed in [34].

Importantly, moving from sample-wise metrics (ACC, Precision, Recall, F₁) to event-based metrics leads to an expected decrease in scores and reveals the most vulnerable load classes—primarily low-power and short-duration loads. This finding supports the use of event-based evaluation specifically for NILM in demand response, because the operational value lies less in “minute-by-minute correctness” and more in the correct detection of ON/OFF episodes.

4.1. Areas for Future Research

In addition, a promising research direction is to integrate the proposed system architecture into a hierarchical two-level condition-monitoring framework for electromechanical equipment [35,36]. The current NILM implementation, operating at a low sampling rate, effectively addresses the first level: status identification. For preventing fault conditions, the second level is critical—technical condition diagnosis based on high-frequency electrical-parameter signals [37].

Within a two-level framework, NILM acts as an intelligent trigger to initiate computationally expensive diagnostic procedures, which involve deep spectral analysis of current signals required for defect detection in induction motor drives [38,39,40]. Consequently, continuous execution of such diagnostic computations on edge devices is inefficient or even infeasible [41]. An event-based model can substantially reduce computational and communication requirements, because the diagnostic algorithm is activated only when a stable operating regime of the motor is identified [42].

Active power, analyzed at the first level, can serve as an adaptive parameter for the second level, since the sensitivity of current-based diagnostic algorithms depends directly on the motor loading factor [43]. Information from the NILM module enables dynamic adjustment of diagnostic features, facilitating adaptive algorithms with high reliability across the full range of operating loads [37]. Taken together, these considerations justify the integration of demand-response control and technical diagnostics within a unified system.

This work demonstrates the practical feasibility of monitoring the states of individual end-use devices from an aggregated electricity consumption signal without installing additional measurement channels. The proposed algorithm combines hysteresis-based labeling, construction of a restricted set of GAP features, selection of informative predictors, and subsequent training of classical gradient boosting models. This approach yields operationally usable probabilistic estimates of the “ON/OFF” states even under partial metering coverage and in the presence of a noise component that approximates real-world networks.

4.2. Feature Importance

Feature importance was also assessed. Figure 7 shows a heatmap of the most informative features derived from the aggregated electricity consumption time series P(t). Each feature represents either local signal statistics within a moving window, signal derivatives (dynamics), or temporal periodicity. The features used are defined as follows:

Figure 7. Importance of GAP features for each model and each load group.

1.

gap—the raw aggregated signal value P(t) at the current time; a baseline informative feature representing the consumption level.

2.

gap_roll3_mean—moving mean ${\displaystyle \frac{1}{3}}{\sum }_{k=0}^{2}P\left(t-k\right)$ over a 3-min window; smooths high-frequency noise and captures short-term trends.

3.

gap_roll5_std—moving standard deviation $\mathsf{\sigma }\left(P\left(t\right)\right)$ over a 5-min window; characterizes local variability and the presence of impulsive changes.

4.

gap_roll15_mean—moving mean over a 15-min window; describes a more inertial load component and stable consumption regimes.

5.

gap_roll15_std—moving standard deviation over a 15-min window; reflects medium-term instability/volatility of the signal on the order of a quarter-hour.

6.

gap_roll30_sum—moving sum ${\sum }_{k=0}^{29}P\left(t-k\right)$ over a 30-min window; interpretable as integrated (accumulated) energy over a half-hour horizon and associated with the duration/intensity of activations.

7.

gap_band_mean—a band-pass feature defined as the difference between two moving means of different lengths: $gap\_band\_mean\left(t\right)={\overline{P}}_5\left(t\right)-{\overline{P}}_{25}\left(t\right).$ It highlights variations on the scale of several minutes while suppressing slow drift and very fast noise.

8.

gap_autocorr_lag20—local autocorrelation of P(t) at a 20-min lag, computed within a moving window (20 points in the implementation); describes pattern repetitiveness over a $\sim 20$ min horizon and is suitable for detecting quasi-periodic operating regimes.

9.

hour_sin—sinusoidal cyclic encoding of the hour-of-day. Computes as follows: $hour\_sin\left(t\right)=\sin \left(2\mathsf{\pi }\cdot {\displaystyle \frac{h\left(t\right)}{24}}\right)$, where $h\left(t\right)\in \{0,\dots ,23\}$ is the hour-of-day extracted from the timestamp; it provides a continuous representation of daily periodicity without an artificial discontinuity between 23:00 and 00:00.

10.

hour_cos—cosine cyclic encoding of the hour-of-day. The pair ($hour\_sin, hour\_cos)$ jointly represents the phase of the daily cycle and allows the model to capture regular diurnal dependencies in device activation probability.

4.3. Cybersecurity Aspect

The proposed NILM-based feedback pipeline assumes a trustworthy metering and communication layer. Targeted adversarial manipulation of aggregate measurements (e.g., false-data injection or load-masking attacks) may distort the inferred device states and is not explicitly addressed by the present work. Developing dedicated detection/mitigation mechanisms for such cyber–physical threats is beyond the scope of this paper; interested readers are referred to recent work on smart-grid attacks and defenses [44,45].

4.4. Transferability to Higher Resolution Measurements

The experimental validation in this work is intentionally restricted to a public residential dataset to ensure reproducibility and to match common DR feedback settings in which only low-rate smart-meter GAP streams are available. Therefore, the reported numerical results should be interpreted as a benchmark under a low-information regime rather than as universal performance bounds.

From an identifiability perspective, increasing the sampling rate is expected to improve detection of short-duration and weak loads by preserving transient switch-on/off edges, reducing temporal aliasing, and enabling more informative causal features. In practice, higher resolution can reduce missed events for brief operating intervals and may allow smaller smoothing windows W while maintaining robustness, thereby improving event-based metrics and decreasing latency. At the same time, higher resolution does not eliminate ambiguity under multi-device overlap; in such cases, additional modalities may still be required.

Transfer to industrial and commercial settings is non-trivial because load composition, schedules, and process-driven variability differ from households. The proposed pipeline remains applicable as a generic, lightweight baseline, but deployment in I&C power supply systems would require target-specific re-tuning of $\mathsf{\beta }$, W, and adaptive thresholds, and validation on representative non-residential datasets. Extending the feature vector with process- and power-quality-related predictors appears particularly promising for I&C scenarios.

5. Conclusions

Comparison of multiple machine learning models indicates that, given similar sample-wise and event-based metrics, computational performance becomes the decisive factor for industrial use. On the Household Power Consumption dataset (1-min resolution) using a strictly chronological split (336 training days followed by 21 validation days), the proposed GAP-only pipeline with LGBM achieves high threshold-free performance and competitive event detection quality across the three load groups. In terms of average precision (AP), the best-performing configuration reaches approximately 0.98 for the boiler/AC group, about 0.60 for the kitchen group, and about 0.44 for the laundry group. Under the event-based evaluation with a ±3-minu tolerance, the corresponding event-based F1 scores are approximately 0.77 (boiler/AC), 0.23 (kitchen), and 0.28 (laundry). These values summarize the main operating region of the method and highlight that the strongest performance is achieved for loads with clearer aggregate signatures, while the weakest/imbalanced group (laundry) remains the most challenging case. Based on the results, LGBM is the most suitable baseline model, providing the shortest training and validation times while maintaining the required accuracy. HistGradientBoosting, XGBoost, and CatBoost also demonstrate satisfactory performance; however, their use under the considered constraints is less justified.

The obtained results can be viewed as a first step toward deploying sensorless monitoring within a demand-response control loop: the algorithm can be deployed on edge devices without requiring a dedicated compute cluster and without imposing stringent bandwidth requirements on communication channels.

Further work should include: (a) expanding the set and number of monitored loads, including weakly distinguishable loads and devices with highly variable consumption patterns; (b) evaluating robustness on datasets with different consumption profiles (shift schedules, seasonality, process shutdowns); (c) investigating hybrid schemes in which classical gradient boosting models serve as a lightweight online detector, while heavier neural network models are used to improve event-based metrics and resolve ambiguous cases; (d) assessing robustness to potential drift in feature importance driven by consumption trends. In addition, it appears promising to augment the training feature vectors with process-related features, as in [46], and with features specific to industrial power supply systems, for example, a power-quality feature vector.