NILM-Based Feedback for Demand Response: A Reproducible Binary State-Detection Algorithm Using Active Power
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis manuscript addresses the use of non-intrusive load monitoring (NILM) as a feedback mechanism for demand response (DR) when direct device-state measurements are unavailable. The authors propose a engineering-oriented pipeline to detect on/off states of end-use load groups from an aggregated active-power time series. The method combines hysteresis-based labelling with adaptive thresholds derived from robust statistics, compact feature engineering restricted to global active power (GAP), collinearity removal and permutation-importance feature selection, probabilistic binary classifiers (LightGBM, HGB, XGBoost, CatBoost) with decision-threshold optimisation and post-processing to stabilise predictions. Model performance is evaluated using both sample-wise metrics and event-based metrics with time tolerance and experiments are conducted on the Individual Household Electric Power Consumption dataset. The topic is interesting and the manuscript is well prepared. However, the reviewer has some concerns regarding the paper's contributions and application in this manuscript so this manuscript is not ready for publication in its current form. Here are some questions and comments:
- (1) For the introduction section the literature review should be more comprehensive.
The review should focus more on the core topic of this manuscript. It is also important to add a short part in the introduction reviewing recent AI and machine learning methods ( doi.org/10.3390/s22155872, doi.org/10.3390/en18133431).
- (2) The manuscript must provide a detailed description of all parameters utilized in the formulas and figures. Please check the whole manuscript and add missed parameters to the "Nomenclature" section.
- (3) The dataset section states a peak power of 2 kW versus an average household consumption of 32.5 kW which appears inconsistent in magnitude/interpretation and should be clarified.
- (4) A thorough review and careful editing are strongly recommended to correct writing and grammatical errors, thereby improving the manuscript's linguistic accuracy and overall clarity.
- (5) The paper removes collinear features using |r| > 0.98 and then applies permutation importance (via an HGB model) to retain informative predictors. Please add a short sensitivity analysis showing how results change if the correlation threshold or the feature-importance procedure changes (like |r|=0.95/0.9, or importance computed using the final selected model).
- (6) The manuscript should explicitly report the exact adaptive-threshold equations based on the median and MAD and clearly define the filtering rule for short-duration false triggers.
- (7) The manuscript reports both a specific train/validation split 336/21 days and also varying validation durations (3/5/7 days up to one year/ 30/60/90 days up to three years). Please clarify the exact splitting protocol used for each reported result and confirm all splits are strictly chronological.
- (8) Step 8 mentions smoothing calibrated probabilities but the calibration method is not clearly described. Please specify whether Platt scaling/ isotonic regression/another calibration approach is used and on which split it is fitted.
- (9) The conclusion should present more detailed results to wrap up the manuscript effectively.
A thorough review and careful editing are strongly recommended to correct writing and grammatical errors, thereby improving the manuscript's linguistic accuracy and overall clarity.
Author Response
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper proposes an engineering-oriented pipeline for Non-Intrusive Load Monitoring (NILM) aimed at providing feedback to Demand Response (DR) systems. The core of the study is a binary state-detection algorithm (ON/OFF) for specific load groups using solely Global Active Power (GAP) data. To enhance the manuscript before publication, the following improvements are suggested:
- Provide a more robust justification or optimization for the post-processing window parameter (W=5 min). A sensitivity analysis demonstrating how this value impacts the trade-off between 'false alarms' and 'missed events' would significantly increase the paper's technical depth.
- Implement a hierarchical classification scheme or adjust class weights to mitigate the data imbalance in the y_laundry category, where the current model's performance is notably deficient.
- Given that consumer behaviour and load patterns vary seasonally (particularly for HVAC systems and boilers), a cross-seasonal validation should be included. This is essential to demonstrate the model's robustness against seasonal shifts in load profiles.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsPlease carefully address the following comments before a decision can be made:
1) The hysteresis-based labeling relies on MAD-derived thresholds from GAP only; please justify how this approach remains stable under strong seasonal load drift or multi-device overlap where the median may shift.
2) Restricting features to GAP simplifies deployment, but it is unclear whether this choice fundamentally limits identifiability for devices with weak or variable signatures; a failure-case analysis would strengthen the claim.
3) Removing features using a fixed |r| > 0.98 threshold is ad hoc; please discuss sensitivity to this threshold and whether non-linear dependencies remain unaddressed.
4) The paper does not explicitly discuss how NILM-based state inference interacts with cybersecurity threats in power systems; please clarify whether the proposed pipeline is resilient to adversarial manipulation of aggregate measurements (e.g., false data injection or load-masking attacks) and situate the contribution relative to prior work on cyber–physical attacks in smart grids such as 10.1016/j.ijcip.2025.100807 and 10.1201/9781032719436-12, to name but two.
5) Optimizing decision thresholds via Fβ is task-dependent; how sensitive are results to β, and how should β be selected in real DR applications with asymmetric costs?
6) The smoothing and trigger-suppression stage may artificially inflate event-based metrics; please quantify the latency introduced and its impact on real-time DR responsiveness.
6) Results are shown on a single low-frequency residential dataset; the paper would benefit from discussion or evidence on transferability to higher-resolution data or non-residential loads.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThe paper presents a reproducible, engineering-focused pipeline for Non-Intrusive Load Monitoring (NILM) aimed at providing actionable feedback for Demand Response systems. The core task is binary classification for specific load groups using only an aggregated active power (GAP) time series. The method involves: 1) robust hysteresis-based labeling with adaptive thresholds, 2) compact feature engineering derived solely from GAP, 3) removal of collinear features and selection of informative predictors via permutation importance, and 4) training probabilistic binary classifiers with optimized decision thresholds. A key contribution is the use of event-based evaluation metrics alongside traditional sample-wise metrics. The study, validated on a public residential dataset, demonstrates that lightweight gradient boosting models, particularly LightGBM, offer a favorable balance of reliability, interpretability, and computational efficiency suitable for edge deployment in DR systems.
Several questions were raised regarding the presentation of the paper:
- Is the dataset too outdated in terms of the year? Are the experimental conclusions still applicable to the current situation? If they remain applicable, relevant discussion and justification should be provided. It should also be noted that residential electricity consumption data are not suitable for industrial and commercial scenarios.
- The description of the steps in Section 2.2 can be further elaborated, with emphasis on the function of each step. In addition, an overall workflow diagram can be included to help readers better understand which problem each step addresses?
- In edge deployment design, there are also neural network-oriented algorithms friendly to edge devices, such as pruning and quantization. A brief discussion on the differences between these methods and the approach proposed in this paper can be added?
- In classification tasks, the F1-score alone is insufficient to diagnose model errors. In which time periods or scenarios is the model prone to misclassification? A brief analysis of typical misclassification cases can be supplemented?
- In this paper, the authors propose a lightweight method based on adaptive threshold and LightGBM, which is compared and validated against multiple gradient boosting models, demonstrating greater reliability and efficiency in device switch state detection. A comparative analysis of different methods can further illustrate its advantages. The following articles may serve as references:
[a]IEEE Transactions on Industrial Informatics, DOI: 10.1109/TII.2024.3390595.
[b]IEEE Transactions on Industrial Informatics, vol. 19, no. 11, pp. 10751-10762, 2023.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors have addressed the previous concerns satisfactorily. I have no further comments.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe manuscript has improved substantially compared to the previous version and now provides a technically sound, well-justified, and practically relevant contribution to NILM-based demand response feedback systems. In my opinion, the paper is now ready for publication.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe revised paper has substantially been improved, and the authors have satisfactorily addressed the reviewer's concerns. Good attempt!
