IoT Monitoring Framework with Physics-Based Energy Loss Modeling for Smart Microgrids: Architecture and Benchmarks

Abstract

Smart microgrids combining photovoltaic arrays, wind turbines, and battery storage generate telemetry that existing open-source monitoring tools cannot process with per-mechanism energy loss visibility in real time. This paper presents the design, implementation, and evaluation of an IoT monitoring framework. The framework incorporates a physics-based microgrid simulator, a hierarchical MQTT communication architecture, and a React-based web-based user interface that supports WebSocket-based real-time data visualization. The framework consists of ten containerized microservices that can be started with a single command: docker compose up -d. All stack performance testing was conducted using a simulated 1 h test case based on a 100 kWp PV system, 10 kW wind turbine, and 50 kWh battery-powered campus microgrid. Median P50 publisher-to-subscriber latency was 27.2 ms and 99th percentile (P99) latency was 48.3 ms, with 100% message delivery across 5840 test messages, with per-topic analysis revealing a 25 ms serialization-order effect in sequential MQTT publishing. Comparative analysis against nine existing platforms including OpenEMS, VOLTTRON, Eclipse Ditto, and pymgrid confirms that, among the platforms surveyed, none unifies physics-based loss telemetry, IoT communication, time-series storage, and real-time visualization in a single reproducible deployment.

1. Introduction

Since 2020, the deployment of distributed photovoltaic systems in Europe has grown by more than 40% annually [1], driving campus-scale and community microgrids into configurations that conventional SCADA systems were never designed to monitor. A microgrid combining PV arrays, small wind turbines, battery storage, and time-varying building loads generates heterogeneous telemetry, power flows, voltages, frequencies, state of charge, and weather conditions at sub-minute intervals. Tracking these measurements in real time has been solved for a single metering point. Tracking them across a dozen assets while simultaneously determining where energy is lost between the panel and the common connection point, contamination, cable resistance, inverter conversion, and transformer core magnetization is yet unresolved. That is the monitoring gap this paper addresses.

Digital twin technology has emerged as a candidate solution. A digital twin, in the energy context, is a virtual replica of a physical system synchronized through IoT communication, enabling monitoring, simulation, and decision support without interrupting the physical asset [2,3]. Digital twins for microgrids have attracted considerable attention in recent years. There are five review papers and articles written between 2024 and 2025 on digital twin applications for smart grids and renewable energy systems [2,3,4,5,6], and two others have addressed specific topologies of microgrids [7,8]. Despite the numerous surveys on the topic, the field is lacking in terms of actual, deployable implementations. Out of 112 papers on digital twins for smart energy published by Aghazadeh Ardebili et al. [5] in 2024, 107 of them are based on the conceptual design of digital twins or simulation results without any runnable code, IoT communication layers, or a step-by-step guide for their real-world deployments.

Current solutions only cover fragments of the required capabilities. OpenEMS [9] provides energy flow calculations in a Java OSGi bundle structure and thus serves as a good base. However, it lacks an MQTT communication layer and a physics-based loss model. VOLTTRON [10], provided by the Pacific Northwest National Laboratory, is an agent-based architecture that provides a communication framework using its own protocol. However, it is focused on building-connected applications and not for microgrid-related use cases. Eclipse Ditto [11] is a generic solution for digital twins and is well-engineered. It supports MQTT messaging and comes with a REST API to manage the state of a digital twin. However, it is without energy equations, microgrid-specific topic hierarchies, and a dashboard. ThingsBoard [12] is an open-source software for data collection from IoT devices using the MQTT protocol, as well as for the creation of configurable interactive web dashboards. However, the community edition is limited in respect to connected devices, and there is no physics simulation engine built in. For microgrid simulation using reinforcement learning, we found pymgrid [13], a Python open-source software library for research simulation of a microgrid. While it has capabilities for microgrid simulation, it outputs arrays of numerical values and lacks functions for deployment and real-time monitoring of time-series data from various IoT devices and sensors. Another relevant numerical calculation library is pvlib [14], which computes PV energy losses with high fidelity using validated models including the CEC inverter curves [15] and NOCT cell temperature estimates, but it is a calculation library, not a monitoring platform. For example, Hamid et al. [16] built a MATLAB-based digital twin of a large PV plant with high accuracy (prediction accuracy of 99.77%) in a controlled environment, although it is a proprietary framework that cannot be freely distributed or even containerized. Sivaneasan et al. [7] came closest: their Cognitive DT for a Singapore microgrid achieved 0.24% cost deviation in a real-world deployment, but it uses proprietary components, focuses on optimization rather than monitoring, and does not publish itemized loss telemetry. Izquierdo Monge et al. [17] built an open-source monitoring and alarm system for microgrids, but it lacks physics-based loss models and does not report MQTT performance benchmarks. Prior MQTT work for microgrids is dated and narrow; for example, Arbab Zavar et al. [18] proposed an MQTT communication infrastructure for grid-connected AC microgrid management but published no latency benchmarks, and the most recent MQTT performance study for microgrid communication (Kondoro et al. [19]) is now five years old and tested simpler payloads on different hardware.

In our previous work [20], we presented a web-based hybrid optimization platform for real-time microgrid energy management at SoftCOM 2025. That system demonstrated the viability of combining genetic algorithm and reinforcement learning optimizers with LSTM-based load forecasting for battery dispatch scheduling. However, it was built on Flask (synchronous), had no IoT communication layer, no physics-based loss models, and no containerized deployment, precisely the limitations that motivated the present study.

The gaps in existing approaches fall into four distinct categories: (a) reliance on proprietary tools that prevent reproducibility, (b) absence of physics-based energy loss models that quantify where generation is lost between source and load, (c) lack of real-time IoT communication layers with structured topic hierarchies and published latency analysis, and (d) no reproducible performance benchmarks against which alternative implementations can be compared. In light of these gaps, this paper makes the following four contributions:

• An end-to-end digital twin architecture comprising ten Docker services: simulator, MQTT broker, data-ingestion subscriber, API gateway, time-series database, in-memory cache, frontend dashboard, reverse proxy, and two supporting services deployable via a single ‘docker compose up -d’ command [21].
• A physics-based microgrid simulator implementing nine distinct energy loss mechanisms applied sequentially at 5 s resolution: PV soiling (Kimber linear model [22]), module mismatch, DC cable I²R losses, CEC inverter efficiency curves [15], AC distribution cable losses, transformer iron-copper losses, battery self-discharge, BMS parasitic draw, and wind mechanical losses, each published as itemized MQTT telemetry per simulation step.
• A hierarchical MQTT topic architecture with QoS 1 at-least-once delivery, six topic patterns carrying per-asset measurements including full loss breakdowns, and a dual-write ingestion pipeline that routes messages simultaneously to TimescaleDB hypertables for historical storage and to Valkey pub/sub channels for sub-second WebSocket delivery to the dashboard.
• Reproducible performance benchmarks from a one-hour automated test demonstrating a P50 publisher-to-subscriber latency of 27.2 ms, P99 of 48.3 ms, 100% message delivery reliability across 5840 messages, and a per-topic latency analysis revealing the serialization-order effect inherent in sequential MQTT publishing, the first such benchmark for a microgrid digital twin framework since Kondoro et al. [19] in 2021.

A partial code release, the physics-based loss model, the simulator configuration, and the benchmark scripts, are available from the corresponding author upon reasonable request. The benchmark in Section 3 can be reproduced with one command (‘docker compose up -d’) followed by one more ‘python scripts/benchmark.py’. The full run completes in about 75 min of wall-clock time on the workstation described in Section 3.1.

The remainder of this paper is organized as follows. Section 2 describes the system architecture, simulator, MQTT, data pipeline, web interface and deployment. Section 3 presents the experimental results including latency, throughput, reliability, loss model verification, and platform comparison. In Section 4 we discuss key findings, Industry 5.0 alignment, and limitations. Section 5 concludes with specific future research directions.

2. Materials and Methods

This section covers the overall architecture of the framework, the detailed simulation model using a physics-based simulator, loss models, MQTT communication, data ingestion, real-time visualization, and containerized deployment. All components are built from open-source software and can be deployed on a single machine with a single Docker Compose command.

2.1. System Architecture Overview

Ten containerized services. That is the full count, not a monolithic application partitioned after the fact, but ten independent services, each with its own failure domain, and each designed and implemented from the ground up to serve a specific purpose, with no shared runtime state.

The architecture used for building the campus microgrid digital twin is shown in Figure 1. The architecture is four-layered, progressing from simulation down to application. The Simulation Layer consists of a physics-based microgrid simulator that models PV generation, wind power, campus loads, battery storage, and nine energy loss mechanisms within the microgrid. The simulator models each at a 5 s time step. The IoT Transport Layer includes the IoT broker, Eclipse Mosquitto MQTT message broker [23], and the data-ingestion service used to subscribe to all telemetry topics on the broker and route them to storage and real-time microservices as required. The Data Layer consists of TimescaleDB [24], a PostgreSQL 16 extension that automatically partitions data into hypertables, supplemented by a Valkey open-source Redis-compatible cache that is used to cache the digital twin-state and provide a WebSocket pub/sub bridge between simulation, data store, and application layers. The Application Layer consists of a FastAPI [25] REST and WebSocket gateway that serves up a React 18 web-based dashboard to the end user, as well as Traefik-based HTTP routing.

In addition to the main data path through the broker, we also have one supporting layer for monitoring and observability. To monitor load and performance, Prometheus is configured to scrape the metrics endpoint from each of the two backends every 15 s, which is rendered out into Grafana dashboards. Six alert rules are evaluated on these metrics to flag anomalies in latency, queue depth, and service health.

In the

Table 1. Technology stack of the proposed framework.

Layer	Technology	Version	License	Role
Simulator	Python 3.12	3.12+	PSF	Physics engine
MQTT Broker	Eclipse Mosquitto	2.0+	EPL-2.0	Message transport
Backend	FastAPI	0.115+	MIT	REST API + WebSocket
Time-Series DB	TimescaleDB (PG 16)	2.17+	Apache-2.0	Measurement storage
Cache/Broker	Valkey	8.0+	BSD-3	Pub/sub + state cache
Frontend	React 18 + TypeScript	18.3+	MIT	Dashboard UI
UI Components	shadcn/ui + Radix	latest	MIT	Accessible primitives
Charts	Recharts	2.12+	MIT	Data visualization
State Management	Zustand	5+	MIT	Client state
Containerization	Docker Compose	27+	Apache-2.0	Orchestration
Monitoring	Prometheus + Grafana	latest	Apache/AGPL	Metrics + alerting
Reverse Proxy	Traefik	3.2+	MIT	Routing + TLS

below is the entire stack of technology used. Every part of the stack consists of open-source components: all of them have permissive free-to-use licenses (open-source MIT license, Apache-2.0, EPL-2.0, BSD-3). We chose TimescaleDB instead of InfluxDB for our time-series database for support of full SQL and the ability to utilize the many PostgreSQL features and tools like pg_dump, psql, and Alembic for schema migrations. For the cache we picked Valkey over Redis after the relicensing of Redis in 2024, an event that created ambiguity in the license. We utilize Zustand [26] for our dashboard state store because we only want to have to write as much boilerplate as necessary. We chose Mosquitto [23] because it uses very little memory and has proven reliability in IoT deployments.

2.2. Physics-Based Microgrid Simulator

Three weather sources feed the simulator. The choice is explicit, set in configuration and independent of the simulation speed. At each step the physics engine reads the current conditions, computes every asset’s electrical state, applies the nine loss mechanisms in sequence, and publishes the result as structured MQTT messages across the per-asset topic hierarchy. The default is a synthetic generator: an AR(1) cloud-cover process layered with seasonal temperature for Lipjan, rain events triggered by high humidity and dense cloud, and intermittent wind gusts, while a recorded CSV profile replays measured data and a third source queries the live Open-Meteo API [27] for the UBT campus coordinates (42.53° N, 21.12° E). The benchmark in Section 3 used the synthetic generator. Its latency, throughput, and reliability follow from the message schedule and the payload structure, not from the weather values, so they do not depend on the particular weather trace. The generator is seeded so that trace can be reproduced exactly, and we set the seed to 42.

2.2.1. Simulation Pipeline Overview

The above results in the long pipeline illustrated in Figure 2 for each simulation timestep. The first four steps of this process compute the basic generation and demand: raw weather (irradiance, temperature, wind speed, humidity, rain indicator), raw generation from Equation (1), raw wind generation from a cubic power curve defined by cut-in, rated, and cut-out (3 m/s, 12 m/s, 25 m/s for the 10 kW turbine), and campus building load from an interpolated diurnal load profile that ranges from a 30 kW nighttime base load to an 80 kW daytime peak load.

In Step 5 of the loss chain, the nine mechanisms for loss are applied in sequence to compute the lossy transformation from gross generation to net power delivered. Following that, Steps 6 through 8 handle demand-response curtailment, diesel, and EV charging, which are disabled in every benchmark reported in Section 3 and are described here only to document the configuration surface of the simulator. Step 9 applies a time-of-use-aware battery dispatch strategy. Step 10 calculates the SOC of batteries, including self-discharge and BMS parasitic current drain according to Equation (8). Step 11 calculates the grid power balance at the PCC per Equation (9). Finally, Step 12 publishes results via MQTT with a detailed breakdown of losses. The simulation speed is configurable, from 1× (real time) up to 3600× (one hour simulated per real second), or even faster if needed. At 3600× speed, with a 5 s physics interval, the simulator can produce an average of 720 steps per simulated hour, and approximately eight MQTT messages per step, spread across seven different topic types.

2.2.2. Photovoltaic Model with Energy Loss Chain

The PV model starts with DC output of the PV system via the following equation:

$$P_{\mathrm{pv},\mathrm{dc}}=G\cdot A\cdot {\eta }_0\cdot [1-\beta \cdot \left(T_{\mathrm{cell}}-T_{\mathrm{ref}}\right)],$$

(1)

where G is plane-of-array irradiance (W/m²), A = 555 m² is total panel area, η₀ = 0.18 is the reference efficiency at standard test conditions, T_ref = 25 °C is the reference cell temperature during standard test conditions, and β = 0.004 K⁻¹ is the temperature power coefficient. Cell temperature is estimated via the NOCT model:

$$T_{\mathrm{cell}}=T_{\mathrm{amb}}+{\displaystyle \frac{(\mathrm{NOCT}-20)}{800}}\cdot G,$$

(2)

with NOCT = 45 °C. At 30 °C ambience and 800 W/m² irradiance, cell temperature reaches 55 °C, causing approximately 12% power derating.

Nine loss mechanisms are then applied in sequence. Figure 3 shows the loss chain from gross DC to net AC delivery and

Table 2. Energy loss model parameters.

#	Loss Mechanism	Model	Default Value	Configurable
1	PV Soiling	Kimber linear + rain reset [22]	0.2%/day	Yes
2	PV Mismatch	Fixed factor	1.5%	Yes
3	DC Cable	I2R at 600 V	R = 0.15 Ω	Yes
4	PV Inverter	CEC weighted [15]	100 kW SMA ref.	Yes
5	AC Cable	I2R at 400 V, 3-phase	R = 0.08 Ω	Yes
6	Transformer	Iron + copper	250 kVA, 0.75 + 3 kW	Yes
7	Battery Self-Discharge	Linear SOC drain	2%/month	Yes
8	BMS Parasitic	Constant draw	50 W	Yes
9	Wind Mechanical	Drivetrain + yaw	5% + 2%	Yes

lists all parameters.

Soiling follows the Kimber linear accumulation model [22]:

$$f_{\mathrm{soil}}\left(t\right)=\mathrm{m}\mathrm{a}\mathrm{x}\left(0.85,\hspace{0.33em}1.0-r_{\mathrm{soil}}\cdot d_{\mathrm{rain}}\left(t\right)\right),$$

(3)

where r_soil = 0.002 per day is the daily accumulation rate and d_rain(t) is days since last rainfall. A rain event resets the soiling factor to 0.95 (not 1.0; residual dust persists) and d_rain(t) restarts from zero. After 75 consecutive dry days, the factor saturates at its floor of 0.85.

Module mismatch is modeled as a fixed 1.5% derating factor, representing manufacturing variance and thermal non-uniformity across the string.

DC cable losses follow an ohmic I²R model:

$$P_{\mathrm{loss},\mathrm{dc}}={\displaystyle \frac{P^2\cdot R_{\mathrm{dc}}}{V_{\mathrm{dc}}^2}},$$

(4)

with R_dc = 0.15 Ω (100 m of 6 mm² copper conductor) and V_dc = 600 V (MPP string voltage). At 50 kW, this yields 1.04 kW loss (2.1%).

Inverter efficiency uses the California Energy Commission (CEC) weighted model [15]:

$${\eta }_{\mathrm{inv}}(p)={\displaystyle \frac{p}{p+k_0+k_{1}p+k_2{p}^2}},$$

(5)

where p = P_actual/P_rated is the normalized load point. The coefficients k₀ = 0.0072, k₁ = 0.0096, and k₂ = 0.0044 are derived from the SMA Sunny Tripower 100 kW datasheet. At half load, efficiency reaches 97.4%. At 10% load, it drops to 92.4%. Below the minimum power threshold, the inverter enters standby at 30 W constant consumption. The CEC model captures what a single fixed-efficiency value cannot: the nonlinear relationship between inverter loading and conversion loss.

AC distribution cable losses use the same I²R formulation for three-phase conductors:

$$P_{\mathrm{loss},\mathrm{ac}}={\displaystyle \frac{P^2\cdot R_{\mathrm{ac}}}{(V_{\mathrm{ac}}\cdot \mathrm{PF}{)}^2}},$$

(6)

with R_ac = 0.08 Ω (200 m of 16 mm² copper), V_ac = 400 V, and PF = 0.95.

Transformer losses combine constant iron (no-load core) losses with load-dependent copper (winding) losses:

$$P_{\mathrm{tf}}=P_{\mathrm{iron}}+P_{\mathrm{copper}}\cdot {\left({\displaystyle \frac{S}{S_{\mathrm{rated}}}}\right)}^2,$$

(7)

with P_iron = 0.75 kW, P_copper = 3.0 kW, and S_rated = 250 kVA. At no load, the transformer still consumes 0.75 kW, an always-on penalty that appearsas a constant baseline across the 24 h simulation.

The remaining three mechanisms are: battery self-discharge at 2% per month (NMC Li-ion typical), BMS parasitic draw of 50 W constant (battery management system electronics and cooling), and wind mechanical losses combining drivetrain friction (η = 0.95) with yaw misalignment (2% average error), yielding a combined 6.9% derating on gross wind output.

2.2.3. Battery Energy Storage Model

The 50 kWh lithium-ion NMC battery is modeled with bidirectional power flow, asymmetric charge/discharge efficiency, and state-of-charge constraints:

$$\mathrm{SOC}(t+\Delta t)=\mathrm{SOC}(t)+{\displaystyle \frac{(P_{\mathrm{ch}}\cdot {\eta }_c-P_{\mathrm{dis}}/{\eta }_d)\cdot \Delta t}{E_{\mathrm{cap}}}}-{\displaystyle \frac{r_{\mathrm{sd}}\cdot \Delta t}{T_{\mathrm{month}}}}-{\displaystyle \frac{P_{\mathrm{bms}}\cdot \Delta t}{E_{\mathrm{cap}}}},$$

(8)

where η_c = η_d = 0.95, E_cap = 50 kWh, and the self-discharge term r_sd = 0.02 per month drains SOC continuously. SOC is bounded between 10% and 95% to prevent deep discharge and overcharge. Maximum charge and discharge rates are each limited to 25 kW (0.5C).

A time-of-use dispatch strategy governs battery behavior: charge from grid during off-peak hours (22:00–06:00) when SOC falls below 80%, discharge to offset load during peak hours (09:00–21:00) when SOC exceeds the minimum, and surplus/deficit during shoulder periods. The battery’s bidirectional inverter introduces its own conversion losses, modeled identically to the PV inverter using scaled CEC coefficients for the 25 kW power rating.

2.2.4. Grid Power Balance

The power balance at the point of common coupling is as follows:

$$P_{\mathrm{grid}}=P_{\mathrm{load}}-P_{\mathrm{pv},\mathrm{ac}}-P_{\mathrm{wind},\mathrm{ac}}+P_{\mathrm{batt},\mathrm{net}}+P_{\mathrm{dist},\mathrm{loss}}$$

(9)

Positive values indicate grid import. Negative values indicate export. The distribution loss term P_dist, loss sums the AC cable and transformer losses from Equations (6) and (7). Grid voltage is modeled at 400 V three-phase with 50 Hz nominal frequency, consistent with Kosovo’s distribution standard. Probabilistic grid disturbance events, voltage sags (~5% per simulated hour), frequency excursions (~2% per hour), and brief outages (~0.5% per hour) are injected stochastically to test the dashboard’s alert handling, following the voltage ride-through requirements of IEEE 1547-2018 [28].

2.3. MQTT Communication Architecture

The topic hierarchy was designed for one purpose: to enable a single wildcard subscription (‘microgrid/+/#’) to capture all telemetry for a given microgrid while preserving per-asset, per-measurement-type granularity.

Figure 4 illustrates the topic patterns which are implemented within this project. They serve six different purposes, which are listed in

Table 3. MQTT message schemas.

Topic Pattern	Key Payload Fields	QoS	Rate	Avg Size
‘microgrid/{id}/energy/{type}’	power_kw, energy_kwh, losses_kw, loss_breakdown, voltage_v, current_a, power_factor, frequency_hz, reactive_power_kvar	1	Per step	374–604 B
‘microgrid/{id}/battery/state’	soc_percent, soh_percent, charging_power_kw, discharging_power_kw, inverter_loss_kw, bms_parasitic_kw, temperature_c, cycle_count	1	Per step	523 B
‘microgrid/{id}/losses/summary’	total_loss_kw, total_loss_percent, generation_gross_kw, generation_net_kw, pv_losses{}, wind_losses{}, battery_losses{}, distribution_losses{}	1	Per step	706 B
‘microgrid/{id}/weather’	temperature_c, humidity_percent, solar_irradiance_wm2, wind_speed_ms, cloud_cover_percent, is_raining	1	Per step	397 B
‘microgrid/{id}/control/command’	command_type, parameters, target_asset_id	1	On demand	Variable
‘microgrid/{id}/control/status’	state, step_count, sim_time, messages_published	0	Every 10 steps	247 B

. Energy topics of the form “microgrid/{id}/energy/{type}” contain per-asset electrical data, such as power, voltage, current, and frequency, and an itemized loss breakdown as a nested JSON object. A battery state topic exists for each step, with information regarding SOC, SOH, charging/discharging power, cycle count, and inverter loss. Additionally, there is a losses summary that sums up all the losses of the nine different mechanisms within a single message. This is unique in comparison with common messaging platforms, a feature that no other platform in our comparison provides via MQTT.

All data topics use QoS 1 (at-least-once delivery) with a PUBACK acknowledgment from the broker before a publisher can send the next message. The control status topic uses QoS 0 (fire-and-forget) because losing an occasional status heartbeat is acceptable, but losing a measurement is not. This is a trade-off where QoS 1 adds about 15 ms of round-trip overhead per message (the dominant contributor to the per-topic latency spread discussed in Section 3.2), but it guarantees that no telemetry point is silently dropped.

All data messages include three timestamps, created_at, published_at, and sim_time. The created_at is the start of the simulation step. The published_at is the point just before the MQTT publish() method is called, and the sim_time is the virtual physics clock at that point. This allows for publisher-to-subscriber latency calculation. The MQTT transit latency is calculated as the difference between published_at and datetime.now() on the ingestion service, as reported in Section 3.2.

2.4. Data Ingestion and Storage Pipeline

Our data-ingestion service is a self-contained FastAPI application that utilizes the aiomqtt async client to connect to our MQTT broker to subscribe to all data topics on the broker. All received messages are then funneled through a six-step processing pipeline. First, messages are deduplicated by checking if a Valkey SET with a TTL of 10 s exists in the database with a key of ‘(asset_id, timestamp)‘. If such a message already exists, the function returns without inserting anything to prevent the QoS 1 retransmit message from creating an additional, duplicated entry in the database. The source of each message is then classified as either simulator, physical sensor (Modbus or MQTT device), or hybrid, together with any applicable optional sensor calibration (CT ratio scaling, offset correction) for the asset the message originated from. After that, level alerts are asynchronously evaluated to check for conditions such as low voltage (below 360 V) and frequency (±0.5 Hz) deviance. These alerts are then saved and broadcast. Finally, the message is inserted into the relevant hypertable in TimescaleDB with the message’s timestamp (not the database server’s ‘NOW()’) used as the time dimension, with the options hypertable energy_measurements, battery_states, weather_data, or energy_losses depending on asset and message. All messages are also published to the Valkey pub/sub channel ws:microgrid:{id} in order to be able to immediately supply connected clients with a WebSocket connection to the dashboard. Inspired by the Eclipse Ditto [11] twin-state cache pattern to store the reported and desired state of a digital twin separately, we have built a twin-state cache for assets in Valkey. This cache stores the latest known value of all assets in Valkey hashes with a TTL of 5 min. The dashboard can then directly query this cache via a REST endpoint as soon as it connects, and then subscribe to the WebSocket stream for live updates as they happen. This means that we do not have to wait for the next simulation step before the interface is populated again.

2.5. Real-Time Visualization Dashboard

The new dashboard is a fully functional React 18 single-page application built with TypeScript and styled with the latest version of Tailwind CSS 4, as shown in Figure 5. The single-page application is powered by React component primitives from shadcn/ui to improve accessibility. Client-side stores are managed with Zustand [26], and server-state is synchronized with TanStack React Query. Charts and graphs are rendered with Recharts [29].

We consume real-time data via a WebSocket connection and JWT token, and handle reconnects with exponential backoff starting at 3 s and maxing out at 30 s. The energy map in the dashboard is persisted by asset ID between flushes, but all other information (battery, weather, and loss) is stored in an ephemeral buffer that is cleared every 2 s. At each 2 s flush, the power value from multiple devices of the same type is summed together (two PV panels produce one aggregate reading for the dashboard, instead of two separate readings), while the voltage and frequency is averaged. This allows us to accurately represent a microgrid with multiple devices without having to manually aggregate the data.

Five metric cards show live PV generation, wind, load, battery (charge/discharge), and grid (import/export), with a 30-point sparkline and a ‘freshness’ indicator that changes from green (received within 15 s) to gray (stale). Animated energy flow diagrams show the direction of power flow between generation sources, battery, load, and grid, with the width of the arrows representative of the power. Figure 6 shows the loss waterfall chart that is also updated in real time to show the breakdown of the nine mechanisms of loss, as described in Section 3. The Communication metrics strip shows the same MQTT latency percentiles, message throughput, and delivery reliability, as shown in Section 3.

2.6. Containerized Deployment

With one command (‘docker compose up-d’), all ten services can be up and running in seconds. The command will create named volumes for all of the persistent data stores (TimescaleDB, Valkey, Mosquitto, Prometheus, Grafana), set up health checks, and enable inter-service networking. Each service supports a health check that the service monitors every 30 s. For TimescaleDB, this is ‘pg_isready’, for Valkey it is ‘valkey-cli ping’, for Mosquitto it is a TCP connect to port 1883, and regarding the Python services, it is a simple health endpoint/health. For the services that have dependencies, we have set ‘depends_on’ on the API gateway to condition: service_healthy so that it does not attempt to connect to the TimescaleDB database until it is ready.

Our multi-stage Dockerfiles keep production image sizes down as only the builder image needs to have all dependencies installed. The build step produces compiled files that are then copied into the runtime image, which is then deployed to production. All containers run as non-root users. The production override (docker-compose.prod.yml) contains a number of important config options, including CPU and memory limits, the number of CPUs and GB of RAM assigned to TimescaleDB (2 each), the number of CPUs and GB of RAM assigned to the API gateway (1 each), and the amount of CPUs and MB of RAM assigned to lightweight services (0.5 and 256 MB, respectively). All services in production have JSON formatted logging with file rotation enabled. The Traefik container has TLS termination enabled, and the API gateway has the debug port disabled.

3. Results

This section presents results from a one-hour automated benchmark applied to the entire service stack. First, we describe the experimental setup and use it to report on latency, throughput, and reliability numbers. We then verify our energy loss models for the individual services and conclude by positioning our stack against nine existing platforms.

3.1. Experimental Setup

All results reported in this section are from a single-host simulated benchmark. Multi-host distributed deployment and validation against physical hardware are outside the scope of this paper and are part of future work (see Section 5).

All services were run on a single workstation. The hardware consisted of an AMD Ryzen 7 5800X (8 cores, 3.8 GHz base speed) and 32 GB DDR4-3200 RAM, with 1 TB storage on a NVMe SSD. The operating system was Windows 11 Pro, with Docker Desktop 4.37. The 10 containers—simulator, Mosquitto, data-ingestion, API gateway, TimescaleDB, Valkey, frontend, Traefik, Prometheus, and Grafana—were orchestrated via a single command: ‘docker compose up-d’. The services were allocated default resources for their containers.

We ran the benchmark using scripts/benchmark.py at 3600× real time. The microgrid consists of 100 kWp solar (2 × 50 kWp), 10 kW horizontal axis wind-turbine-generating, and 50 kWh lithium-ion NMC battery-storing-energy to supply the building load of 80 kW that we modeled after the UBT Campus in Lipjan, Kosovo (42.53° N, 21.12° E). We excluded a 60 s warmup period and reported all numbers gathered over 3540 s in 355 polling intervals.

Latency breakouts are summarized in

Table 4. Publisher-to-Subscriber MQTT Latency Results (Aggregate).

Metric	Value
P50 (mean across polling intervals)	27.2 ms
P50 (max across polling intervals)	27.7 ms
P95 (mean across polling intervals)	42.3 ms
P95 (max across polling intervals)	46.4 ms
P99 (mean across polling intervals)	48.3 ms
P99 (max across polling intervals)	74.8 ms
Messages under 50 ms	96.1% (5610/5840)
Messages under 100 ms	100.0% (5840/5840)
Messages over 100 ms	0

. Latency per topic is provided in

Table 5. Per-Topic Latency Breakdown.

Topic	Message Count	Rate (msg/min)	Avg. Latency (ms)	Avg Payload Size (B)
control/status	75	1	1.01	247
energy/solar_pv	1474	24	15.09	604
energy/wind_turbine	737	12	21.65	485
energy/critical_load	737	12	25.83	374
energy/grid_connection_point	737	12	30.00	524
battery/state	737	12	30.73	523
weather	737	12	36.71	397
losses/summary	737	12	39.53	706

. The same data is presented graphically in the following sections.

3.2. Publisher-to-Subscriber MQTT Latency Analysis

Every one of the 5840 messages arrived below 75 ms. The P50 had a latency of 27.2 ms, and the P99 had a latency of 48.3 ms. For clarity, in this paper “end-to-end latency” refers specifically to publisher-to-subscriber MQTT latency. It is the time from the moment the simulator stamps a “published_at” value right before it sends the MQTT packet to the time the data-ingestion service receives and has parsed the message. This therefore includes the time to transmit the message down the broker, the time for TCP ACKs (QoS 1 requires a PUBACK), as well as the time to deserialize the JSON object and perform initial processing. It does not, however, include the time to write to the database, which occurs asynchronously after this measurement is taken, and not the WebSocket delivery to the dashboard.

Figure 7 shows the cumulative distribution function. The function rises steeply in the 10 to 40 ms range, indicating that 96.1% of messages arrived within 50 ms. A total of 18 messages (0.3%) took longer than 50 ms and interestingly enough, not a single message took longer than 100 ms to complete delivery. The P50 value also remained consistent during the 1 h test period, fluctuating within a 1.8 ms window of 25.9 ms to 27.7 ms.

Further examination of the average latency per topic in Table 5 shows that latency increases monotonically from 15.09 ms for the first topic, ‘energy/solar_pv’, to 39.53 ms for the last topic, ‘losses/summary’. The ~25 ms difference between the first and last topics is due to serialization overhead and not network congestion or broker queuing. Each 5 s step of the simulator publishes seven different topic types. Each ’publish()’ call to the serializer must wait for the corresponding PUBACK message before continuing, which introduces delay. Interestingly, the aggregate P50 latency of 27.2 ms masked this effect. We have learned that topic publication order should generally follow a logic that ensures the topics operators need to know about the soonest are those published with the lowest latency. In this case, that means PV generation and battery state.

The control/status topic was published at QoS 0 every 10 steps, with an average latency of 1.01 ms, showing that QoS 0 eliminates the PUBACK round-trip overhead that is so dominant of the delivery times for QoS 1.

3.3. Throughput Analysis

The pipeline maintained an average throughput of 1.62 msg/s, with a standard deviation of 0.08 msg/s over a 3540 s effective run window. Figure 8 is a throughput time-series updated at 10 s intervals.

Throughput was constant at 353/355 polling intervals, with values of 1.59–1.70 msg/s. There were two brief anomalies, with a single decrease to 0.80 msg/s at around the 45 min mark followed by a spike to 2.50 msg/s in the next interval. This is consistent with a brief TCP stall likely caused by garbage collection in the Python simulator process, followed by transmission of messages buffered during the stall. The pipeline resumed normal operation within one interval. Message count was unchanged and there were no lost messages.

So, why is 1.62 msg/s worth talking about? The simulator publishes eight messages every 5 s, or seven regular data topics and one status message every 10 steps, which at 3600× speed completes in approximately 1.4 ms of wall-clock time, yielding a theoretical rate of ~1.6 msg/s averaged over the polling interval. Of course, this rate reflects the simulator’s step frequency, not the pipeline’s capacity ceiling, and the Mosquitto broker, the data-ingestion subscriber, and the TimescaleDB writer can each handle traffic of greater orders of magnitude than a single microgrid generates.

3.4. Reliability Analysis

No reconnections or service restarts occurred and no message drops were observed. The data-ingestion service received and stored 5840 messages published by the message-simulator service, which were stored in TimescaleDB. Failures: zero. Delivery reliability: 100.0%. Effective window: 3540 s.

In order to guarantee QoS 1 (at-least-once) delivery, we implemented a layer of deduplication in the data-ingestion service, where Valkey SET messages are stored with a TTL of 10 s, indexed on (asset_id, timestamp). As it turned out, no retransmissions were ever needed during the benchmark, the deduplication key was never triggered for the 5840 messages sent, demonstrating a clean first-attempt delivery across the board.

3.5. Energy Loss Model Verification

The loss values presented in this section are physics-plausible against published parameter sets (see [15,22,30]). They are not field-calibrated. Field calibration against the UBT Campus installation is the subject of the follow-up paper described in Section 5.

This is shown in Figure 9 for a typical day. Each loss, starting from the gross PV DC power generation, successively step-reduces the ultimately delivered power: soiling (0.2–1.5% lost for days since last rain, etc.), mismatch (1.5% loss), DC cable I²R (0.3–2.1% loss depending on power level), inverter DC/AC conversion (2.6–8.7% loss at varying load points via CEC datasheet curves), AC cable I²R (0.5–4.4% loss), and transformer iron-copper loss (0.75 kW constant plus a component proportional to load). For average meteorological conditions, this results in a total PV chain loss of 11.4% at peak midday for the 85 kW AC that is delivered after losses for the 100 kWp DC nameplate system power.

Detailed losses for the system are presented in Figure 10 in the form of a 24 h stacked area chart. The inverter’s losses are generally highest during times of high irradiance. This is due in large part to the effect of the CEC model’s quadratic term (k₂ = 0.0044). At dawn and dusk, the PV system is generating less than 10% of nameplate. Under these conditions, inverter efficiency falls to 92.4%, from 97.4% at half-nameplate, and the standby consumption of 30 W becomes a measurable fraction of the already small output. The transformer’s iron losses (0.75 kW) are constant, and appear as the constant baseline in Figure 10.

The loss values obtained in the analysis are physics-plausible rather than field calibrated. These values were validated against the manufacturer-provided inverter power versus voltage characteristics for a SMA Sunny Tripower series [15], as well as against the Kimber linear soiling model [22]. The magnitude of the losses obtained in this analysis falls in the range reported in IEC 61724-1 [30] for monitoring grade PV systems. It is important to note that the absolute loss percentages obtained here are representative rather than definitive until physical sensor data from an operational microgrid installation is used to calibrate the model parameters.

3.6. Platform Comparison

Table 6. Comparison of Messaging Platforms.

Platform	Simulator	MQTT	Dashboard	Loss Model	Container	Benchmark
This Work	12-step, 9 losses	Hierarchical	React + WS	Itemized, real-time	Docker × 10	P50/P95/P99
OpenEMS [9]	No	No	Partial	No	Partial	No
VOLTTRON [10]	No	No	No	No	No	No
Eclipse Ditto [11]	No	Generic	No	No	Yes	No
Cognitive DT [7]	Proprietary	Custom	Custom	Partial	Unknown	Cost only
pymgrid [13]	Basic	No	No	No	No	No
pvlib [14]	PV only	No	No	PV Only	No	No
ThingsBoard [12]	No	Yes	Generic	No	Yes	No
Hamid et al. [16]	MATLAB	No	No	PV only	No	No
Izquierdo et al. [17]	No	No	Partial	No	No	No

compares our developed framework to nine current frameworks/platforms based on five criteria: IoT layer for communication, loss modeling using physics, real-time interactive dashboard, benchmarking results, and reproducibility.

Each of the software solutions considered for monitoring provides one or more of the necessary monitoring layers. OpenEMS [9] provides monitoring of energy flows in the form of Java OSGi bundles; however, it does not provide a layer for MQTT communication, loss modeling, or a dashboard to view the information. VOLTTRON [10] is an agent-based architecture that provides a messaging layer, but the energy models implemented are specific to buildings and do not include PV loss chains or battery degradation. Eclipse Ditto [11] is a generic digital twin framework that is well-engineered, but it does not include energy equations, microgrid-specific topic hierarchy, or a visualization layer. ThingsBoard [12] provides a dashboard and MQTT support, but the community edition has a device limit and does not include a physics engine. pymgrid [13] is a python simulation of microgrids for AI research purposes but has no IoT layer, no deployment tooling, and no real-time interface. pvlib [14] provides very accurate computation of PV losses but is a library, not a deployable platform.

The closest work to this project is Sivaneasan et al.’s Cognitive DT [7], which achieved 0.24% cost deviation in a Singaporean microgrid deployment. However, it used proprietary components and focused on optimization rather than monitoring. Another open-source microgrid monitoring system by Izquierdo Monge et al. [17] is able to issue proper alerts and provides a Grafana dashboard, but does not include any physics-based loss modeling or MQTT performance benchmarks.

There is currently no platform or solution that integrates all of the requirements outlined in the above framework (physics simulator with loss telemetry, MQTT IoT communication with per-topic QoS, time-series data pipeline and interactive real-time dashboard). Specifically, there is no single containerized image that combines all of these functionalities along with performance benchmarking metrics.

4. Discussion

4.1. Key Findings and Implications

We present four findings based on benchmark data and platform comparison results, and discuss them in the context of designing a monitoring system for a microgrid. We show that sub-50 ms publisher-to-subscriber latency can be achieved with existing hardware and software. The P99 of 48.3 ms for 5840 messages sent over a period of one hour is below the monitoring interval set forth in IEC 61724-1 [30], and is suitable for monitoring and supervisory display. However, it is not suitable for protection-class control, which requires latency on the order of <4 ms, according to IEC 61850.

Secondly, we observe the serialization effect in how these topics are published sequentially from the simulation. This effect is not captured in our aggregate latency metrics. Specifically, there is a 25 ms delta between the fastest topic (solar_pv at 15 ms) and the slowest topic (losses/summary at 40 ms). That means for an administrator who cares most about seeing the battery state (e.g., during a grid outages), simply reordering the publication of these topics would result in battery telemetry arriving 19 ms sooner, with no change to the underlying code. This observation, which provides a per-topic latency insight that is not available in prior microgrid MQTT studies, required no code modification to determine.

Third, physics-based loss modeling can provide insight into the losses on a per-mechanism basis. While energy metering can provide useful aggregate information, it does not offer the type of granular per-mechanism information provided by a detailed loss model. For example, the dominant loss mechanism at the CEC inlet inverter is seen to shift from 42% of the total loss at midday to 18% at dawn in Figure 9 due to the highly nonlinear form of the k₂ term that a constant-efficiency model would entirely miss. This insight would not otherwise be available to the microgrid operator. Note that, in contrast to the measured energy loss values provided by the energy meters, these fractional contributions are model-derived and not measured at the sensor. Nevertheless, they can be very useful for making scheduling, maintenance, and economic decisions. The soiling loss is seen to be the dominant loss during dry weeks, with the transformer iron loss equal to 0.75 kW running continuously 24 h per day, for all levels of generation.

Fourth, our benchmark package includes single commands that enable deployment of the ten services, start-up, health check, and connectivity within 90 s. This removes what the Linux Foundation [31] identified as the primary barrier to open-source microgrid adoption: complex setup procedures that discourage evaluation by practitioners and researchers who are not DevOps specialists. Our ten services start with a single ’docker compose up -d’ command. Reproducing the benchmark requires two commands.

4.2. Industry 5.0 Alignment

The European Commission’s Industry 5.0 vision [32] encompasses three key aspects: human-centricity, sustainability, and resilience. These aspects have recently been studied in the context of smart grids in several reviews [33,34], but first a working implementation needs to be developed for each of them. Our framework implements each of the three pillars by a set of testable features.

The connection to sustainability is most direct. The nine-mechanism loss chain quantifies exactly how much energy loss occurs between generation and consumption, mapped to the physical mechanisms driving the loss. We see that soiling has accumulated 3% over a dry week. The site operator knows immediately that scheduling a panel cleaning will result in recovered energy and a reduction in the carbon intensity per delivered unit of energy. Real-time loss data means this information is always available, not left buried in monthly reports.

The dashboard is human-centric. The energy flow diagram is designed first and foremost for non-technical building managers who run the actual campus microgrid. The color-coded metric cards and the loss waterfall graph are as well. The small green/gray dots next to each widget are designed to remove ambiguity as to whether the data is fresh, and the dashboard updates every 2 s to match human perceptual limits of what constitutes “real-time” data viewing.

We have some, but not complete, coverage of resilience. Health monitoring classifies each device as online (we received data in the last 60 s), stale (we received data 60–120 s ago), or offline (we have not heard from it in more than 120 s). The health monitoring alert pipeline does fire for occasional voltage sags and frequency deviations, as per IEEE 1547 [28]. However we do not yet have automatic failover or redundant MQTT paths, and these remain for future work.

4.3. Limitations and Threats to Validity

Only simulated data were validated for this benchmark. No physical devices were connected. The simulator’s physics-based models use published parameters (CEC [15], Kimber [22], NOCT) and the output should fall within the expected IEC 61724-1 [30] ranges, but absolute loss percentages are not intended to be used as a basis of performance in actual field scenarios.

We did not test network saturation. Based on a throughput of 1.62 msg/s at the simulator step frequency, operation of a microgrid in realistic scenarios is supported by the pipeline. We do not have a measurement for the pipeline’s maximum capacity in terms of messages per second before Mosquitto packet drop or TimescaleDB insertion latency becomes unacceptable.

Loss model parameters are provided as literature default values, and not calibrated to a particular installation. The 0.15 Ω DC cable resistance assumes 100 m of 6 mm² copper cable, which is deemed reasonable at campus scale but not verified against real cable routes at the specific UBT Campus in Lipjan. Loss mechanisms are treated as independent sequential multipliers to the capacity factor loss. Reality differs in that soiling changes the thermal absorptance of the glass surface, affecting the cell temperature, which in turn affects the temperature coefficient in Equation (1).

Currently, the implementation does not include LSTM-based load forecasting. The sections for genetic algorithm/reinforcement learning optimization, 3D digital twin visualization, demand-response/electric vehicle charging, etc., are present in the overall platform [20] but have been left out of this paper in order to analyze monitoring-/IoT-related contributions in isolation.

The benchmarks in this post were run on a single machine, but with 10 containers (one writer and nine readers) all competing for the same single CPU core and single network stack. Running distributed on multiple hosts or in the cloud would add latency due to the networks involved, additional overhead from a container orchestration system like Kubernetes (as opposed to Docker Compose used here), and the synchronization of clocks on different machines that a localhost benchmark cannot test for. It remains to be seen whether the P99 stays below 100 ms in a distributed scenario.

Three further limitations are worth flagging explicitly. First, the nine loss mechanisms are applied as independent sequential multipliers, but in reality soiling alters glass transmittance, which alters cell temperature, which alters the temperature coefficient in Equation (1). Quantifying the error from this independence assumption is left to future work alongside the field calibration described in Section 5. Second, the latency numbers reported in Section 3 are from a single one-hour run, without confidence intervals or repeated trials. The variance characterization is left to future work. Third, the benchmark was run on a single workstation, so unpredictable network delays between hosts, clock skew across nodes, and container orchestration overhead are not represented; these will be quantified once the field deployment is in place.

5. Conclusions

We ran a test that pushed 5840 messages and confirmed a P50 latency of 27.2 ms with zero packet loss over an hour. This was all hosted on a single workstation comprising ten containerized services (a physics simulator, an MQTT broker, an async data-ingestion service, time-series database, an in-memory cache, and a REST/WebSocket gateway service) with a React dashboard.

Table 6 shows a comparative analysis with existing platforms. To the best of our knowledge, there is no single platform available in the literature that combines physics-based simulators, an IoT communication layer with packet loss statistics, real-time visualization, and benchmarking performance. OpenEMS, VOLTTRON, Ditto, ThingsBoard, pymgrid and pvlib address individual components. In our work, we integrated all four aspects under one roof and made it deployable with a single command with a Docker Compose file.

The paper does not include several aspects of the larger platform that it is based on. Most notably, the paper does not include connecting physical hardware sensors to gather real-time data, validating the loss model with field data, or testing the pipeline at scale (i.e., with hundreds of microgrids in the network) under adversarial network conditions. Additionally, three additional modules within the larger platform that are enabled by the presented pipeline (LSTM-based forecasting, GA+RL-based optimization, and 3D data visualization) are not demonstrated.

Regarding future work, we will incorporate into the real-time platform the 24 h load forecast from the LSTM network run on historical campus data with a 4 h look back window. This will enable the use of the forecast in the genetic and reinforcement-learning-based battery dispatch optimizers discussed in [20]. In parallel, we are also establishing physical Modbus RTU links to monitoring equipment at the UBT Campus in Lipjan, and will be starting a multi-week field trial to validate the loss model parameters. Details of this work will be presented in a forthcoming paper.