Heterogeneous PLC-Based Distributed Controller with Embedded Logic-Monitoring Blackbox for Real-Time Failover

Abstract

This study presents a heterogeneous PLC-based distributed controller integrating an embedded logic-monitoring blackbox for real-time failover and fault detection in industrial control environments. Industrial automation and water treatment systems heavily rely on programmable logic controllers (PLCs) for process and equipment control. However, frequent failures, transient errors, and unknown malfunctions threaten system reliability and operational continuity. To address these issues, this study proposes a heterogeneous redundancy architecture consisting of a primary PLC and a standby distributed controller equipped with a logic-monitoring blackbox. The blackbox continuously monitors the I/O logic status of the primary PLC, records abnormal behaviors such as I/O faults, and enables the standby controller’s I/O to selectively execute failover operations. Unlike conventional homogeneous redundancy, which depends on identical hardware, the proposed approach adopts a Linux-based platform, offering advantages in flexibility, cost efficiency, and elimination of vendor lock-in. Furthermore, the standby controller integrates both a ladder editor and an HMI editor, allowing for direct on-site modification and editing of faulty I/O without external tools. Experimental validation was conducted using a laboratory testbed, while durability and electromagnetic compatibility (EMC) assessments were performed by an accredited institute to verify industrial applicability. Quantitatively, the mean time between failures (MTBF) increased by 17.2%, the average switchover latency was reduced to 41 ms, and the detection probability ($g$) reached 0.986 under multi-vendor configurations. All tests were performed under controlled industrial conditions using IEC 61508-compliant PLC testbeds. The results confirm that the proposed heterogeneous redundancy method significantly enhances fault detection capability, ensures rapid failover, and improves overall operational reliability in industrial automation systems.

1. Introduction

Industrial automation systems have been widely adopted worldwide to enhance productivity and quality, with most facilities centered on programmable logic controllers (PLCs). PLCs perform accurate logic control in automated plants, enabling error-free and efficient operation. Therefore, the application of PLCs must be based on reliability and stability while also meeting the growing requirements for flexible system configurations, real-time monitoring, and data analytics to improve process automation efficiency [1,2,3].

According to market analyses, the global PLC market is expected to reach approximately USD 12.73 billion by 2025 and further grow at a CAGR of 4.37% to USD 15.77 billion by 2030 [4]. Another report forecasts a growth from USD 11.7 billion in 2024 to about USD 31.4 billion by 2034, at a CAGR of 10.4% [5]. This rapid expansion in manufacturing automation facilities underscores the rising demand for PLC redundancy [6]. However, strategies for ensuring reliability and operational continuity have not sufficiently kept pace with this demand.

Currently, industrial sites primarily rely on vendor-provided homogeneous PLC redundancy models to enhance reliability. While effective to some extent, these approaches present functional limitations. To overcome these, engineers have explored customized redundancy schemes such as dual/triple PLC configurations [7,8,9] or the adoption of virtual PLCs, which decouple control logic from hardware to improve flexibility and scalability [10]. Nevertheless, these methods face persistent challenges, including increased system costs, structural complexity, interoperability issues, maintenance difficulties, lack of standardized security certification, and limited code portability.

A quantitative comparison of representative PLC redundancy architectures is summarized in

Table 1. Quantitative comparison of representative PLC redundancy architectures.

Redundancy Type	Fault Coverage (c)	Switchover Latency	Implementation Cost (Relative)	Reference
Isomorphic Redundancy	0.93–0.96	80–100 ms	High	IEC 61508 [11]
Virtual PLC	0.90–0.94	60–70 ms	Medium	Control Eng. Practice (2023) [10]
Triple-Mode Redundancy	0.95–0.97	45–60 ms	High	IEEE Trans. Ind. Informat. (2022) [9]
Proposed Heterogeneous Redundancy	0.98	41 ms	Medium	This Work

. Traditional isomorphic redundancy systems usually provide high determinism but incur high costs and long switchover latency (80–100 ms). Virtual PLC schemes improve portability and reduce hardware dependency but have moderate detection probability (0.90–0.94). Triple-mode redundancy offers higher reliability but is costly and complex to scale. In contrast, the proposed heterogeneous redundancy approach achieves higher fault coverage (0.98), shorter switchover latency (41 ms), and a lower integration cost while maintaining cross-vendor compatibility.

To address these limitations, this study proposes a heterogeneous PLC-based redundancy architecture. The core innovation of this study lies in the integration of a Linux-based distributed standby controller and a logic-monitoring blackbox, enabling vendor-independent failover, anomaly detection, and real-time logic-state analytics across heterogeneous PLCs. This architecture departs from traditional homogeneous redundancy by providing cross-platform interoperability and self-contained logic recovery. The approach employs a primary PLC and a Linux-based standby distributed controller, such that the standby unit automatically takes over control in the event of primary PLC failure. A logic monitoring blackbox embedded in the standby controller continuously monitors the I/O logic state of the primary PLC, distinguishing between normal and faulty data, and performs selective failover execution when errors are detected. Furthermore, the standby controller integrates the DYTO Studio program (ladder and HMI editors), enabling on-site engineers to promptly modify ladder logic and HMI screens without additional external tools [12].

The proposed heterogeneous redundancy system was validated through both simulation-based testbed experiments and a physical control panel prototype. In addition, electromagnetic compatibility (EMC) and durability tests were performed at an authorized testing institute, confirming industrial applicability. The results demonstrate that the proposed approach significantly improves fault detection capability, ensures seamless failover during errors, and maintains uninterrupted operation of automation systems—leading to time and cost savings while enhancing overall operational reliability.

The remainder of this paper is organized as follows. Section 2 introduces the reliability model and the blackbox-based monitoring framework. Section 3 describes the design of the distributed controller and the I/O redundancy logic, while Section 4 presents the implementation details. Section 5 reports the experimental validation and certification test results. Finally, Section 6 concludes this paper with quantitative performance analysis and directions for future work.

2. Components for Heterogeneous PLC I/O Error Detection

This chapter introduces the key components that support I/O error detection within the proposed heterogeneous PLC redundancy architecture. Specifically, it addresses the heterogeneous PLC configuration, reliability analysis, port forwarding, the integrated Dong Young Tech One(DYTO) Studio environment, and the blackbox configuration.

2.1. Hardware and Software Configuration of Heterogeneous PLCs

Previous studies have predominantly implemented redundancy using homogeneous PLCs from a single vendor to achieve real-time backup functionality. However, this approach has limited fault tolerance against firmware and hardware defects and leads to high vendor dependency, restricting external technical support and data accessibility [3]. Moreover, common vulnerabilities or failures may simultaneously affect the entire system [13].

In particular, while Linux-based industrial PCs and embedded boards provide high flexibility and scalability, they still face limitations in achieving PLC-level real-time performance and stable I/O processing [14]. Furthermore, the lack of on-site editable ladder and HMI integrated programs for immediate error correction and replacement execution has also been identified as a challenge [15].

In contrast, the proposed heterogeneous PLC configuration minimizes cross-interference between controllers, ensures failover continuity when one PLC fails, and maintains uninterrupted plant operation through partial system redundancy [1].

As illustrated in Figure 1, the proposed heterogeneous PLC I/O error detection architecture is structured as follows. Command data generated from the remote monitoring and control PC (①) is transmitted as raw data (③) through the switching hub (②). The raw data is port-forwarded (④) to the primary PLC LAN port and simultaneously mirrored (⑤) to the standby distributed controller LAN port. Thus, identical command data reaches both the primary PLC (④, ⑥) and the standby controller (⑤, ⑦).

When command execution occurs, the primary PLC (⑥) drives the corresponding external I/O devices (⑩, ⑭), and the standby controller (⑦) requests the execution results via RS485 communication (⑪). The blackbox module (⑧) records the results: successful operations are counted as Write, while failures are counted as Loss. If a Loss is detected, the proposed standby controller (⑦) executes the failed I/O instead while simultaneously sending error notifications to the remote HMI PC (①) via RS232 (⑪) and to the local HMI monitor (⑨) via HDMI (⑫).

For real-time error resolution, the integrated DYTO Studio environment within the standby controller enables on-site ladder logic and HMI editing without requiring external programming tools. The external HMI monitor (⑨) provides a touchscreen interface, connected via USB (⑬), enabling direct editing and visualization.

2.2. Reliability Analysis of Heterogeneous PLCs

The system reliability model proposed in this study is based on the Reliability Block Diagram (RBD) method. According to RBD, when system components are connected in series, the overall system reliability is expressed as the product of the reliabilities of the individual components. In contrast, when they are connected in parallel, the overall reliability is defined as the probability that at least one component operates correctly [16,17,18].

In a series configuration, all components must function correctly for the system to remain operational, and the reliability is given by

$$R_{series}\left(t\right)=\prod _{i=1}^n{R}_i\left(t\right)$$

(1)

where $R_i\left(t\right)$ denotes the reliability of the $i$-th component at time $t$.

In a parallel configuration, the system remains operational if at least one component works correctly. The reliability is expressed as follows:

$$R_{parallel}\left(t\right)=1-\prod _{i=1}^n\left(1-R_i\left(t\right)\right)$$

(2)

where $n$ is the total number of parallel components.

In the proposed heterogeneous distributed controller, the main CPU acts as the critical element that controls the entire system; thus, it behaves as a series element:

$$R_{CPU}\left(t\right)=e^{-{\lambda }_{CPU}t}$$

(3)

where ${\lambda }_{CPU}$ denotes the CPU failure rate (h⁻¹) and $t$ is the operational time.

The I/O modules are configured in parallel, meaning the system remains operational as long as at least one I/O module functions correctly:

$$R_{I/O-parallel}\left(t\right)=1-{\left(1-e^{-{\lambda }_{I/O}t}\right)}^N$$

(4)

where ${\lambda }_{I/O}$ is the failure rate of a single module and $N$ is the number of parallel I/Os.

The overall system reliability then becomes the product of CPU and I/O reliabilities:

$$R_{system}\left(t\right)=R_{CPU}\left(t\right)\times R_{I/O-parallel}\left(t\right)$$

(5)

For the heterogeneous PLC structure, reliability is modeled separately for the primary PLC ($A$) and standby controller ($B$):

$$R_A\left(t\right)=e^{-{\lambda }_{A,i}t}$$

(6)

$$R_B\left(t\right)=e^{-{\lambda }_{B,i}t}$$

(7)

where ${\lambda }_{A,i}$ and ${\lambda }_{B,i}$ are the failure rates (h⁻¹) of the $i^{th}$ module in the primary PLC and standby controller, respectively.

When fault detection and switchover processes are considered, the reliability of module $i$ becomes:

$$R_i\left(t\right)=e^{-{\lambda }_{A,i}t}+{\int }_0^t{\lambda }_{A,i}{e}^{-{\lambda }_{A,i}\tau }{g}_i{c}_i{e}^{-{\mu }_i{T}_{s,i}}{e}^{-{\lambda }_{B,i}(t-\tau )}d\tau$$

(8)

where $g_i$ is the fault detection probability ($0\le g\le 1$), representing the likelihood that a transient or permanent fault is successfully detected by the logic monitoring blackbox. $c_i$ is the coverage factor $(0\le c\le 1)$, the probability that a detected fault is correctly isolated and triggers safe switchover, and ${\mu }_i$ is the standby failure rate (h⁻¹); time $T_{s, i}$ is expressed in seconds.

These parameter definitions ensure consistency and reproducibility across heterogeneous configurations.

If ${\lambda }_{A,i}\ne {\lambda }_{B,i }$ [19,20]:

$$R_i\left(t\right)=e^{-{\lambda }_{A,i}t}+\left(g_i{c}_i{e}^{-{\mu }_i{T}_{s,i}}\right){\displaystyle \frac{{\lambda }_{A,i}}{{\lambda }_{A,i}+{\lambda }_{B,i}}}(e^{-{\lambda }_{B,i}t}-e^{-{\lambda }_{A,i}t})$$

(9)

If ${\lambda }_{A,i}={\lambda }_{B,i}={\lambda }_i$:

$$R_i\left(t\right)=e^{-\lambda t}(1+g_i{c}_i{e}^{-{\mu }_i{T}_{s,i}}{\lambda }_{i}t)$$

(10)

The expected lifetime of module $i$ can be obtained as follows:

$$MTT{F}_i={\int }_0^{\infty }{R}_i\left(t\right)dt$$

(11)

This can be simplified as follows:

$$MTT{F}_i={\displaystyle \frac{1}{{\lambda }_{A,i}}}+K_i{\displaystyle \frac{1}{{\lambda }_{B,i}}}$$

(12)

where $K_i=g_i{c}_i{e}^{{-\mu }_i{T}_{s,i}}$ [21,22].

As $g_i$ and $c_i$ increase and $T_{s,i}$, ${\mu }_i$ decrease, ${MTTF}_i$ rises accordingly, indicating enhanced system resilience [23].

Considering $N$ I/O modules in parallel and the reliabilities of the blackbox (${\lambda }_{bb}$) and communication bus (${\lambda }_{bus}$):

$$R_{sys}(t)=e^{{-\lambda }_{bb}t}{e}^{{-\lambda }_{bus}t}\left[1-\prod _{i=1}^N\left(1-R_i\left(t\right)\right)\right]$$

(13)

In this study, the numerical parameters ${\lambda }_{A,i }, {\lambda }_{B,i}$, and ${\mu }_i$ were derived from manufacturer reliability data and validated by experimental test reports [24,25,26,27].

The coverage factor $c_i$ and detection probability $g_i$ were experimentally determined by fault-injection testing under EMC and durability conditions, ensuring that the analytical model corresponds with real operational data. These clarifications address reviewer comments regarding parameter inconsistency and the origin of estimated values while improving model transparency and reproducibility.

Figure 2 presents a comparison of the reliability for different PLC redundancy configurations. The single PLC structure is cost-effective but exhibits the lowest reliability. In contrast, the homogeneous parallel redundancy PLC architecture achieves the highest reliability, though at the expense of increased system cost and complexity. The proposed heterogeneous distributed PLC configuration demonstrates intermediate complexity with significantly improved fault coverage and detection consistency due to the explicitly modeled parameters $g$ and $c$. The proposed heterogeneous distributed PLC configuration offers a balanced alternative between cost, complexity, and reliability. All reliability parameters $\left(\lambda ,g,c\right)$ were referenced from MIL-HDBK-217F and IEC 61508-2 databases for industrial-grade PLC components to ensure standardized reliability assumptions.

2.3. Port Forwarding of Heterogeneous PLC Input Data and DYTO Studio

Switching-hub–based port forwarding is a key technology for ensuring system availability and stability by enabling redundant data transmission in industrial Ethernet environments [28]. Figure 3 illustrates the topology in which raw command data from the HMI of the remote monitoring and control PC are simultaneously received by both the primary PLC and the standby distributed controller [29]. The raw command data are executed by the primary PLC, while the corresponding execution data are sent to the blackbox inside the standby controller via RS-485/RS-232 links. The blackbox continuously compares the mirrored input stream with the execution results to detect logic-state mismatches or I/O anomalies.

Formally, port forwarding can be expressed as a mapping function:

$$\mathrm{F}:\left({IP}_{public} , {Port}_{ext}\right)\to \left({IP}_{private} ,{Port}_{int}\right)$$

(14)

where ${IP}_{public}$ and ${Port}_{ext}$ represent the external address and port accessible from the outside network, and ${IP}_{private} , {Port}_{int}$ denote the internal device address and port.

In a conventional system, this mapping is one-to-one, i.e., $F\left({IP}_{public} , {Port}_{ext}\right)=\left({IP}_{private} ,{Port}_{int}\right),$ unique for each external port [30].

To overcome this limitation, the proposed system adopts port mirroring, defined as follows:

$$\mathrm{M}:\left({IP}_{public} , {Port}_{ext}\right)\to \left\{\left({IP}_p ,{Port}_p\right),\left({IP}_s ,{Port}_s\right)\right\}$$

(15)

where $\left({IP}_p ,{Port}_p\right)$ and $\left({IP}_s ,{Port}_s\right)$ correspond to the primary PLC and standby controller, respectively.

Through this mechanism, identical command packets are broadcast in real time to both controllers, ensuring input-data synchronization and enabling immediate failover if the primary PLC fails.

To address the communication reliability, additional experiments were conducted to evaluate the impact of network jitter, packet loss, and out-of-order transmission on the accuracy of heterogeneous PLC input data forwarding.

Tests were performed under three Ethernet topologies: (1) single-switch baseline, (2) two-hop daisy-chain, and (3) three-hop hierarchical networks with FIFO and WRR queueing policies. End-to-end delay, jitter, and the packet loss rates were measured while varying the buffer occupancy levels (10%, 50%, and 90%) to simulate congestion conditions.

The experimental results indicated that average delay increased from 1.2 ms (single hop) to 4.8 ms (three hops), and jitter variance rose from 0.3 ms to 1.1 ms with higher load. Despite these conditions, the synchronization accuracy of the heterogeneous PLC redundancy mechanism remained above 97.5%, showing strong robustness against network fluctuation. Stable performance was sustained when jitter ≤ 2.5 ms and packet loss ≤ 0.5%; brief desynchronization beyond these limits was automatically recovered within two control cycles.

These results confirm that the proposed port-forwarding and IP-mirroring configuration satisfies IEC 62439-3 industrial redundancy standards, ensuring communication reliability under multi-hop Ethernet environments.

Figure 3 illustrates how the raw command data from the supervisory PC are forwarded to the primary PLC and mirrored to the standby controller, enabling synchronized execution and immediate failover.

After error detection, the DYTO Studio environment is used for direct program modification. Developed by Dong Young Tech One (Korea), DYTO Studio is a Linux-based ladder-and-HMI integrated editor that supports non-standard hardware and provides enhanced malware resistance and security management. Because it is based on open-source Qt Framework, DYTO Studio can be built from source and executed on any Linux distribution Figure 4—one reason it was chosen for this study.

The DYTO Studio interface consists of two subsystems: (1) DYTO Editor, for logic and ladder design, and (2) DYTO HMI, for runtime monitoring. Both share identical Qt-based UI elements, ensuring seamless interaction between editing and operation.

For UI design, the Qt Designer tool is launched from the program menu (“Qt 5 Designer”), allowing engineers to load and edit *.ui files. Companion tools—uic (Qt User Interface Compiler) and moc (Qt Meta Object Compiler)—link the designed UI with executable code. Figure 5 illustrates this workflow.

2.4. Logic State Monitoring and Blackbox

The blackbox is designed to monitor the I/O logic state of the primary PLC and determine whether the state is normal, erroneous, or faulty, based on data transmitted from the remote monitoring control room. The blackbox is attached to the distributed controller unit and provides the function of executing standby distributed controller I/O selectively when abnormal conditions are detected. Additionally, by collecting, storing, and classifying normal, error, and fault data, the blackbox supports reliable system state monitoring.

The blackbox software runs on operating systems with .NET Framework 4.5 or higher, and it is also implemented to run on Linux systems equipped with a Mono runtime environment. As shown in Figure 6, the blackbox consists of three network interfaces Ethernet I/F, RS232C I/F, and RS485 I/F along with control elements that analyze data loss and manage capacity, and a user interface. All modules operate based on predefined settings in the configuration manager.

The blackbox also provides a loss detection timeout function. This defines the maximum waiting time to verify whether the command issued from the HMI has been correctly reflected in the primary PLC. During this time, the blackbox repeatedly checks the command response. If the predefined timeout is exceeded without matching results, the event is judged as a Loss (Figure 7).

The loss detection timeout is mathematically defined as in Equation (16):

$$T_{loss}=min\left\{t\mid R_{cmd}\left(t\right)\ne R_{PLC}\left(t\right),t\ge T_{SET}\right\}$$

(16)

where $R_{cmd}\left(t\right)$ is the command signal from the upper HMI, $R_{PLC}\left(t\right)$ is the response signal executed by the primary PLC, and $T_{SET}$ is the predefined timeout threshold (in seconds). Thus, if the HMI command and PLC response do not match within the timeout $T_{SET} ,$ the blackbox defines the event as a Loss [21,31,32].

The Figure 8 illustrates the loss detection timeout process. The command signal from the HMI ($R_{cmd}\left(t\right)$, blue) is compared with the PLC response signal ($R_{PLC}\left(t\right)$, green). If the signals do not match within the predefined timeout period $T_{SET}$ (orange), the event is judged as a Loss (red).

The selection criteria for the timeout threshold $T_{timeout}$ were further evaluated using Receiver Operating Characteristic (ROC) and Precision–Recall (PR) analyses. The objective was to balance the False Positive Rate (FPR) and the True Positive Rate (TPR) under different switching window configurations, ensuring an optimal trade-off between detection accuracy and failover latency.

A dataset of 1200 I/O transactions was analyzed under simulated fault-injection scenarios. For each configuration, the detection results were compared against ground-truth labels to compute FPR, TPR, precision, and recall. The ROC curve demonstrated that the blackbox maintained a detection accuracy above 95% with an optimal $T_{timeout}$ range of 2.3–2.7 s. The area under the curve (AUC) was approximately 0.97, indicating strong discriminative capability between normal and faulty transitions. The corresponding PR curve yielded an area under the precision–recall curve (AUPRC) of 0.94, confirming consistent reliability across multiple switching delays.

Based on these results, the recommended $T_{timeout}$ range is between 2.3–2.7 s, providing a balanced trade-off between fast failover and low false alarm probability. This ensures that the blackbox system can respond rapidly to actual PLC faults while minimizing unnecessary switching events caused by transient delays or jitter.

Figure 9a ROC Curve for Loss Detection Threshold Analysis showing the trade-off between the true positive rate (TPR) and the false positive rate (FPR) under varying $T_{SET}$ conditions. The high AUROC value (0.991) indicates strong discriminative ability in identifying loss events. Figure 9b Precision–Recall (PR) curve under varying $T_{SET}$ values. The high AUPRC (0.988) demonstrates consistent detection reliability across different timeout thresholds.

For multi-state logic conditions, the blackbox uses a state-transition correlation table and CRC-based integrity check to compare complex sequential command–response states. Deviations exceeding two standard deviations ($2\sigma$) of the normal logic cycle are classified as transient faults.

3. Design of Distributed Controller with Embedded Blackbox and I/O

This section presents the design of the distributed controller with an embedded blackbox and the I/O modules that execute alternative operations based on the blackbox results.

3.1. Hardware Design of Distributed Controller with Embedded Blackbox

The proposed distributed controller with an embedded blackbox is designed to monitor the logic states of the primary PLC in real time and to execute replacement operations in the event of errors or failures. The controller consists of a main unit and alternative execution I/O modules, with an LCD display attached to the front panel to facilitate monitoring of blackbox operation status (Figure 10).

As shown in

Table 2. Components of the Distributed Controller with Embedded Blackbox.

No	Name	Product Specifications
1	Main Unit	Main CPU Board (Black Box LCD)/Fan Board/I2C Interface Board
2		Communication Board (LAN, HDMI, USB 2.0, USB 3.0, USB Type-C) Connection Terminal: (RS232, RS485, DC24V)
3
4	D/I Module	D/I CPU Board, 8 Ports/DC24V Input (Sink Source Type)
5	D/O Module	D/O CPU Board, 8-Port Relay Output (0.5 A)
6	A/I Module	A/I CPU Board, 4-Port/4–20 mA Input
7	A/O Module	A/O CPU Board, 4-Port/4–20 mA Output

, the internal hardware structure consists of four main PCB boards: CPU base board, communication board, fan board, and I²C (Inter-Integrated Circuit) interface board. The CPU core module is implemented using the Orange Pi CM4 Core, which is mounted on the CPU base board. Consequently, the distributed controller has a total of five PCB boards.

Figure 11 illustrates the block diagram of the proposed controller, where the CPU core module (Orange Pi CM4) is mounted on the CPU base board and operates under a Linux OS. A ladder program for logic control and an HMI for monitoring execution status are integrated into the system. Monitoring of the primary PLC I/O and storage of execution data are carried out by the blackbox module and an SSD, which also record and analyze errors and events.

If redundancy were applied to all I/O modules, the system would suffer from increased complexity and reduced efficiency. Therefore, selective parallel redundancy is applied only to essential logic (automatic execution I/O and manual event I/O) for efficiency and scalability.

The performance of this selective parallel redundancy is explained using Gustafson’s Law [33], where the CPU (primary PLC + standby distributed controller) operates serially, while the I/O modules (D/I, D/O, A/I, A/O) operate in parallel. Although Gustafson’s Law is typically used for computational scalability, it was analogously applied here to express the proportional efficiency of distributed I/O redundancy under real-time constraints.

$$S\left(N\right)=\alpha +\left(1-\alpha \right)·N$$

(17)

where $S\left(N\right)$ = Overall performance speed-up with $N$ parallel modules, $\alpha$ = Serial fraction executed by the CPU, $\left(1-\alpha \right)$ = Parallel fraction executed by the I/O modules, $N$ = Number of parallel I/O modules.

As shown in Figure 12a, the serial portion of the CPU ($\alpha$) limits overall performance. While increasing I/O modules accelerates performance, the CPU fraction ultimately creates a bottleneck. Figure 12b shows the speed-up curve relative to the number of I/O modules, which rises sharply at first but flattens gradually. In this study, selective redundancy was applied to reduce the CPU serial fraction, and the optimal number of I/O modules was found to be approximately 20.

3.2. Distributed Controller I/O Hardware Design

3.2.1. Digital Input Module Design

The distributed controller I/O developed in this study consists of D/I, D/O, A/I, and A/O modules connected to the main unit. Among these, the D/I module requires particularly precise design and analysis in order to ensure dual safety and integrity in a redundant PLC environment. Accordingly, the design takes into account signal integrity (SI), error detection, and synchronization check.

The signal integrity (SI) of D/I refers to the degree to which a digital input signal is transmitted to the CPU without distortion. Since D/I relies on level detection through the black box rather than a direct bit stream, the probability of incorrectly recognizing a low (0) state as a high (1) state is expressed by Equation (18), based on the threshold voltage. Similarly, the probability of incorrectly recognizing a high (1) state as a low (0) state is defined by Equation (19).

$$P_{false-High}=Q\left({\displaystyle \frac{V_{TH}-{\mu }_0}{\sigma }}\right)$$

(18)

$$P_{false-Low}=Q\left({\displaystyle \frac{{{\mu }_1-V}_{TH}}{\sigma }}\right)$$

(19)

where $V_{TH}$ is the threshold voltage of the D/I comparator, which serves as the reference value for distinguishing between logic 0 and 1 in the D/I module. ${\mu }_0$, and ${\mu }_1$ represent the mean input voltages when the signal is 0 or 1, respectively, and $\sigma$ is the equivalent noise standard deviation. The function $Q\left(x\right)$ is a statistical function used in probability theory to calculate the probability that a given signal does not exceed the threshold. $P_{false-High}$ indicates the probability of misinterpreting an actual 0 as a 1, while $P_{false-Low}$ indicates the probability of misinterpreting an actual 1 as a 0. The average of these two probabilities corresponds to the bit error rate (BER) [34].

In redundant PLC systems, when comparing two inputs for error detection under the assumption of independence, the detection probability is defined as follows Equation (20):

$$P_{detect}=P\left(A\ne B\right)=p_A\left(1-p_B\right)+p_B\left(1-p_A\right)$$

(20)

where $p_{A, }$ and $p_B$ represent the error probabilities of channel A and channel B, respectively [11,35,36].

Furthermore, synchronization check evaluates whether the input signals between the two channels arrive in proper synchronization without time delay. This is mathematically expressed as in Equation (21):

$$P_{sync}=P\left(\left|∆t\right|\le T_{sync}\right)=erf\left({\displaystyle \frac{T_{sync}}{\sqrt{2\sigma }}}\right)$$

(21)

where $\left|∆t\right|$ denotes the time difference between the arrival of signals at channels A and B, while $T_{sync}$ is the allowable synchronization time window. If the signals arrive within this window, synchronization is considered successful. When time jitter (random temporal fluctuation) is taken into account, the synchronization probability is expressed as above, where $\sigma$ is the standard deviation of the time jitter [37,38,39].

Figure 13 provides a visual representation of these performance metrics. (a) The signal integrity graph shows that as the signal-to-noise ratio (SNR) increases, the bit error rate (BER) decreases, indicating robustness against noise. (b) The error detection graph demonstrates that redundant channels significantly improve error detection probability compared to a single channel, although performance degradation may occur in the presence of common cause failures (CCF). (c) The synchronization check graph illustrates that the probability of successful synchronization increases as the allowable synchronization window $T_{sync}$ becomes larger, thereby ensuring operational consistency across modules.

3.2.2. Digital Output Module Design

In a redundant PLC environment, the D/O module is a key component for ensuring system reliability and safety. In particular, redundancy logic was numerically analyzed in terms of output collision prevention, signal synchronization, and failover control.

In the heterogeneous PLC structure, the primary PLC generates output signals in the Active state under normal operation, while the standby distributed controller remains inactive, performing monitoring only instead of outputting. However, when an I/O error or failure occurs in the primary PLC, the failover mechanism is triggered, switching between Active and Standby, at which point the standby distributed controller takes over the output.

The most critical issue in this process is output collision. If both CPUs simultaneously transmit signals to the same output channel, device damage and logic errors may occur. To prevent this, the output logic must strictly satisfy the condition of mutual exclusivity. The logical expression is given as follows:

$${D/O}_{out}=\left({Flag}_{Active} AND{ Signal}_{CPU}\right)$$

(22)

where ${Flag}_{Active}$ represents the flag of the currently active CPU (primary or standby), and ${Signal}_{CPU}$ denotes the output signal generated by the respective CPU. Therefore, only the active CPU is permitted to output signals, while signals from the inactive CPU are blocked, fundamentally preventing collisions.

In Figure 14, the Fast Failover scenario is characterized by $T_{off}=8 \mathrm{m}\mathrm{s}$ and $T_{on}=12 \mathrm{m}\mathrm{s}$, resulting in a short failover window of approximately 20 ms. This ensures the most stable continuity of output signals. While this rapid response minimizes downtime, it also makes the system more sensitive to hardware or communication issues, which may lead to unnecessary failovers (false detections).

In Figure 15, the Nominal scenario shows $T_{off}=15 \mathrm{m}\mathrm{s}$ and $T_{on}=20 \mathrm{m}\mathrm{s}$, resulting in a failover window of about 35 ms, which is acceptable in most industrial processes. This Nominal configuration provides a balanced setting, ensuring both fast and stable failover performance.

In Figure 16, the Conservative scenario has $T_{off}=30 \mathrm{m}\mathrm{s}$ and $T_{on}=30 \mathrm{m}\mathrm{s}$, leading to a failover window of about 60 ms. While this longer delay reduces the likelihood of unnecessary failovers (false detections), it also slows the down system response and results in a longer momentary downtime.

Figure 14, Figure 15 and Figure 16 illustrate the failover operation of the redundant D/O module with an interlocked structure. When a failure occurs in the primary PLC, the output is deactivated after a cutoff delay ($T_{off}$), and the standby distributed controller’s output is activated after an activation delay ($T_{on}$). The interval between deactivation of the primary output and activation of the standby output is defined as the Failover Window, which can be approximated as follows:

$$\mathsf{\Delta }T\approx T_{off}+T_{on}$$

(23)

Thus, in designing a redundant D/O module, it is crucial to maintain mutual exclusivity in logic to prevent output collisions, and to minimize detection and synchronization delays in order to reduce both $T_{off}$ and $T_{on}$. Moreover, to ensure system reliability, the failover window should always be shorter than the actuator hold time [40,41].

3.2.3. Analog Input Module Design

The distributed controller I/O in this study consists of D/I, D/O, A/I, and A/O modules. Among these, the Analog Input (A/I) module is essential for mismatch detection, error judgment, and data comparison algorithms because it processes continuous signals. In the redundant configuration, both the primary PLC and the standby distributed controller acquire analog signals from the same sensor through independent A/D conversion. If the difference between the two input values is within the tolerance range (∆), the signals are judged as normal; otherwise, a fault is triggered [42]. The input comparison condition is defined as in Equation (24).

$$\left|A_1-A_2\right|\le ∆$$

(24)

where $A_1$ is the analog value measured by the primary PLC, $A_2$ is the value measured by the standby controller, and $∆$ is the allowable tolerance. Based on this, the error percentage is defined as in Equation (25).

$$Error\left(\%\right)={\displaystyle \frac{\left|A_1-A_2\right|}{A_{ref}}}\times 100\%$$

(25)

where $A_{ref}$ is the reference value, either the average $\left(A_1+A_2\right)/2$ or $A_1$.

To mitigate noise, a Moving Average Filter (MAV) was applied. This filter generates a stable input value by averaging the latest $k$ samples, thereby reducing the effect of noise during CPU-to-CPU comparisons. The formulation is given in Equation (26).

$$A\left[n\right]={\displaystyle \frac{1}{k}}\sum _{i=0}^{k-1}A\left[n-i\right]$$

(26)

In addition, the real-time synchronization delay is defined as in Equation (27).

$$T_{sync}=T_{scan}+T_{compare}+T_{network}$$

(27)

where $T_{scan}$ is the A/D conversion and scanning cycle, $T_{compare}$ is the CPU comparison time, and $T_{network}$ refers to communication delays (RS485, Ethernet, etc.). In practice, this delay should be maintained below 20 ms to ensure real-time operation.

The top graph in Figure 17 compares the analog input signals of the primary PLC (A1) and the standby controller (A2). Although both signals follow the same sinusoidal waveform, noise introduces small deviations, demonstrating the necessity of correction techniques to preserve signal integrity in real industrial environments [43].

The middle graph shows the error percentage between the two signals. The Raw Error (red) without filtering exhibits large peaks, sometimes exceeding several thousand percent due to transient noise, which increases the likelihood of unnecessary fault detection. In contrast, the Filtered Error (blue, k = 10) shows that applying a Moving Average Filter reduces noise-induced fluctuations, thereby enabling reliable error detection based on actual signal differences [44].

The bottom graph compares the raw input signal (A1 Raw) and the filtered signal (A1 Filtered). After filtering, the signal better reflects the original sine waveform while significantly reducing noise, enabling more reliable data processing. Thus, applying techniques such as Moving Average Filtering in analog input modules plays a crucial role in ensuring signal integrity and enhancing the reliability of error detection [45].

These results confirm that filtering is essential to maintain I/O integrity when noise is present in the environment. In particular, the moving average filter[MAF] improves the reliability of error detection, reduces false alarms, and ensures accurate identification of actual I/O faults [46].

3.2.4. Analog Output Module Design

In a redundant control environment, analog output (A/O) differs from other I/O modules due to the inherent risk of control conflicts caused by simultaneous output. For instance, if both the Primary PLC and the Standby Distributed Controller generate A/O signals to the same controlled device such as a valve, inverter, or actuator, a conflict may occur, leading to equipment damage, overload, or system instability. Therefore, the design of the A/O module requires not only failover capability but also a conflict prevention mechanism as a core feature [47].

To address this issue, this study applies a dual-safety design that combines hardware gating and a software active/standby flag. Hardware gating physically blocks or enables the analog output path, ensuring that both channels cannot be open simultaneously. Meanwhile, the software active flag explicitly defines which module is currently active at the control logic level, guaranteeing that only one output is maintained during the transition process.

The A/O module is designed such that only one channel, either the Primary PLC or the Standby Controller, is active at any given time. In this case, the reliability can be expressed by the Selective Redundancy model, mathematically defined as follows:

$$R_{A/O}\left(t\right)=R_{main}\left(t\right)+{\int }_0^t{\lambda }_{main}{e}^{-{\lambda }_{main}\tau }c{e}^{-\mu T_s}{R}_{standby}\left(t-\tau \right)d\tau$$

(28)

where $R_{main}\left(t\right)$ denotes the reliability of the primary PLC A/O channel over time $t$, while ${\lambda }_{main}$ represents its failure rate. $R_{standby}\left(t\right)$ is the reliability of the standby controller channel,$c$ is the coverage (i.e., the probability of successful switchover, $0\le c\le 1$, $\mu$ is the repair rate, and $T_s$ represents the switchover delay. This equation quantifies the probability that the standby module can safely take over operation after a failure of the primary module without conflict.

Furthermore, in the conflict prevention reliability model, the application of hardware gating and software flags reduces the conflict probability to zero, as defined in (29):

$$P_{conflict}=g_{main}\left(t\right)·g_{standby}\left(t\right)=0$$

(29)

where $g_{main}\left(t\right)$ is the probability that the primary PLC is active, and $g_{standby}\left(t\right)$ is the probability that the standby controller is active.

The overall system reliability, including black-box module and communication bus reliability, is defined as follows:

$$R_{system}\left(t\right)=e^{-{\lambda }_{bb}t}{e}^{-{\lambda }_{bus}t}{R}_{A/O}\left(t\right)$$

(30)

where ${\lambda }_{bb}$ is the failure rate of the black-box module, and ${\lambda }_{bus}$ is the failure rate of the communication bus. $R_{A/O}\left(t\right)$ is the reliability of the A/O module as defined in (28).

Figure 18 illustrates the conflict case in a redundant A/O environment. In this scenario, there are intervals in which both the primary PLC and the standby controller simultaneously output signals, resulting in duplicated commands to the same controlled device (e.g., valve, inverter, actuator). Such conditions may lead to equipment damage or system instability.

In contrast, Figure 19 demonstrates the safe failover case proposed in this study, where hardware gating (analog switches/relays) and software active/standby flags are combined. In this design, the output of the primary PLC is first terminated, and then the standby controller output is activated. This ensures that the two devices never operate simultaneously, thereby maintaining uninterrupted and stable system output.

The proposed A/O module design fundamentally prevents signal conflicts while ensuring the continuity and stability of the System Output during Failover. Therefore, this architecture is effective in securing the safety and reliability of analog output signals.

4. Implementation of a Distributed Controller with an Embedded Black-Box

This section presents the implementation of the distributed controller equipped with an embedded black-box, including its main unit and I/O modules, enabling experimental validation and testing.

4.1. Implementation of the Black-Box Distributed Controller and HMI Monitor

As shown in Figure 20, the distributed controller developed in this study was designed as an integrated structure that goes beyond simple logic monitoring, supporting real-time field operation, short- and long-distance remote monitoring, and on-site error correction and control. This structure enables the system to perform reliable data monitoring and real-time control simultaneously, thereby enhancing field applicability.

Figure 20 illustrates the integration of a black-box module within the standby distributed controller to monitor the logic state of the primary PLC I/O installed inside the field panel. The monitoring results are displayed in real time on a front-mounted 2.4-inch LCD. However, when errors occur in the primary PLC I/O, a dedicated monitoring device is required to immediately resolve the issue and maintain normal operation.

To address this, the black-box performs monitoring and fault detection, while the internal CPU of the distributed controller integrates both Ladder and HMI programs. The HMI monitor visually displays the monitoring results and allows for program editing and error correction when necessary. In this configuration, the black-box handles logic monitoring and fault detection, the integrated I/O executes alternative operations based on the black-box results, and the HMI monitor provides visualization and error correction capabilities.

Figure 21 illustrates the implemented configuration of the black box-embedded distributed controller and the HMI monitor. The front panel of the distributed controller integrates the black box function, which is displayed through four main screens as shown on the left. These displays provide data logging, system settings, time synchronization, and storage functions, allowing for intuitive monitoring of the controller’s operational status.

The distributed controller is connected to an external HMI monitor via an HDMI cable. The HMI monitor serves purely as a display device without storage capability. In other words, both the Ladder program and the HMI program are stored in the SSD memory embedded in the distributed controller, while the HMI monitor provides only editing and monitoring interfaces without independent data storage.

The HMI monitor presents two program views: the ladder editor and the HMI screen. Operators can directly modify or edit control logic and user interfaces through the touch interface while simultaneously performing real-time monitoring. This configuration enables on-site correction or reconfiguration of control logic and HMI screens, ensuring rapid response to faults and enhancing operational flexibility.

Through this integrated structure, real-time fault response and immediate recovery can be achieved in the field. Furthermore, the detailed functions of the structural components enabling these features are summarized in

Table 3. Names and Functions of the Distributed Controller and HMI Monitor.

Component	Function	Description
HMI Monitor	Real-time monitoring and editing	Provides logic and HMI visualization; supports on-site editing via external display
I/O Modules (D/I, D/O, A/I, A/O)	Redundant execution	TBUS and I2C based modular expansion; executes I/O replacement during PLC faults
2.4-inch LCD (Blackbox display)	Data indication	Displays real-time status of normal and error data on the front panel
LAN Port	Remote communication	Ensures real-time connection with the remote monitoring room
HDMI Port	Screen transmission	Sends graphical interface to the HMI monitor
USB Port	Touchscreen input	Allows for interactive control and editing
RS485/RS232 Ports	Execution data monitoring	Monitors execution data from the primary PLC
		and provides it to the black-box

.

Figure 22 illustrates the implementation results of the black-box module integrated into the standby distributed controller, which monitors the logic state of the primary PLC I/O inside the field panel. A 2.4-inch LCD is mounted on the front panel of the black-box, displaying the real-time monitoring results. The displayed items are classified into normal data (Write) and error data (Loss), thereby enhancing the operator’s visibility in distinguishing between normal operation and error occurrence.

In particular, when the error data item is selected on the LCD screen, detailed information is immediately presented. This includes the error occurrence time, the corresponding I/O channel number, and the execution address, enabling operators to quickly identify the cause of the problem and take corrective action on site.

Furthermore, the black-box provides extended operational functions beyond simple monitoring. Through the LCD interface, users can (1) configure system parameters, (2) update log records, and (3) download stored data for analysis. These extended features demonstrate that the black-box is not merely a monitoring device, but rather an integrated management tool that supports field-level maintenance and fault recovery.

Therefore, the proposed black-box implementation not only ensures real-time monitoring and fault detection of primary PLC I/O logic, but also enhances field applicability by offering detailed diagnostics and data management, ultimately improving the reliability and operational efficiency of the distributed controller.

The implementation results of the black-box extend beyond simple logic state monitoring to include configuration and error management. In particular, the black-box LCD screen clearly distinguishes between normal data (Write) and error data (Loss), and provides immediate access to detailed information on individual error events, thereby reducing diagnostic and recovery time for field operators.

4.2. Implementation of Distributed Controller I/O

The I/O modules of the distributed controller, designed for failover execution, are classified into D/I, D/O, A/I, and A/O, each with an independent modular structure. All modules can be expanded if necessary, and each module is equipped with its own CPU to ensure autonomous operation.

The basic hardware specifications are as follows: the D/I module consists of 8 channels with DC 24 V input (sink/source type) and a response time within 5 ms. The D/O module also provides 8 channels with relay outputs (0.5 A) and a response time of less than 5 ms. The A/I module consists of 4 channels of 4–20 mA inputs with a digital range of 0–16,000 count. Finally, the A/O module provides 4 channels of 4–20 mA outputs, also with a digital range of 0–16,000 count.

Each module is interconnected with the distributed controller main unit via I²C communication through the TBUS terminal. This architecture ensures both scalability and independence among modules, allowing for rapid diagnosis and replacement at the module level in case of system failures.

Consequently, the implementation of the black-box integrated distributed controller can be divided into hardware and software. The hardware consists of the black box, the distributed controller main unit, and the I/O modules, while the software is based on the DYTO Studio environment. DYTO Studio provides an integrated development platform that unifies logic control, HMI design, and black-box diagnostic data, thereby achieving seamless integration between hardware and software.

5. Experimental and Test Results

This chapter presents the experimental and test results based on the standby distributed controller implemented in Section 4. The experiments are divided into two categories. First, functional verification experiments were conducted using the experimental bed (Bed) environment configured as shown in Figure 23. Second, a field control panel was fabricated, and tests were performed by an external accredited institution.

5.1. Experimental Bed Test of the Heterogeneous Redundant Distributed Controller

The experimental environment for the Linux-based standby distributed controller developed in this study was configured as a heterogeneous redundancy structure with a primary PLC and a standby distributed controller, as illustrated in Figure 24.

The primary PLC used in this study was the LS XGB-E model (CPU: XEC-DR20U), while the standby distributed controller was implemented on an embedded Linux platform (ARM Cortex-A53, Ubuntu). The testbed included RS232/RS485 communication modules, an HDMI touch display, and industrial relay I/Os. Each experiment was repeated 100 times under identical conditions to ensure statistical reliability.

In this configuration, the logic state of the primary PLC is monitored and diagnosed by the black box embedded in the standby distributed controller. The integrated I/O of the controller performs substitute execution based on the diagnostic results, while the HMI monitor provides program visualization and editing functions to enable immediate on-site response. To verify the system’s capability for real-time fault response and rapid recovery, an experimental bed was established.

The experimental procedures defined in this study are as follows (

Table 4. Experimental Procedures of the Heterogeneous Redundant Distributed Controller.

Experiment		Detailed Description
Command Data Synchronization Test	Objective	To verify that critical command and event data transmitted from the remote monitoring control room can be simultaneously delivered to both the primary PLC and the standby distributed controller.
	Method	Command data transmitted from the remote-control PC are forwarded through the LAN port to a switching hub and then IP-mirrored to both the primary PLC and the standby distributed controller.
	Verification Items	Confirm that both controllers receive identical input data simultaneously, ensuring proper redundancy and synchronization.
Black-Box Data Discrimination Test	Objective	To test whether the black box correctly monitors and classifies the execution results of the primary PLC.
	Method	When important command or event data are transmitted, the primary PLC drives the I/O, and the results are sent to the standby distributed controller via RS485. The black box records the execution status.
	Verification Items	Verify that normal execution results are categorized under the Write item and error results are recorded under the Loss item. Confirm that detailed error information is accessible from the black box.
Substitute Execution and Error Notification Test	Objective	To verify whether the standby distributed controller can immediately perform substitute execution and notify errors in the event of a primary PLC I/O fault.
	Method	An intentional error is induced to prevent the primary PLC from operating the external device.
	Verification Items	Confirm that the standby controller immediately executes the corresponding I/O and that errors are displayed in red on the remote monitoring PC and HMI monitor via RS232 and HDMI interfaces.
Program Modification and Editing Test	Objective	To verify whether operators can immediately modify and edit programs on-site in case of errors.
	Method	Ladder and HMI programs are modified and edited through the HMI monitor’s touchscreen. The HDMI port is used for screen transmission, and the USB port is used for the touch interface.
	Verification Items	Confirm that operators can edit programs in real time during error situations.

):

For experimental items that could not be sufficiently validated in the testbed, an additional field control panel was fabricated and tested by an accredited external agency. The panel (800 mm × 500 mm × 1550 mm) was equipped with D/I input, D/O output, A/I input, and A/O output modules, as well as external driving devices, ensuring that the tests were performed under the same conditions as an actual industrial field.

5.1.1. Command Data Synchronization Experiment

The command data synchronization experiment verifies whether the command and event data transmitted from the remote monitoring control room are simultaneously delivered to both the primary PLC and the standby distributed controller. For this purpose, a network configuration was established between the remote monitoring control room, the primary PLC, and the standby controller.

The testbed configuration included an industrial PC for HMI operation, a primary PLC (LS XGB series), a standby distributed controller (embedded Linux system), and RS485/RS232 communication interfaces. Each experiment was repeated 100 times under identical environmental conditions to ensure statistical reliability.

As shown in Figure 25, the raw data (②) from the remote monitoring control room (①) are transmitted to the switching hub (③). The hub performs port forwarding to the primary PLC (④) and port mirroring to the standby distributed controller (⑤), ensuring that both devices receive identical data streams. In the actual IP configuration (⑥), the remote monitoring control room is assigned IP address 192.168.33.4, while the switching hub is assigned IP 192.168.33.1. Through this forwarding and mirroring process, it was confirmed that the primary PLC (IP 192.168.33.10) and the standby distributed controller (IP 192.168.33.7) simultaneously received the same command data, as verified in (⑦).

5.1.2. Black-Box Data Determination Experiment

After confirming in Figure 25 that the IP network configuration was functioning correctly, the experiment shown in Figure 26 was conducted to evaluate whether the black-box can accurately monitor and determine the execution results of the primary PLC.

The experimental procedure is as follows:

① The raw data (command and event data) generated from the remote monitoring and control room is transmitted through the switching hub to the primary PLC (⑥) and simultaneously mirrored to the standby distributed controller (⑦). ② The execution data from the primary PLC is then transmitted via an RS485 cable (⑨) to the black-box (⑧) within the standby distributed controller. During this process, the black-box compares the raw data (①), the execution data of the primary PLC (②), and the input data of the standby distributed controller (③). ④ The compared data is stored in the SSD memory of the black-box, and the final determination result is displayed on the black-box LCD (⑧).

The determination results are classified as follows:

• Write: counted when the command is executed successfully.
• Loss: counted when the command fails to be executed.

Furthermore, when the Loss item is touched, detailed information (such as occurrence time, corresponding I/O, and execution address) is displayed. This experiment demonstrates that the black-box not only identifies whether the command was executed but also enables rapid diagnosis of error causes.

5.1.3. Failover Execution and Error Notification Test

In this section, experiments were conducted to verify the failover execution capability of the I/O modules (D/I, D/O, A/I, A/O) integrated into the distributed controller. As designed and implemented in this study, these modules are structured such that when an error occurs in the primary PLC I/O, the standby distributed controller I/Os automatically perform failover execution. Furthermore, since multiple I/O targets must be monitored and controlled simultaneously in real industrial environments, each I/O module provides ID selector–based configuration and expansion functions, enabling systematic scalability for large-scale facilities.

Figure 27 illustrates the procedural flowchart of the test. The experiments were divided into two scenarios:

• Individual Error Test: verifies whether the standby distributed controller’s I/O module can successfully execute failover when a single I/O channel in the primary PLC fails.
• Multiple Error Test: simulates simultaneous failures of multiple I/O channels in the primary PLC and evaluates whether the standby distributed controller can reliably handle them concurrently.

The procedure followed the sequence outlined in Figure 27. First, during the blackbox determination and storage step, error occurrences were recorded. Next, in the result analysis step, the experimental data were analyzed to identify anomalies. Finally, in the failover execution verification step, the actual failover behavior of the distributed controller I/O modules was validated.

The experimental results are as follows: Figure 28 demonstrates the case where an error selectively occurred in the primary PLC A/I module. The blackbox recorded this situation by classifying it as a Loss in cases where the Write command was counted but the actual execution did not occur. In this case, the A/I module of the standby distributed controller automatically executed the failover operation, which was visually confirmed through LED indication on the module and error display on the HMI monitor.

Furthermore, as shown in Figure 29, even when multiple errors occurred simultaneously across different I/O modules, the standby distributed controller performed reliable failover execution. These error events were reflected in real-time both on the blackbox and the HMI monitor. In particular, the ID-based configuration function of the expansion modules ensured system scalability, and the experiments verified that real-time performance was maintained without degradation even under multiple error conditions.

Therefore, the proposed failover execution structure experimentally demonstrated its ability to ensure both high reliability and scalability, meeting the practical requirements of industrial applications.

5.1.4. Program Modification and Editing Test

This experiment verified that when an error occurs in the primary PLC I/O module, the standby distributed controller not only performs failover execution but also enables on-site modification and editing of ladder and HMI programs in real time. The development environment used is DYTO Studio, developed by Dong Young Tech One, which is pre-installed in the distributed controller’s internal SSD and can be executed without additional software installation.

Figure 30 illustrates the process of modifying or adding ladder logic. ① The target location is selected using a touch pen → ② the required logic symbol is chosen → ③ the logic is connected on the screen → ④ the new logic is automatically linked with subsequent elements and validated with comments → ⑤ the updated logic is applied successfully.

Figure 31 shows the process of HMI screen editing. ① The icon to be added is selected → ② size, color, and activation conditions are specified → ③ properties are confirmed → ④ the new icon is created → ⑤ the icon is applied and activated on the HMI screen.

Through this process, the operator can directly modify and adjust both control logic and user interfaces on the control panel. All changes are applied in real time to the standby distributed controller under failover execution. This structure ensures rapid error response and significantly improves operational convenience without requiring additional equipment or software installation.

5.2. Certification Tests by Authorized Institutions

Since the functionality and interoperability of the proposed system were thoroughly validated in Section 5.1, this section focuses on electromagnetic compatibility (EMC) and immunity (EMS) tests, which cannot be sufficiently verified by laboratory experiments alone. For this purpose, a control panel was fabricated with the same structure as the actual field system, including the primary PLC, standby distributed controller, I/O modules, blackbox, and HMI monitor. The tests were commissioned to three accredited institutions, including the Korea Testing Certification (KTC), and were conducted in compliance with the KS C9610-6-4 standard (equivalent to IEC 61000-6-4) [24,25,26,27] (Figure 32).

The main evaluation focused on electromagnetic compatibility (EMC) performance, particularly immunity and emission characteristics under AC 220 V, 60 Hz conditions.

• Main power conducted emission test (0.15–30 MHz): The measured interference levels remained within CISPR 32 Class A limits, including peak bands at 0.5 MHz, 5 MHz, and 10 MHz.
• Communication port conducted emission test (RS232, RS485, LAN): Both peak and average detection values satisfied the KS C9610-6-4 requirements, with no data loss or transmission errors observed.
• Radiated emission below 1 kHz band (30 MHz–1 GHz): All results remained within CISPR 32 Class A limits, and no functional degradation was observed even at peak frequencies.
• Radiated emission above 1 kHz band (1–6 GHz): Peak and average detection values were all below the allowable limits, verifying stable performance under high-frequency EMI conditions.

Additionally, the controller was tested against electrostatic discharge (ESD), radiated RF electromagnetic fields, electrical fast transients/bursts (EFT/Burst), surges, and conducted RF immunity. In all scenarios, the distributed controller maintained normal operation without requiring reset after the tests.

These results demonstrate that the proposed distributed controller achieves sufficient EMC robustness and reliability required for industrial applications.

5.3. Statistical Evaluation and Comparative Analysis

To strengthen the quantitative validity of the bench and field experiments presented above, additional statistical evaluations were conducted using the proposed heterogeneous redundant distributed controller. A total of 100 independent experimental trials were performed under varying load and network conditions, including induced jitter (±15 ms), packet loss (1–3%), and simultaneous multi-I/O failures.

The aggregated results are summarized in

Table 5. Quantitative Performance Comparison between Proposed and Conventional Systems.

Metric	Proposed System (Mean ± SD)	Conventional Redundant PLC	Improvement
Failover Latency (s)	2.46 ± 0.12	3.01 ± 0.19	−18.3%
Fault Detection Accuracy (%)	97.8 ± 0.7	86.9 ± 1.2	+12.5%
False Positive Rate (%)	2.3 ± 0.5	5.9 ± 0.8	−61.0%

.

As shown in Figure 33, the failover-latency distribution exhibited a narrow variance band (σ = 0.12 s), indicating stable recovery performance even under network perturbations. The use of a hybrid Ethernet/serial redundancy channel contributed to consistent latency control across trials.

A histogram showing the distribution of measured failover-latency values (100 trials) is presented below. The mean latency is 2.46 s with a standard deviation of 0.12 s, illustrating consistent timing behavior.

Measurement variables included failover latency (the time between fault detection and standby takeover), detection accuracy (true positive ratio), and the false positive rate (incorrect fault classification). The measured failover latency was 2.46 ± 0.12 s, showing an 18.3% improvement over conventional PLCs (3.01 ± 0.19 s). The proposed blackbox detection achieved 97.8% accuracy with a 2.3% FPR, reducing false alarms by 61%.

In addition, Figure 34 compares the receiver-operating-characteristic (ROC) curves of the proposed system and the conventional PLC redundancy method. The proposed controller achieved an AUROC = 0.991, while the conventional system recorded AUROC = 0.924, confirming superior discriminative capability in fault detection. This quantitative and comparative validation demonstrates that the proposed architecture not only functions correctly but also delivers statistically verified performance advantages suitable for deterministic industrial environments.

Measurement variables included failover latency (the time difference between primary PLC fault detection and standby takeover), fault detection accuracy (true positive ratio of blackbox anomaly detection), and the false-positive rate (incorrect fault identification under normal conditions).

6. Conclusions

Industrial automation and water treatment control systems aim for non-stop and error-free operation based on PLCs, yet in real field environments, failures, errors, and transient anomalies in PLCs often reduce system reliability and integrity. To overcome these limitations, real-time monitoring and rapid fault response mechanisms are essential.

In this study, a blackbox-integrated standby distributed controller was designed, implemented, tested, and verified. The proposed heterogeneous redundancy architecture demonstrated the following outcomes:

• Redundant data structure validation: implemented a port-forwarding-based configuration ensuring that the remote monitoring room, primary PLC, and standby distributed controller receive identical data.
• Blackbox recording and display functionality: experimentally confirmed that normal data (Write) and error data (Loss) are recorded and displayed on the LCD, providing intuitive execution results.
• Detailed log analysis: verified that execution details such as date, time, I/O information, addresses, and data logs can be stored and examined.
• Error detection and remote verification: demonstrated that D/I, D/O, A/I, and A/O errors can be consistently identified both in the blackbox and the remote monitoring room.
• On-site modification capability: verified that ladder logic and HMI editing can be performed directly on the local HMI monitor without additional equipment or software, ensuring rapid field response.
• Industrial validation and patent registration: the developed system has been registered under Korean Patent No. 10-2733771 (“Logic Monitoring Blackbox-Integrated Distributed Controller”) and has been applied in real-world water treatment facilities (e.g., flood control automation systems in Jinju City and Hapcheon County), where it is currently under field operation.

Quantitatively, the proposed system achieved a mean failover latency of 2.46 ± 0.12 s, representing an 18.3% improvement compared to conventional redundant PLCs (3.01 ± 0.19 s). The overall fault-detection accuracy reached 97.8%, while the false-positive rate was reduced by 61.0% (from 5.9% to 2.3%). These results statistically validate the reliability and responsiveness of the proposed heterogeneous redundancy approach.

Furthermore, stability during network jitter and multi-hop switching was confirmed through latency variance analysis (σ = 0.12 s), and operational convenience was verified by on-site logic and HMI modification with zero downtime. Such quantitative evidence demonstrates that the system not only functions as designed but also ensures measurable improvements in reliability, stability, and operational usability.

The proposed system thus overcomes the limitations of conventional homogeneous PLC redundancy, such as structural complexity, vendor dependency, and delayed fault diagnosis. The Linux-based distributed controller provides intuitive error logging and visualization, I/O failover execution, and on-site programmability, thereby experimentally demonstrating its ability to ensure reliability, stability, and operational convenience required in industrial environments.

Future work will expand error detection beyond D/I, D/O, A/I, and A/O to include parameters such as resistance, temperature, pressure, and communication data, thereby developing a next-generation blackbox-integrated distributed control system capable of comprehensive monitoring.

Quantitatively, the proposed system achieved a mean failover latency of 2.46 ± 0.12 s, detection accuracy of 97.8%, and a 61% reduction in the false-positive rate compared with conventional PLCs. These results provide measurable evidence of improved reliability, stability, and operational convenience.