Next Article in Journal
On the Evolution Operators of a Class of Time-Delay Systems with Impulsive Parameterizations
Previous Article in Journal
Flow Shop Scheduling with Shortening Jobs for Makespan Minimization
Previous Article in Special Issue
An Integrated T-Spherical Fuzzy Einstein Interaction Aggregator Group Decision-Making Approach: A Case Study of Concrete 3D Printing Robot Application in Vietnam
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Spatiotemporal Fuzzy Modeling Approach Combining Automatic Clustering and Hierarchical Extreme Learning Machines for Distributed Parameter Systems

School of Mechatronics and Automation, Shanghai University, Shanghai 200444, China
*
Author to whom correspondence should be addressed.
Mathematics 2025, 13(3), 364; https://doi.org/10.3390/math13030364
Submission received: 24 December 2024 / Revised: 17 January 2025 / Accepted: 21 January 2025 / Published: 23 January 2025
(This article belongs to the Special Issue Intelligent and Fuzzy Systems in Engineering and Technology)

Abstract

:
Modeling distributed parameter systems (DPSs) is challenging due to their strong nonlinearity and spatiotemporal coupling. In this study, a three-dimensional fuzzy modeling method combining genetic algorithm (GA)-based automatic clustering and hierarchical extreme learning machine (HELM) is proposed for DPS modeling. The method utilizes GA-based automatic clustering to learn the premise part of 3D fuzzy rules, while HELM is employed to learn spatial basis functions and construct a complete fuzzy rule base. This approach effectively captures the spatiotemporal coupling characteristics of the system and mitigates the information loss commonly observed in dimensionality reduction in traditional fuzzy modeling methods. Through experimental verification, the proposed method is successfully applied to a rapid thermal chemical vapor deposition system. The experimental results demonstrate that the method can accurately predict temperature distribution and maintain good robustness under noise and disturbances.

1. Introduction

Distributed parameter systems (DPSs) are complex systems consisting of spatially distributed state variables, usually described by partial differential equations (PDEs) [1]. These systems are widely found in complex engineering applications such as chemical reactors [2], battery thermal processes [3], and forging processes [4], where the dynamic behavior of the system involves both spatial and temporal dimensions. Modeling and controlling DPSs face many challenges due to their infinite dimensionality, strong nonlinearity, and spatiotemporal coupling [5]. Traditional modeling methods, such as the finite element method [6] and the finite difference method [7], usually require dimensionality reduction or rely on some assumptions, which may lead to the loss of system dynamic information, thus affecting the accuracy of prediction and control [8].
Data-driven modeling methods have gradually gained widespread attention to address these limitations [9]. In particular, machine learning techniques offer the potential to capture complex nonlinear behaviors without relying on explicit mathematical formulas [10]. Fuzzy modeling is a very promising solution due to its ability to handle uncertainty and ambiguity in system behavior. However, traditional fuzzy models often have difficulty effectively capturing the complex interactions between space, time, and system variables when faced with high-dimensional spatiotemporal data, which limits their application in DPS [11]. Wang and Li [12] proposed a dynamic spatiotemporal modeling method based on sliding windows to handle DPS with time-dependent boundary conditions. The sliding window is used to capture the latest spatiotemporal data, and the forgetting factor is combined to adjust the data influence. The traditional spatiotemporal modeling method is improved and shows superior performance in battery simulation with unknown boundary cooling. Xu et al. [10] proposed a Hammerstein–Wiener model based on an extreme learning machine (ELM) to approximate complex nonlinear systems. The model approximates the nonlinear part by two independent static ELM networks and estimates the linear part structure by combining Lipschitz quotient criteria. It is fast to learn and has low computational complexity. Chen et al. [13] proposed a learning-based framework for online spatiotemporal modeling of distributed thermal processes in soft-pack lithium-ion batteries under sparse sensors. The spatial basis functions were extracted through offline learning and dynamically updated online to achieve accurate temperature prediction under limited sensor conditions. Jin et al. [14] solved the complex nonlinear DPS modeling problem through nonlinear time domain transformation and spatiotemporal domain reconstruction. First, the spatiotemporal output was transformed using local nonlinear dimensionality reduction; then, the time model was established through ELM, and, finally, the spatiotemporal output was reconstructed. Deng et al. [8] proposed a physical information space fuzzy system framework that combines physical knowledge and fuzzy systems to capture the spatial characteristics of complex DPS through 3D fuzzy input and reasoning mechanisms. Experiments have verified its high accuracy in modeling complex spatiotemporal systems. However, these traditional modeling methods based on spatiotemporal separation and spatiotemporal integration require model simplification, which will lose model accuracy and are not linguistically interpretable.
In recent years, a new type of fuzzy modeling method, three-dimensional (3D) fuzzy modeling, has been proposed to address the spatiotemporal characteristics of complex systems [15]. Unlike traditional fuzzy models that only deal with lower-dimensional spaces, 3D fuzzy models can simultaneously represent changes in spatial and temporal dimensions, which enables the model to more comprehensively describe the behavior of the system and capture the complex spatiotemporal coupling relationships present in DPS [16]. In 3D fuzzy models, spatiotemporal separation and spatiotemporal synthesis are naturally integrated into 3D fuzzy rules. In each 3D fuzzing rule, spatiotemporal separation is achieved by treating the premise part as a time coefficient and the consequent part as a spatial basis function. The combination of multiple 3D fuzzing rules achieves spatiotemporal reconstruction. Compared to traditional DPS modeling methods, 3D fuzzy modeling has two significant advantages: first, it does not rely on model dimensionality reduction, and, second, it has linguistic interpretability. Three-dimensional fuzzy models retain the complexity of the system without dimensionality reduction. However, in order to accurately construct 3D fuzzy models, several challenges need to be addressed, especially in determining appropriate fuzzy rules and learning the relationship between input and output variables.
Therefore, this paper proposes a novel 3D fuzzy modeling method that combines genetic algorithm (GA)-based automatic clustering and hierarchical extreme learning machine (HELM). The GA-based automatic clustering method [17] is used to divide the input and output data into different clusters, each cluster corresponds to a system behavior pattern and automatically adjusts the cluster center through the optimization process. This can not only reduce redundant rules but also improve the adaptability of the model under different system behaviors. At the same time, HELM is used to learn spatial basis functions, which can effectively capture changes in system behavior at different spatial locations. The hierarchical structure of HELM enhances the expression and generalization capabilities of the model, significantly improves computational efficiency, and is especially suitable for processing complex systems with spatiotemporal nonlinear characteristics [18]. The primary contributions of this work are as follows.
  • A 3D fuzzy modeling method combining GA-based automatic clustering and HELM is proposed for complex DPS.
  • A GA-based automatic clustering method is developed to learn the premise of the 3D fuzzy model and achieve adaptive optimization of 3D fuzzy rules.
  • Experiments in a rapid thermal chemical vapor deposition process show that the proposed method is superior to traditional modeling methods due to its ability to adaptively optimize fuzzy rules and effectively learn spatial basis functions.
This paper is organized as follows: Section 2 describes the background and modeling challenges of the RTCVD system in detail. Section 3 introduces the modeling framework of the proposed method. Section 4 presents the experimental setup and results, and provides a comparative analysis. Finally, Section 5 summarizes the work in this paper and provides prospects for future research directions.

2. Problem Description

The Rapid Thermal Chemical Vapor Deposition (RTCVD) system is an advanced deposition technology commonly used in semiconductor and materials processing [19]. This system combines the principles of chemical vapor deposition with rapid thermal processing (RTP) to achieve high-quality thin film deposition at significantly higher speeds than traditional methods. In a typical RTCVD system, a 6-inch silicon wafer is placed on a rotating stage within the reaction chamber. The wafer is heated by a set of three independent light sources, each with specific functions: Zone 1 uniformly heats the entire wafer surface, Zone 2 heats the wafer edges, and Zone 3 provides almost uniform heating across the wafer. The precise control of the heat flux from these light sources ensures the rapid, yet controlled, temperature rise required for effective deposition. The diagram of the RTCVD is shown in Figure 1.
During the process, silane (SiH4) gas is introduced into the reactor, where it decomposes into silicon and hydrogen. At temperatures of approximately 800 K or higher, a 0.5 μm thick polysilicon layer is deposited on the wafer within a very short time, typically around 1 min. The rotating stage serves to maintain uniform temperature distribution across the wafer, with particular attention to the wafer radial temperature uniformity. Due to the thin nature of the silicon wafer, the temperature variation across the wafer’s azimuthal direction can be neglected. However, to achieve uniform polysilicon deposition, it is critical to maintain a consistent temperature across the wafer radius. This is achieved by adjusting the power levels of the three heating zones (light groups), which are carefully controlled during the deposition process.
The system is modeled using a one-dimensional PDE that describes the temperature and deposition dynamics, simplified from the underlying multidimensional thermodynamic processes. This model allows for accurate predictions of temperature and film thickness, crucial for optimizing process conditions and improving film uniformity. The real-time control of temperature, deposition rate, and material properties is challenging due to the system’s nonlinear spatiotemporal dynamics. The rapid changes in both spatial and temporal domains demand high-precision modeling to ensure uniform deposition and high-quality thin films. The dimensionless partial differential equation description of the RTCVD system is as follows:
T r t = k 0 1 r T r r + 2 T r r 2 + ϵ 1 T r 4 + ω [ q a r u a + q b r u b + q c r u c ]
Subject to the boundary conditions as follows:
T r r = σ e d 1 T r + q e d u b ,     r = 1 T r r ,     r = 0
where T r = T r / T a m b denotes dimensionless wafer temperature. r = r / R w dimensionless radial position, normalized by the wafer radius R w = 7.6   c m , k 0 = 0.0021 is the thermal conductivity of the wafer. ϵ = 0.0012 is the radiation coefficient of the quartz chamber. ω = 0.026 is the wafer density. q a r , q b r , q c r : thermal radiation flux distributions in the radial direction from the three heating zones. u a , u b , u c : control inputs for the respective heating zones. σ e d = 0.0037 is the radiation coefficient of the wafer. q e d = 4.022 is the incident radiation flux at the wafer edge.
The RTCVD system is a complex spatiotemporal system with infinite dimensionality. For practical purposes, only a finite number of sensors are used to measure the system output. Assume that the sensors are located at positions [ z 1 , z 2 , …, z P ], which captures the system output. The spatial domain is represented by the vector Z ¯ = [ z 1 , z 2 , . . . ,   z P ] T . At each time step k , the spatial output is given by the vector: y Z ¯ , t = [ y z 1 , t , y z 2 , t , , y z P , k t ] T . The temporal input is denoted by u t = [ u 1 t , u 2 t , u 3 t ] T , where k R is the time variable. The objective of the modeling problem is to develop a spatiotemporal model based on the input data { u t } t = 1 T and the output data { y z i , t } i = 1 , t = 1 P , T , where T is the length of the time series. The model aims to capture the spatiotemporal dynamics of the system.

3. 3D Fuzzy Modeling Based on Automatic Clustering and HELM

The 3D fuzzy modeling method proposed in this section combines automatic clustering based on GA and HELM to effectively model complex systems with spatiotemporal dynamic characteristics. This method fully utilizes the advantages of each component to accurately capture the spatial and temporal relationships of the system while avoiding the limitation of dimensionality reduction.
The proposed 3D fuzzy modeling method is shown in Figure 2. First, the premise part of the fuzzy rule is learned by applying automatic clustering techniques, which are optimized using GA. Specifically, the GA is employed to guide the clustering process, ensuring that the fuzzy rules are constructed based on the optimal division of the input data. This process divides the input and output data into several clusters, each corresponding to a different mode of system behavior. Each cluster defines a fuzzy rule to describe the relationship between the input (spatial and temporal) and the output. The clustering process is optimized by GA, which not only reduces redundant rules but also improves the efficiency and adaptability of the model. Then, the spatial basis functions are learned using hierarchical ELM. The spatial basis functions learned by HELM can effectively represent the behavioral changes of the system at different spatial locations. After learning the premise part and spatial basis functions of the fuzzy rules, the final 3D fuzzy model combines the two to form a complete spatiotemporal modeling framework.

3.1. 3D Fuzzy Model

The 3D fuzzy model is an advanced modeling method for dealing with complex nonlinear DPS [20]. Unlike traditional fuzzy models, the 3D fuzzy model does not rely on dimensionality reduction techniques and can directly model the dynamics of systems with spatiotemporal coupling, avoiding information loss [21]. The model is based on the concept of 3D fuzzy sets and can simultaneously represent changes in time and space dimensions. By treating time and space as closely related variables, the 3D fuzzy model can comprehensively capture the behavioral changes of the system at different time points and spatial locations. The 3D fuzzy modeling framework is shown in Figure 3. A detailed description of the 3D fuzzy model is given in the Appendix A.
The core advantage of the 3D fuzzy model is that it can preserve the complexity of the system without loss. It combines spatiotemporal separation with spatiotemporal reconstruction to achieve a coordinated modeling framework. The 3D fuzzy model can describe the DPS using the following 3D fuzzy language rules.
R ¯ l : i f   y Z ¯ , t 1   i s   C ¯ 1 l   a n d     a n d   y Z ¯ , t H   i s   C ¯ H l a n d   u 1 t 1   i s   U 1 1 l   a n d     a n d   u 1 t K   i s   U 1 K l a n d   u m t 1   i s   U m 1 l   a n d     a n d   u m t K   i s   U m K l   t h e n   y Z ¯ , t   i s   φ l ( Z ¯ )
where { u t 1 , u t 2 , , u t H } and { y Z ¯ , t 1 , y Z ¯ , t 2 , , y Z ¯ , t K } are taken as the input variables for a 3D fuzzy system. C ¯ i l ( i = 1,2 , , H ) and U j K l ( j = 1,2 , , m ) are the 3D fuzzy set and the traditional fuzzy set, respectively. φ l ( Z ¯ ) is a spatial basis function, and Z ¯ = [ z 1 , z 2 , , z H ] is the space domain. m a x   H , K is the order of the model. l ( l = 1,2 , , N u m R ) is the number of fuzzy rules.
For each 3D fuzzy rule R ¯ l , the premise is used to compute the temporal coefficient, while the consequence represents the spatial basis functions. Thus, the fuzzy rule inherently realizes spatiotemporal separation. When multiple 3D fuzzy rules ( R ¯ 1 , R ¯ 2 , , R ¯ N u m R ) are activated simultaneously, their union generates the fuzzy output of the 3D fuzzy model, thereby naturally achieving spatiotemporal synthesis.

3.2. GA-Based Automatic Clustering Learning for the Premise Part

In 3D fuzzy systems, the premise part determines the input part of the fuzzy rules. Its main task is to automatically determine the best clustering center based on the characteristics of the input data. The main challenge is how to determine the optimal number of 3D fuzzy sets in the premise part and their corresponding membership functions. Traditional methods often rely on manual adjustment or prior knowledge, which may not be feasible in the case of complex systems and large-scale data sets. This paper uses the GA-based automatic clustering learning premise part to automatically optimize 3D fuzzy rules. The detailed process of automatic clustering based on GA is given below.
(1)
Initializing the Population
The first step is to generate the initial population. Each individual represents a clustering structure, consisting of a set of clustering centers c = { c 1 , c 2 , , c n } . The initial clustering centers are randomly selected from the data points.
(2)
Fitness Evaluation
The fitness of each individual is evaluated using a clustering quality measure, such as the Davies–Bouldin (DB) index. This is performed by calculating the distance matrix d i j between the data sets X = { u t , y z i , t } i = 1 , t = 1 P , T and the clustering centers c . Each data point X i is assigned to the nearest clustering center m j , and the minimum distance d m i n ( X i ) is calculated. Based on this, the spread of each cluster S j is computed, and then the relative spread r i j between clusters is calculated along with the DB index. The specific calculations are as follows.
d i j = | | X i c j | |
S j = ( 1 N j i C j d m i n ( X i ) q ) 1 / q
r i j = S i + S j D i j
D B = 1 k i = 1 k r i j j i m a x
where N j is the number of data points assigned to cluster C j , and q is the Minkowski distance parameter. D i j is the distance between clusters C i and C j , k is the number of clusters, and r i j is the relative spread between clusters C i and C j . The fitness of each individual is given by the DB index value. A higher fitness indicates a better clustering result, meaning that the clustering centers and their weights more effectively describe the structure of the data.
(3)
Selection and Crossover
In the selection process, roulette wheel selection is used to select parent individuals. The higher the fitness (i.e., the lower the DB index), the greater the probability that the individual will be selected. The selection probability is calculated as follows.
P i = exp   ( β     C o s t i / W o r s t C o s t ) j = 1 P o p exp   ( β     C o s t i / W o r s t C o s t )
where C o s t i is the fitness of the i -th individual. W o r s t C o s t is the largest fitness value in the current population. β = 8 is a parameter used to adjust the steepness of the selection. P o p is the total number of individuals in the population.
The purpose of the crossover operation is to generate new offspring individuals by combining the genes of two parent individuals, promoting extensive exploration of the knowledge space. The crossover operation retains the advantages of the parents while combining the genetic information of the two parents to provide more potential solutions. The crossover operation generates new offspring individuals by weighted average. First, a weight matrix α is randomly generated. The random number matrix α determines the weight of the combination of genes of two parent individuals. The introduction of the weight matrix α makes the crossover process more flexible and can generate different offspring according to different parent information. This operation can effectively avoid premature convergence of the algorithm. Then, gene recombination is performed. For the two parent individuals, the two offspring are calculated by weighted average. The specific calculation process is as follows.
α ~ U ( γ , 1 + γ )
where α is a random value drawn from a uniform distribution in the range [ γ , 1 + γ ] and γ = 0.2 controls the randomness and diversity of the crossover process.
Y 1 = α · X 1 + ( 1 α ) · X 2
Y 2 = α · X 2 + ( 1 α ) · X 1
where α acts as a weight factor between the two parent individuals. X 1 X 2 are two parent individuals, Y 1 , Y 2 are two offspring individuals.
(4)
Mutation
The purpose of the mutation operation is to introduce random changes in the genes of individuals to prevent the algorithm from falling into a local optimal solution during the search process. Randomly select parent individuals for random perturbation to generate mutated individuals. For each gene position j , the mutated gene value X i j is calculated as follows.
n m u = c e i l ( μ · n V a r )
X i j = X i j + σ · N ( 0,1 ) ,     i f   j s e l e c t e d   g e n e s X i j ,     o t h e r w i s e
where n m u is the number of genes that need to be mutated, μ = 0.02 is the mutation rate. σ = 0.1 · ( V a r M a x V a r M i n ) is the disturbance amplitude, which determines the range of variation. n V a r is the number of genes in an individual. V a r M a x and V a r M i n are the maximum and minimum values of the decision variables, respectively. The iterative change process of the fitness function is shown in Figure 4. The specific parameter values are shown in Table 1.
After automatic clustering, X is divided into N u m R groups and N u m R is the number of clusters. Each group is a cluster, each cluster corresponds to a 3D fuzzy rule, and N u m R 3D fuzzy rules are obtained. The cluster center constitutes the premise part of the 3D fuzzy rule. This paper adopts the Gaussian membership function, and the cluster center corresponds to the center of the Gaussian membership function in the premise part. The calculation method of the time coefficient (also called a fuzzy basis function in the context of the 3D fuzzy system) of the premise part is as follows.
ψ l t = j = 1 P a j l i = 1 H exp   ( y z j , t i c i j l σ i j l ) 2 × i = 1 m k = 1 K e x p   ( ( u i t k d i k l δ i k l ) 2 )
where c i j l and σ i j l represent the center and width of the Gaussian 3D fuzzy set C ¯ i l , d i k l and δ i k l represent the center and width of the traditional Gaussian fuzzy set U i k l

3.3. Learning the Consequent Part Based on HELM

HELM is a neural network model based on a hierarchical structure, which combines the advantages of ELM and introduces the structure of the autoencoder (AE) hierarchy. The architecture consists of multiple autoencoder layers (ELM-AE layers) and an output layer [22,23]. HELM implements feature learning and mapping through multiple layers of ELM-AE layers, and can effectively handle complex nonlinear problems. The architecture includes an input layer, multiple hidden layers, and an output layer. The input layer receives the original data and performs linear transformation and nonlinear activation through each layer of the ELM-AE layer to gradually extract the high-level features of the data. Each ELM-AE layer contains input weights, biases, and output weights, and its parameters are trained through unsupervised learning to gradually capture richer feature representations in multiple levels. Finally, the output layer optimizes the output weights through supervised learning and combines the features of all hidden layers for final prediction. The HELM framework structure is shown in Figure 5.
In the ELM-AE training phase, the autoencoder (ELM-AE) of each layer is trained. Each layer is linearly transformed by the input data and input weights, and a nonlinear activation function (ELU) is applied to generate the feature representation of the layer. In this way, the model gradually extracts high-level features of the data. During the training of each layer, the output weights are updated and the features of the layer are passed to the next layer. Then, the model enters the output layer training phase. At this time, the weights of the output layer are optimized by the least squares method.
The motivation for using HELM to learn spatial basis functions lies in its efficient training mechanism and powerful nonlinear mapping ability, which can capture complex input–output relationships [24]. The hierarchical structure enhances the representation and generalization capabilities of the model and is particularly suitable for processing systems with spatiotemporal nonlinear characteristics. Through layer-by-layer feature learning, HELM not only reduces computational complexity but also improves the adaptability and interpretability of the system.
The training process of HELM maintains the high efficiency of ELM. The weights of the hidden layer are randomly selected, avoiding the computational complexity of backpropagation in traditional neural networks. Since the fuzzy rules directly correspond to the hidden layer nodes, the output of the system can be clearly linked to the consequent part of each fuzzy rule, which enhances the interpretability of the model.
In this study, HELM is used to learn the spatial basis functions of the consequent part. The size of the input layer is equal to the size of the input vector, and the number of nodes in the output layer and the hidden layer corresponds to the number of rules. The weight φ l between the hidden layer and the output layer is called the l -th spatial basis function. Each spatial basis function φ l corresponds to the consequent part of a fuzzy rule. These weights learn the mapping between the input signal and the output signal in the rule. HELM can effectively represent spatial basis functions by learning weights.
The learning results of the premise part serve as the input of HELM. Specifically, given an input sample S = { ( ψ ( x i ) , y i ) | ψ ( x i ) R N R , y i R P × 1 , i = 1,2 , , n } , where each sample ψ ( x i ) is an input vector and y i z is the corresponding target output. For each hidden layer i (from the input layer to the penultimate layer), the input is the output from the previous layer. The hidden layer output matrix H l is computed, where the elements are the activation values of the hidden nodes. The computation formula is as follows.
H i = G ( α i y i 1 + b i )
where y i 1 is the output of the previous layer, α i is the input weight of the current layer, b i is the bias, and G ( · ) is the activation function.
For each layer l , the output weights β i are computed using the least squares method, as follows.
β i = ( I C + H i T H i ) 1 H i T y i 1
where C is the regularization coefficient, and I is the identity matrix.
For each layer i = 1,2 , , y 1 , the output from the previous layer is used as the input for the current layer. The hidden layer output matrix H i and output weights β i are computed sequentially. For the final layer i = y , the output weights φ l for the last layer are computed directly using the least squares method:
φ l = ( I C + H y T H y ) 1 H y T T
where H y is the hidden layer output matrix for the final layer, and T = [ y 1 , y 2 , , y n ] is the final target output.
The proposed 3D fuzzy modeling flowchart is shown in Figure 6. The output model of the online 3D fuzzy system is as follows.
y Z ¯ , t = l = 1 N R ψ l t φ l ( Z ¯ )
where ψ l t is the temporal coefficient, φ l ( Z ¯ ) is the spatial basis function.

4. Experimental Validation

The RTCVD described in Section 2 is used to evaluate the effectiveness of the proposed modeling approach. In this experiment, random disturbances with amplitudes no greater than 10% were applied to the system input variables u a , u b , u c to capture sufficient dynamic information. Specifically, the disturbed inputs can be expressed as follows:
u a t = u a 1 + 0.1     u a 1     n o r m r n d ( 0,1 ) u b t = u b 1 + 0.1     u b 1     n o r m r n d ( 0,1 ) u c t = u c 1 + 0.1     u c 1     n o r m r n d ( 0,1 )
where n o r m r n d ( 0,1 ) simulates the random direction of the disturbance signal. This approach effectively simulates the fluctuations of input variables and enhances the dynamic response of the system.
Additionally, 11 sensors were uniformly distributed along the radial direction for data acquisition. To simulate noise effects in real-world operation, independent Gaussian white noise with zero mean and a standard deviation of n = 0.2 % was added to the sensor data. The inclusion of noise introduces uncertainty into the experimental data, enhancing the model’s robustness to disturbances and errors, and thereby improving the practical representativeness of the experimental results. Under steady-state conditions, the values of u a 1 , u b 1 , and u c 1 are 0.2028, 0.1008, and 0.2245, respectively, corresponding to the stable input values at a furnace temperature of 1000 K. These steady-state values serve as the baseline inputs, with disturbances applied on top of them.
In this study, the sampling interval was set to 0.5 s, with a total simulation period of 2500 s. Consequently, 5000 samples were generated. Among these, 2000 samples were randomly selected for training experiments, and another 500 samples were randomly chosen for testing experiments.
The premise part of the 3D fuzzy rules is learned through the GA-based automatic clustering method. The data set X is divided into 13 groups to determine the input space of the premise part of the rule base. Specifically, the GA-based automatic clustering method automatically divides the input data into 13 different categories, each of which corresponds to a fuzzy rule. These 13 fuzzy rules are generated through automatic clustering to ensure that each rule can effectively reflect the different characteristics of the input data. Subsequently, the weights of the output layer are learned through HELM to obtain 13 spatial basis functions. By combining the premise part with the consequent part, a complete three-dimensional fuzzy rule base is constructed, which contains 13 fuzzy rules. The first six three-dimensional fuzzy rules are listed below.
R ¯ 1 : if   y ( Z ¯ , t 1 ) is   sightly   NM   and   u 1 ( t 1 ) is   less   PM   and   u 2 ( t 1 ) is   very   ZO   and   u 3 ( t 1 ) is   less   than   PB ,   then   y ( Z ¯ , t ) is [−1.1756 −1.1146 −1.1229 −1.3121 −1.1951 −1.0693 −1.1978 −1.0612 −1.2563 −1.1267 −1.2271];
R ¯ 2 : if   y ( Z ¯ , t 1 ) is   very   NS   and   u 1 ( t 1 ) is   less   than   PB   and   u 2 ( t 1 ) is   sort   of   ZO   and   u 3 ( t 1 ) is   slightly   PB ,   then   y ( Z ¯ , t ) is [−0.0504 −0.3260 −0.6957 −0.4022 −0.6820 0.1543 −0.0835 0.3123 −0.1161 0.3101 0.2692];
R ¯ 3 : if   y ( Z ¯ , t 1 ) is   very   ZO   and   u 1 ( t 1 ) is   PM   and   u 2 ( t 1 ) is   less   PS   and   u 3 ( t 1 ) is   slightly   PM ,   then   y ( Z ¯ , t ) is [0.2303 0.4975 0.2830 0.1465 0.0785 0.5908 0.1568 0.2411 0.3195 0.2691 0.1395];
R ¯ 4 : if   y ( Z ¯ , t 1 ) is   not   NM   and   u 1 ( t 1 ) is   very   PS   and   u 2 ( t 1 ) is   sort   of   ZO   and   u 3 ( t 1 ) is   less   PL ,   then   y ( Z ¯ , t ) is [−0.0494 −0.4380 −0.5462 −0.2444 −0.3338 −0.5723 −0.2176 −0.2473 −0.2097 −0.2519 −0.2786];
R ¯ 5 : if   y ( Z ¯ , t 1 ) is   slightly   PS   and   u 1 ( t 1 ) is   less   PM   and   u 2 ( t 1 ) is   very   ZO   and   u 3 ( t 1 ) is   less   than   PB ,   then   y ( Z ¯ , t ) is [−0.2324 −0.3239 −0.3178 −0.4397 −0.3438 −0.0708 −0.5105 −0.3961 −0.3791 −0.4069 −0.8812];
R ¯ 6 : if   y ( Z ¯ , t 1 ) is   sort   of   NS   and   u 1 ( t 1 ) is   sort   of   PS   and   u 2 ( t 1 ) is   slightly   ZO   and   u 3 ( t 1 ) is   less   than   PM ,   then   y ( Z ¯ , t ) is [0.0625 0.0325 0.0335 −1.8757 0.0708 −0.1220 0.0312 −0.0611 0.0167 −0.0973 −0.1266].
The fuzzy set of each input variable is represented by a Gaussian membership function, where NB represents negative big, NM represents negative middle, NS represents negative small, ZO represents zero fuzzy set, PS represents positive small, PM represents positive middle, and PB represents positive big. The membership function of the premise part of the 3D fuzzy rule is shown in Figure 7.
The first six spatial basis functions are shown in Figure 8. The true output and prediction results of the proposed model method in 2000 training samples are shown in Figure 9, and the true output and prediction results of the model in 500 test samples are shown in Figure 10. The prediction error and relative error of the proposed modeling method are shown in Figure 11. The results show that the spatial distribution of the predicted output can well match the true temperature distribution. The matrix representations of the first six spatial basis functions are shown below.
φ l ( z ) = 1.1756 0.0504 0.2303 0.0494 0.2324 0.0625 1.1146 0.3260 0.4975 0.4380 0.3239 0.0325 1.1229 0.6957 0.2830 0.5462 0.3178 0.0335 1.3121 0.4022 0.1465 0.2444 0.4397 1.8757 1.1951 0.6820 0.0785 0.3338 0.3438 0.0708 1.0693 0.1543 0.5908 0.5723 0.0708 0.1220 1.1978 0.0835 0.1568 0.2176 0.5105 0.0312 1.0612 0.3123 0.2411 0.2473 0.3961 0.0611 1.2563 0.1161 0.3195 0.2097 0.3791 0.0167 1.1267 0.3101 0.2691 0.2519 0.4069 0.0973 1.2271 0.2692 0.1395 0.2786 0.8812 0.1266 T
The proposed modeling method was compared with two existing methods, including KL-LS [25] and KL-LS-SVM [26], to verify the performance of the proposed modeling method. The KL-LS modeling approach effectively models the dynamics of DPS by combining the Karhunen–Loeve (KL) decomposition and least squares estimation (LS). It first uses KL decomposition to transform the infinite-dimensional output of the system into a finite-dimensional approximation, capturing the spatial correlation of the system through spatial basis functions. This reduction allows the extraction of time-varying coefficients, which are then identified through least squares estimation and instrumental variables to establish the dynamic relationship between the system input and output. This method can accurately model spatial and temporal behavior even under nonlinear conditions. The effectiveness of the method has been demonstrated through simulations of parabolic and hyperbolic systems, highlighting its ability to simulate complex spatiotemporal dynamics in various distributed processes with improved computational efficiency and accuracy.
The KL-LS-SVM is a time/space separation based SVM model identification method for unknown nonlinear DPS. The method first uses KL decomposition to separate the time and space domains and reduce the dimensionality of the system. Then, the spatiotemporal outputs are projected onto the low-dimensional KL space, and the time coefficients capture the dynamics of the system. A least squares support vector machine is used to model the system in this simplified time domain. After integrating the time and space components, the nonlinear spatiotemporal dynamics of the system can be reconstructed. The parameters of all comparison methods are set according to the original text.
Figure 12 and Figure 13 compare their performance in time normalized absolute error (TNAE) and relative L 2 norm error (RLNE), respectively. The experimental results in Figure 12 show that the proposed method has lower error than the other methods at all sensor locations. The RLNE results in Figure 13 show that the proposed method performs better than other methods with good stability. Table 2 shows the root mean square error (RMSE) of the compared methods in the training and testing stages. For a reliable performance evaluation, we performed the experiments 10 times with different random data splits. The RMSE values were then averaged across all 10 repetitions to ensure that the results are not biased by the random variation in the data selection.
In summary, the proposed 3D fuzzy modeling approach successfully captures the spatiotemporal coupled dynamics of DPS by effectively combining the advantages of GA automatic clustering and HELM. GA-based automatic clustering can accurately divide input and output data and generate high-quality fuzzy rules, avoiding information loss and dimensionality reduction errors in traditional methods; HELM builds a complete three-dimensional fuzzy rule library by learning spatial basis functions, improving the accuracy and expression ability of the model. Experimental verification results show that this method can accurately predict the temperature distribution of RTCVD and shows good robustness under the influence of noise.

5. Conclusions

In this paper, an innovative 3D fuzzy modeling method combining GA-based automatic clustering and HELM was proposed for DPS modeling. The combination of GA-based automatic clustering and HELM effectively captures the spatiotemporal coupling relationship in DPS, while avoiding the information loss that may occur in the dimensionality reduction process of traditional methods. Experimental results demonstrated the effectiveness of this method in the RTCVD system. Compared with the existing KL-LS and KL-LS-SVM methods, the proposed model has significant advantages in prediction accuracy, computational efficiency, and robustness. Of course, the proposed 3D fuzzy modeling method still has certain limitations. This approach may face challenges in dealing with complex systems with rapid dynamic changes. Future work will explore more effective online learning methods to improve the adaptability and real-time performance of this method in dynamic environments.

Author Contributions

Conceptualization, G.Z.; methodology, G.Z. and X.Z.; software, G.Z.; validation, X.Z.; formal analysis, B.W.; investigation, T.W.; resources, G.Z.; data curation, B.W. and T.W.; writing—original draft preparation, G.Z.; writing—review and editing, G.Z. and X.Z.; visualization, B.W.; supervision, X.Z.; project administration, X.Z.; funding acquisition, X.Z. and B.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the project from the National Natural Science Foundation of China under Grant number 62073210 and Grant number 62173219.

Data Availability Statement

The original contributions presented in this study are included in the article. For further inquiries, please contact the corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

3D Fuzzy Model for Distributed Parameter Systems (DPS)

Let the spatial domain of a Distributed Parameter System (DPS) be denoted as Z ¯ = { z 1 , z 2 , , z Q } , where Q represents the number of sensors. The system has m spatial input variables, ω Z ¯ = ω 1 Z ¯ , ω 2 Z ¯ , , ω m Z ¯ R Q     m , where each spatial input ω s Z ¯ is represented as a vector of values [ ω s z 1 , ω s z 2 , , ω s z P ] R Q     1 , for s = 1,2 , , m . These spatial input vectors serve as the input to the 3D fuzzy system, which is structured as follows.
(1)
3D Fuzzification
The system employs a singleton fuzzifier for 3D fuzzification, where each spatial input ω s Z ¯ is fuzzified using Gaussian-type membership functions. The Gaussian functions define the degree of membership of each spatial input within different 3D fuzzy sets.
(2)
3D Fuzzy Rule Set
The 3D fuzzy system operates based on a set of N u m R fuzzy rules, denoted as R l for l = 1,2 , , N u m R . These rules describe the relationships between the spatial inputs and the system’s output.
(3)
3D Fuzzy Rule Reasoning
The product t n o r m is used to combine the fuzzy sets associated with different spatial inputs. This approach ensures that the system accounts for the interactions between the various spatial variables in the DPS.
(4)
Defuzzification
A linear defuzzifier is applied to compute the final output from the combined fuzzy sets. This step transforms the fuzzy output into a crisp value that corresponds to the system’s response.
The output of the 3D fuzzy system, denoted as y ( Z ¯ ) , is a weighted sum of spatial fuzzy basis functions, which is given as follows:
y Z ¯ = l = 1 N u m R j = 1 Q a j l s = 1 m μ C s l ω s z j ϕ l ( Z ¯ )
where ψ l ( Z ¯ ) = j = 1 Q a j l s = 1 m μ C s l ω s z j is the spatial fuzzy basis function associated with rule R l . μ C s l ω s z j is the membership function of the fuzzy set C s l associated with the s -th spatial input variable ω s Z ¯ . a j l is the spatial weight associated with sensor z j . ϕ l ( Z ¯ ) is the consequent function associated with the fuzzy rule R l .
In this formulation, the system computes the output by aggregating the weighted contributions of the 3D fuzzy rules, where each rule involves a combination of 3D fuzzy sets defined for each spatial input. This structure allows the system to capture the spatial and temporal dynamics present in the distributed parameter system.

References

  1. Wei, P.; Li, H.-X. Spatiotemporal Transformation-Based Neural Network with Interpretable Structure for Modeling Distributed Parameter Systems. IEEE Trans. Neural Netw. Learn. Syst. 2024, 36, 729–737. [Google Scholar] [CrossRef]
  2. Xu, K.; Fan, B.; Yang, H.; Hu, L.; Shen, W. Locally Weighted Principal Component Analysis-Based Multimode Modeling for Complex Distributed Parameter Systems. IEEE Trans. Cybern. 2022, 52, 10504–10514. [Google Scholar] [CrossRef] [PubMed]
  3. Zhu, C.; Yang, H.; Jin, X.; Xu, K.; Li, H. Multilayer Online Sequential Reduced Kernel Extreme Learning Machine-Based Modeling for Time-Varying Distributed Parameter Systems. IEEE Trans. Cybern. 2024, 54, 624–634. [Google Scholar] [CrossRef] [PubMed]
  4. Xu, B.; Lu, X. Application of a Stochastic Gradient Descent-Based SVR Online Modeling Method to Time-Varying Forging Processes. IEEE Sens. J. 2023, 23, 15444–15453. [Google Scholar] [CrossRef]
  5. Zhu, C.; Xie, Y.; Yang, H.; Li, Z.; Hu, L.; Xu, K. A Nonlinear Spatiotemporal Modeling Method Combined with T-Distributed Stochastic Neighbor Embedding and Broad Learning System for the Lithium-Ion Battery Thermal Process. Eng. Appl. Artif. Intell. 2024, 135, 108433. [Google Scholar] [CrossRef]
  6. Chen, J.; Xiao, X.; Feng, X. A Mixed Immersed Finite Element Method for Fourth-Order Interface Problems on Surfaces. Comput. Math. Appl. 2024, 176, 122–139. [Google Scholar] [CrossRef]
  7. Mengi, Y.; Korkut, F. Interaction Analysis Revisited by Generalized Finite Difference Method with Perfectly Matched Layer. Soil Dyn. Earthq. Eng. 2023, 170, 107923. [Google Scholar] [CrossRef]
  8. Deng, H.-P.; Wang, B.-C.; Li, H.-X. Physics-Informed Spatial Fuzzy System and Its Applications in Modeling. IEEE Trans. Fuzzy Syst. 2024, 32, 5951–5962. [Google Scholar] [CrossRef]
  9. Tan, T.; Zhao, T. A Data-Driven Fuzzy System for the Automatic Determination of Fuzzy Set Type Based on Fuzziness. Inf. Sci. 2023, 642, 119173. [Google Scholar] [CrossRef]
  10. Xu, K.-K.; Yang, H.-D.; Zhu, C.-J. A Novel Extreme Learning Machine-Based Hammerstein-Wiener Model for Complex Nonlinear Industrial Processes. Neurocomputing 2019, 358, 246–254. [Google Scholar] [CrossRef]
  11. Lu, X.; Zou, W.; Huang, M. An Adaptive Modeling Method for Time-Varying Distributed Parameter Processes with Curing Process Applications. Nonlinear Dyn. 2015, 82, 865–876. [Google Scholar] [CrossRef]
  12. Wang, B.-C.; Li, H.-X. A Sliding Window Based Dynamic Spatiotemporal Modeling for Distributed Parameter Systems with Time-Dependent Boundary Conditions. IEEE Trans. Ind. Inf. 2019, 15, 2044–2053. [Google Scholar] [CrossRef]
  13. Chen, L.; Shen, W.; Zhou, Y.; Mou, X.; Lei, L. Learning-Based Sparse Spatiotemporal Modeling for Distributed Thermal Processes of Lithium-Ion Batteries. J. Energy Storage 2023, 69, 107834. [Google Scholar] [CrossRef]
  14. Jin, X.; Wu, D.; Yang, H.; Zhu, C.; Shen, W.; Xu, K. A Temporal–Spatiotemporal Domain Transformation-Based Modeling Method for Nonlinear Distributed Parameter Systems. J. Comput. Des. Eng. 2023, 10, 1267–1279. [Google Scholar] [CrossRef]
  15. Zhang, X.-X.; Zhao, L.-R.; Li, H.-X.; Ma, S.-W. A Novel Three-Dimensional Fuzzy Modeling Method for Nonlinear Distributed Parameter Systems. IEEE Trans. Fuzzy Syst. 2019, 27, 489–501. [Google Scholar] [CrossRef]
  16. Li, H.-X.; Qi, C. Modeling of Distributed Parameter Systems for Applications—A Synthesized Review from Time–Space Separation. J. Process Control. 2010, 20, 891–901. [Google Scholar] [CrossRef]
  17. Liu, Y.; Wu, X.; Shen, Y. Automatic Clustering Using Genetic Algorithms. Appl. Math. Comput. 2011, 218, 1267–1279. [Google Scholar] [CrossRef]
  18. Tang, J.; Deng, C.; Huang, G.-B. Extreme Learning Machine for Multilayer Perceptron. IEEE Trans. Neural Netw. Learn. Syst. 2016, 27, 809–821. [Google Scholar] [CrossRef]
  19. Theodoropoulou, A.; Adomaitis, R.A.; Zafiriou, E. Model Reduction for Optimization of Rapid Thermal Chemical Vapor Deposition Systems. IEEE Trans. Semicond. Manuf. 1998, 11, 85–98. [Google Scholar] [CrossRef]
  20. Li, H.-X. A Three-Dimensional Fuzzy Control Methodology for a Class of Distributed Parameter Systems. IEEE Trans. Fuzzy Syst. 2007, 15, 470–481. [Google Scholar] [CrossRef]
  21. Zhang, X.; Yan, R.; Zhou, G.; Wang, L.; Wang, B. Online Three-Dimensional Fuzzy Reinforcement Learning Modeling for Nonlinear Distributed Parameter Systems. Electronics 2024, 13, 4217. [Google Scholar] [CrossRef]
  22. Deng, W.-Y.; Qu, Y.-T.; Zhang, Q. Domain Adaption Based on ELM Autoencoder. Math. Probl. Eng. 2017, 2017, 1239164. [Google Scholar] [CrossRef]
  23. Deng, Z.; Chan, S.H.; Chen, Q.; Liu, H.; Zhang, L.; Zhou, K.; Tong, S.; Fu, Z. Efficient Degradation Prediction of PEMFCs Using ELM-AE Based on Fuzzy Extension Broad Learning System. Appl. Energy 2023, 331, 120385. [Google Scholar] [CrossRef]
  24. Liu, Z.; Li, H.-X. Extreme Learning Machine Based Spatiotemporal Modeling of Lithium-Ion Battery Thermal Dynamics. J. Power Sources 2015, 277, 228–238. [Google Scholar] [CrossRef]
  25. Qi, C.; Li, H.-X. A Karhunen−Loève Decomposition-Based Wiener Modeling Approach for Nonlinear Distributed Parameter Processes. Ind. Eng. Chem. Res. 2008, 47, 4184–4192. [Google Scholar] [CrossRef]
  26. Qi, C.; Li, H.-X.; Zhang, X.; Zhao, X.; Li, S.; Gao, F. Time/Space-Separation-Based SVM Modeling for Nonlinear Distributed Parameter Processes. Ind. Eng. Chem. Res. 2011, 50, 332–341. [Google Scholar] [CrossRef]
Figure 1. System structure of RTCVD.
Figure 1. System structure of RTCVD.
Mathematics 13 00364 g001
Figure 2. Proposed 3D fuzzy modeling framework based on automatic clustering and HELM.
Figure 2. Proposed 3D fuzzy modeling framework based on automatic clustering and HELM.
Mathematics 13 00364 g002
Figure 3. The framework of 3D fuzzy model.
Figure 3. The framework of 3D fuzzy model.
Mathematics 13 00364 g003
Figure 4. Fitness function iteration process.
Figure 4. Fitness function iteration process.
Mathematics 13 00364 g004
Figure 5. HELM network structure.
Figure 5. HELM network structure.
Mathematics 13 00364 g005
Figure 6. The flowchart of the proposed 3D fuzzy modeling method.
Figure 6. The flowchart of the proposed 3D fuzzy modeling method.
Mathematics 13 00364 g006
Figure 7. Membership function of the premise part of 3D fuzzy rules.
Figure 7. Membership function of the premise part of 3D fuzzy rules.
Mathematics 13 00364 g007
Figure 8. The first six spatial basis functions.
Figure 8. The first six spatial basis functions.
Mathematics 13 00364 g008
Figure 9. The real temperature distribution and predicted output of the proposed model on the training set.
Figure 9. The real temperature distribution and predicted output of the proposed model on the training set.
Mathematics 13 00364 g009
Figure 10. The real temperature distribution and predicted output of the proposed model on the test set.
Figure 10. The real temperature distribution and predicted output of the proposed model on the test set.
Mathematics 13 00364 g010
Figure 11. Prediction error and relative error of the proposed model.
Figure 11. Prediction error and relative error of the proposed model.
Mathematics 13 00364 g011aMathematics 13 00364 g011b
Figure 12. TNAE comparison of the different methods.
Figure 12. TNAE comparison of the different methods.
Mathematics 13 00364 g012
Figure 13. RLNE comparison of the different methods.
Figure 13. RLNE comparison of the different methods.
Mathematics 13 00364 g013
Table 1. Parameter setting of automatic clustering based on GA.
Table 1. Parameter setting of automatic clustering based on GA.
ParametersValueDescription
β 8Selection Pressure
γ 0.2Control the randomness and diversity of the crossover process
μ 0.02Mutation rate
Pop100Population size
N i t e r 100Maximum Number of Iterations
N i n d 11The number of genes per individual
Table 2. Comparison of RMSE performance of different methods.
Table 2. Comparison of RMSE performance of different methods.
MethodTraining RMSETesting RMSE
KL-LS2.44722.1649
KL-LS-SVM1.63172.2634
Proposed model1.33491.3093
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhou, G.; Zhang, X.; Wang, T.; Wang, B. A Spatiotemporal Fuzzy Modeling Approach Combining Automatic Clustering and Hierarchical Extreme Learning Machines for Distributed Parameter Systems. Mathematics 2025, 13, 364. https://doi.org/10.3390/math13030364

AMA Style

Zhou G, Zhang X, Wang T, Wang B. A Spatiotemporal Fuzzy Modeling Approach Combining Automatic Clustering and Hierarchical Extreme Learning Machines for Distributed Parameter Systems. Mathematics. 2025; 13(3):364. https://doi.org/10.3390/math13030364

Chicago/Turabian Style

Zhou, Gang, Xianxia Zhang, Tangchen Wang, and Bing Wang. 2025. "A Spatiotemporal Fuzzy Modeling Approach Combining Automatic Clustering and Hierarchical Extreme Learning Machines for Distributed Parameter Systems" Mathematics 13, no. 3: 364. https://doi.org/10.3390/math13030364

APA Style

Zhou, G., Zhang, X., Wang, T., & Wang, B. (2025). A Spatiotemporal Fuzzy Modeling Approach Combining Automatic Clustering and Hierarchical Extreme Learning Machines for Distributed Parameter Systems. Mathematics, 13(3), 364. https://doi.org/10.3390/math13030364

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop