Kolmogorov–Arnold Networks for System Identification of First- and Second-Order Dynamic Systems ^†

Abstract

System identification—originating in the 1950s from statistical theory—has since developed a wealth of algorithms, insights, and practical expertise. We introduce Kolmogorov–Arnold neural networks (KANs) as an interpretable alternative for model discovery. Leveraging KANs’ inherent property to approximate data and interpret it by employing learnable activation functions and decomposition of multivariate mappings into univariate transforms, we test its ability to recover the step responses of first- and second-order systems both numerically and symbolically. We employ synthetic datasets, both noise-free and with Gaussian noise, and find that KANs can achieve very low RMSE and parameter error with simple architectures. Our results demonstrate that KANs combine ease of implementation with symbolic transparency, positioning them as a compelling bridge between classical identification and modern machine learning.

1. Introduction

Understanding and modeling dynamic systems is a fundamental part of control engineering, data processing, and science-centered problems as a whole. Correctly identifying the underlying systems from observational data, known as system identification, is essential. Accurate methods for system identification directly help engineers develop, simulate, control, and optimize real-world systems. The field has its origins in the 1950s [1], grounded in statistical theory. It has grown dramatically over the decades, and a base of great theoretical foundations of algorithms and techniques has been developed. Some of the fundamental aspects of the traditional approach to system identification, focusing on the core components that define the field, include the following: (1) the system’s observed input–output data, (2) a set of parameterized candidate models, known as the model structure, (3) an estimation technique for fitting model parameters to the data, and (4) a validation process to guide the selection of an appropriate model. Among these, choosing the right model structure is especially critical. Recursive models, such as the ARX model—a linear difference equation relating inputs and outputs—are commonly used in linear system identification. They serve as universal approximators for linear systems, capable of achieving high accuracy given sufficiently high model orders. However, selecting appropriate structural parameters is essential to ensure a good fit [2].

Classical techniques like the Prediction Error Method, Least Squares, and Subspace Identification Methods provide reliable ways for parameter estimation of linear systems [3]. However, they can prove to be insufficiently accurate when dealing with highly nonlinear dynamics, systems with mostly unknown internal structures, or heavily noisy data [2,4]. As a result, neural networks are gaining popularity as an alternative due to their ability to approximate complex nonlinear mappings with arbitrary precision [4,5]. Architectures such as feedforward neural networks, recurrent neural networks, and physics-informed neural networks have been applied to a wide range of system identification problems with promising results [4,5]. These models, however, often suffer from a lack of interpretability, sensitivity to hyperparameter tuning, and challenges with generalization.

Kolmogorov–Arnold Networks (KANs), inspired by the Kolmogorov–Arnold representation theorem, offer a novel framework that decomposes multivariate functions into compositions of univariate nonlinear transformations and additive operations. Unlike conventional deep neural networks that rely on learned weights and fixed activation functions, KANs can learn both the structure and the nonlinearities of the system, enabling more interpretable and potentially more accurate approximations with fewer parameters [6].

There has been a rapid increase in the literature on the topic of system identification with neural networks in the last few years, with some notable contributions from the deep learning community. For instance, non-asymptotic learning theory has been applied to analyze the performance of neural estimators in system identification [7]. Physics-informed models have also been shown to incorporate prior knowledge into models effectively, and as a result, they reduce the amount of data needed and improve the overall fit of models [8,9]. However, interpretability still continues to be a challenge in the field. Most neural networks operate as black boxes, and extracting the internal mechanisms of deep learning models remains difficult. KANs can directly attack this issue by providing easy-to-understand, readable models due to their structural features.

This study explores the potential of KANs for the purpose of system identification. We create synthetic datasets of both clean and noisy time-response of first- and second-order systems and approximate them by training KAN models and then interpret the underlying equations, which describe the time-response of the systems, by making use of the architecture’s inherent capabilities for symbolic representation of the activation functions. We then compare the modeled responses and equations to the real ones to gauge the potential for future use for real-world scenarios. The results are promising and point to the conclusion that KANs might prove to be a solid alternative—they are easy to implement, reach sufficient levels of accuracy using only a relatively small number of parameters, and provide opportunity for easy and intuitive interpretation of unknown systems.

2. Methods

2.1. Kolmogorov–Arnold Networks as an Approximation Tool

In comparison to MLPs, which are based on the universal approximation theorem [10], KANs [6] are based on the Kolmogorov–Arnold representation theorem [11], which states that any multivariate function $f\left(x\right) : [0, 1]n \to \mathbb{R}$ that is continuous and smooth can be represented by a finite sum of continuous univariate functions, as follows:

$$f\left(\mathbf{x}\right)=f\left(x_1,\dots ,x_n\right)=\sum _{q=1}^{2n+1}{\mathsf{\Phi }}_q\left(\sum _{p=1}^n{\varphi }_{q,p}\left(x_p\right)\right),$$

(1)

In this formulation, where ${\varphi }_{q,p} : \left[0, 1\right]\to \mathbb{R}$ and ${\mathsf{\Phi }}_q : \mathbb{R}\to \mathbb{R}$, each ${\varphi }_{q,p}$ is a univariate function, and the outer functions ${\mathsf{\Phi }}_q$ combine the inner univariate components to reconstruct $f\left(\mathbf{x}\right)$. Equation (1) exactly describes a 2-layer KAN with a hidden layer width of $2n + 1$ and an output dimension of 1. The Kolmogorov–Arnold representation theorem is extended to a more general KAN structure of arbitrary depth and width by making use of the structure of Equation (1) and stacking individual layers [6], exploiting the hierarchical structure of Equation (1) [6].

$${\mathbf{x}}_{l+1}={\mathsf{\Phi }}_l{x}_l=\left(\begin{array}{cccc}{\varphi }_{l,\mathrm{1,1}}(\cdot )& {\varphi }_{l,\mathrm{1,2}}(\cdot )& \dots & {\varphi }_{l,1,nl}(\cdot )\\ {\varphi }_{l,\mathrm{2,1}}(\cdot )& {\varphi }_{l,\mathrm{2,2}}(\cdot )& \dots & {\varphi }_{l,2,nl}(\cdot )\\ ⋮& ⋮& \ddots & ⋮\\ {\varphi }_{l,nl+\mathrm{1,1}}(\cdot )& {\varphi }_{l,nl+\mathrm{1,2}}(\cdot )& \dots & {\varphi }_{l,nl+1,nl}(\cdot )\end{array}\right){\mathbf{x}}_l$$

(2)

${\mathsf{\Phi }}_l$ is the function matrix corresponding to the $l^{th}$ KAN layer. A general KAN network can be composed of $L$ layers: given an input vector ${\mathbf{x}}_0 \in {\mathbb{R}}^{n_0}$, the output of KAN is as follows:

$$KAN\left(\mathbf{x}\right)=\left({\mathsf{\Phi }}_{L-1}\circ {\mathsf{\Phi }}_{L-2}\circ \dots \circ {\mathsf{\Phi }}_1\circ {\mathsf{\Phi }}_0\right)\mathbf{x}$$

(3)

A single layer with $n_{in}$-dimensional inputs and $n_{out}$-dimensional outputs can be described as a matrix of 1D functions as follows:

$$\mathsf{\Phi }=\left\{{\varphi }_{q,p}\right\}, p=\mathrm{1,2},\dots ,n_{in} ,\hspace{1em}q=\mathrm{1,2}, \dots ,n\_out$$

(4)

Each ${\varphi }_{q,p}$ has trainable parameters. In the original Kolmogorov–Arnold theorem, the inner functions form a layer with $n_{in} = n$ and $n_{out} = 2n + 1$, while the outer functions form another layer with $n_{in} = 2n + 1$ and $n_{out} = 1$. Because these operations are all differentiable, KANs can be trained through standard backpropagation [12,13].

The activation function $\varphi \left(x\right)$ for a layer is defined as follows:

$$\varphi \left(x\right)=w_{b}b\left(x\right)+w_{s}spline\left(x\right)$$

(5)

where $b\left(x\right)$ is a basis function and $spline\left(x\right)$ is a spline function of the form as follows:

$$b\left(x\right)=silu\left(x\right)={\displaystyle \frac{x}{1+e^{-x}}}$$

(6)

$$spline\left(x\right)=\sum _i{c}_i{B}_i\left(x\right)$$

(7)

The weights $w_b$, $w_s$, and the coefficients $c_j$ are all learned during training, while the grid resolution and polynomial degree of the spline basis $B_i\left(x\right)$ are treated as user-tuned hyperparameters.

The second revision of the architecture introduces multiplication operations to help with the approximation of multiplicative structures. This version is named MultKAN, and the difference lies in the transformation from one layer’s subnodes to the next layer’s nodes. In the new version, addition nodes are directly copied from corresponding subnodes, and multiplication nodes realize multiplication on $k$ subnodes from the previous layer [13].

A MultKAN layer is composed of a standard KAN layer, denoted as $\mathsf{\Phi }ₗ$, and a multiplication layer, $\mathbf{M}ₗ$. The KAN layer Φₗ takes an input vector $\mathbf{x}ₗ \in \mathbb{R}ⁿₗᵃ⁺ⁿₗᵐ$ and produces an output ${\mathit{z}}_{\mathrm{l}} = {\mathsf{\Phi }}_{\mathrm{l}}\left({\mathit{x}}_{\mathrm{l}}\right) \in {{{\mathbb{R}}^{\mathrm{n}}}_{\mathrm{l}+1}}^{\mathrm{a}+}{{2^{\mathrm{n}}}_{\mathrm{l}+1}}^{\mathrm{m}}$. The multiplication layer has two components: one that performs pairwise multiplications between subnodes and another that applies the identity transformation. The entire MultKAN layer can be concisely expressed as $\mathsf{\Psi }ₗ \equiv \mathbf{M}ₗ ◦ \mathsf{\Phi }ₗ$. Therefore, the full MultKAN network is structured as follows:

$$MultKAN\left(\mathbf{x}\right)=\left({\mathsf{\Psi }}_L\circ {\mathsf{\Psi }}_{L-1}\circ \dots \circ {\mathsf{\Psi }}_1\circ {\mathsf{\Psi }}_0\right)\mathbf{x}$$

(8)

2.2. Kolmogorov–Arnold Networks for System Identification

In this paper, we investigate the use of Kolmogorov–Arnold Networks for the purposes of system identification by looking at two cases—first- and second-order dynamic systems—and finding the parameters and underlying functions that describe their input-output relationships.

2.2.1. Datasets

We generate all datasets synthetically using the open-source pykan library. The datasets represent the time domain response of two canonical dynamic systems to a unit step input:

• First-order system: Modeled by the transfer function $G_1\left(s\right)={\displaystyle \frac{K}{\tau s+1}}$ with a time-domain transient characteristic of the kind $K\left(1 - e^{{\displaystyle \frac{-t}{\tau }}}\right)$. We test for the following parameter values: $K=2, \tau =10;20;40;50$.
• Second-order underdamped system: Modeled by the transfer function $G_{2\left(s\right)}={\displaystyle \frac{K{\omega }_n^2}{s^2+2\zeta {\omega }_{n}s+{\omega }_n^2}}$ with a time-domain transient characteristic of the kind $x(t)=1-e^{-\zeta {\omega }_{n}t}\left(cos\left({\omega }_{d}t\right)+{\displaystyle \frac{\zeta }{\sqrt{\left(1-{\zeta }^2\right)}}}sin\left({\omega }_{d}t\right)\right)$. We test for the following values: $\zeta =0.5; 0.25; 0.1, {\omega }_n=1$.

For the first-order system, we simulate two variants—a noise-free response and a noisy response, where zero-mean Gaussian white noise is added to the output signal [14], and for the second-order system in this study, we explore only the noise-free variant.

Each dataset consists of 10,000 discrete time samples, recorded at a fixed sampling interval of 0.01 s. We split each dataset into 8000 training samples and 2000 testing samples.

2.2.2. Network Architecture

For the first-order systems, we use the standard KAN implementation with only addition on the nodes. The starting architecture has the shape [1,5,1], and we train it with regularization and grid extension to reach the final form of [1,1,1] after pruning. The starting size was chosen after some initial testing of wider and deeper networks proved to be inefficient and led to unsatisfactory loss parameters.

For the second-order systems, we employ the MultKAN with starting architecture [1,[0,6],1], and we again train with regularization and grid extension to reach a final form of [1,[0,2],1] after pruning. A comparison of initial and final architecture can be seen in Figure 1.

2.2.3. Training Procedure

The training parameters we use include the following:

• Loss function: We use mean squared error (MSE) between the predicted output and the true system response.
• Optimizer: LBFGS [15] with a learning rate of 0.1.
• Regularization coefficient: For both first- and second-order systems, we use regularization coefficient lamb = 0.0001, which corresponds to the λ in the following equation:

  $$l_{total}=l_{pred}+\lambda \left({\mu }_1\sum _{l=0}^{L-1}{\left|{\mathsf{\Phi }}_1\right|}_1+{\mu }_2\sum _{l=0}^{L-1}S\left({\mathsf{\Phi }}_l\right)\right),$$

  (9)
  And controls the overall regularization magnitude as follows:
• Grid extension: We start at grid size 3 and employ grid extension between trainings as suggested in the original KAN paper [6] to deal with loss plateaus and improve accuracy.
• Batch size: We train on the whole dataset.

Each model is trained separately on the first-order or second-order datasets, with and without noise, yielding three distinct KAN instances. After the initialization of the model and a grid extension cycle with regularization, we prune the insignificant edges and nodes and run one more training cycle with the final model shape before evaluating and fixing the symbolic functions on the edges and running final trainings. This process proves successful for both first- and second-order systems with slight parameter adjustments, necessary due to the difference in model size.

2.2.4. Evaluation Metrics

We evaluate identification performance with the help of the root mean square error (RMSE) on the test set, the parameter estimation error (the absolute difference between true and learned values of the parameters of the first- and second-order systems), and a comparison of the graphs of simulated and real outputs. We also use the library’s capabilities for extracting the symbolic formulas and presenting them in a readable format to compare to the initial equations we selected for the creation of the synthetic datasets. These metrics allow us to assess both the accuracy of the time domain predictions and the fidelity of the recovered system parameters.

3. Results

Our experiments demonstrate that KANs effectively recover both the time-domain behavior and the underlying symbolic models of first- and second-order systems under noise-free and noisy conditions. The original equation and the one recovered by the network match exactly.

Time domain accuracy: Across all four KAN instances, the average test set RMSE remained around the order of ten to the minus fourth degree for unfixed activation functions and around ten to the minus eighth degree after fixing the appropriate functions for both the noise-free models and the noisy ones—

Table 1. Test loss results and parameter recovery for first-order system.

System Parameters	Test Loss (Noise-Free and with Noise)	Parameter Recovery (Noise-Free)
$K=2;\tau =10$	$3.67 \times$ and $5.12\times {10}^{-3}$	$K=$1.99999983330918 $\tau =9.99999819211726$
$K=2;\tau =20$	$4.52\times {10}^{-5}$ and $5.14\times {10}^{-3}$	$K=1.99978709220886;$ $\tau =20.0180257893$
$K=2;\tau =50$	$1.48\times {10}^{-7}$ and $5.22\times {10}^{-3}$	$K=1.99999990551827;$ $\tau =50.0000069833$

and

Table 2. Test loss results and parameter recovery for second-order system.

System Parameters	Test Loss	Parameter Recovery
${\omega }_n=1;\zeta =0.5$	$7.94\times {10}^{-8}$	${\omega }_n=0.99999998;\zeta =0.5$
${\omega }_n=1;\zeta =0.25$	$1.81\times {10}^{-7}$	${\omega }_n=1;\zeta =0.25$
${\omega }_n=1;\zeta =0.1$	$3.58\times {10}^{-6}$	${\omega }_n=0.994961857795715;$ $\zeta =0.099986051359080$

.

Parameter recovery: Learned coefficients were accurate to the seventh decimal place for the noise-free datasets and to the fifth decimal place for the noisy datasets—the recovered equations yield the same time-domain plots as shown in Figure 2 and Figure 3.

Symbolic function extraction: The extracted symbolic equations matched the original time-domain response equations of each system exactly, confirming correct structural recovery.

We also observed that KANs can be treated as a filter for noisy data, recovering the underlying system dynamics despite the noise.

Comparison with MLPs and ARX Kolmogorov–Arnold Networks for System Identification

We trained an MLP-based network implemented in PyTorch 2.2.2 on the same datasets. The MLP is designed as a fully connected feedforward neural network composed of configurable hidden layers with ReLU activation functions. The architecture is of the shape [1,32,32,1] for the first-order systems and [1,64,64,1] for the second-order systems. We train with the Adam optimizer [16], a learning rate of $1\times {10}^{-3}$, and the MSE loss function. The test error for noise-free first- and second-order datasets is, respectively, $7.6\times {10}^{-5}$ and $9.2\times {10}^{-5}$; for noisy first-order datasets, it is $2.298\times {10}^{-3}$; and for noisy second-order datasets, it is $2.544\times {10}^{-3}$.

Our tests concluded that KANs can reach sufficient accuracy with fewer network parameters than MLPs, and while the numerical results obtained are comparable and plot comparisons of estimated and actual output are satisfactory, as seen in Figure 4, there’s no direct possibility for model interpretation in the MLP case. Without employing further identification techniques, MLP models remain a black box.

These results confirm our hypothesis that KANs can both approximate dynamic responses accurately and reveal the underlying model equations.

4. Discussion

We find that KANs can be a competitive alternative to traditional identification techniques by combining high predictive accuracy with symbolic interpretability while using a relatively small number of parameters. This architecture offers both interpretability and transparency by allowing us to extract the closed-form equations directly and solve the black-box nature problem of other methods. Their robustness to synthetically added noise also promises that they could offer a solution for real-world problems.

Our study has mostly exploratory character, as its scope is quite limited—we test only for single-input and single-output systems and use only synthetic data. Expanding to multivariable or systems of higher order will most certainly increase model complexity, training time, and parameter count. We also observe that the standard machine learning intuition that helps for hyperparameter tuning often does not directly transfer when it comes to KANs.

Future study might include investigation of KAN’s efficiency for system identification on noisy datasets for systems of higher order or real-world datasets, including multi-input multi-output systems and nonlinear dynamics.

Overall, KANs offer a promising framework for system identification, marrying data-driven modeling with interpretable symbolic output.