Time-Series Modeling Based on a Modified Volterra Neural Network

Abstract

This paper proposes a novel neural network model that integrates a modified Volterra digital filter with a feedforward neural network for time-series modeling. In the proposed architecture, all input signals in the conventional Volterra filter are replaced by corresponding output signals, since time-series problems typically consist of observable output sequences over time without explicit external inputs. These output signals, together with their cross-product terms, are constructed as input vectors for the feedforward neural network. To optimize the network parameters, including weights and thresholds, the well-known particle swarm optimization (PSO) algorithm is employed. Based on the proposed PSO-trained neural network model, two types of time series are investigated: chaotic time series and financial time series involving exchange rates. For each case, multiple independent runs with different initial conditions are conducted to ensure the robustness of the proposed method. Furthermore, the effects of varying filter orders and population sizes on modeling performance are also examined.

1. Introduction

To successfully solve most system-modeling problems, three steps are typically completed: (i) collection of input–output pairs of the system, (ii) construction of a suitable mathematical model, and (iii) development of a parameter adjustment mechanism. First, an appropriate mathematical model is established. The input signal, identical to the collected system input, is then applied to the model to generate the corresponding output. For system modeling, it is necessary to compute the error signals between the model outputs and the actual system outputs. Subsequently, a parameter adjustment mechanism is employed to update the model parameters so as to minimize these errors. In other words, the model output is expected to approximate the actual system output as closely as possible through iterative optimization. Once the mathematical model has been successfully trained, it can produce outputs identical or very close to those of the original system when the same input signal is applied.

Neural network models are among the most commonly used mathematical tools for system modeling due to their strong learning capability and the large number of weights connecting neurons across different layers. Recently, extensive studies on and applications of neural network models have been widely reported in the literature [1,2,3,4,5,6,7,8,9,10,11]. In [1], for example, the author proposed a novel robust model predictive control scheme using neural networks. In the proposed structure, the neural network model is employed both to construct the nominal plant model and capture plant uncertainties, respectively. A pneumatic servomechanism example was successfully tested and validated using the proposed neural-network-based predictive control scheme. In [3], an artificial neural network model was developed based on data recorded by wave-monitoring instrumentation for assessing wave energy potential at a given site. The results obtained via simulation showed that the neural network can effectively utilize the measured data and is useful for characterizing the wave energy resources of coastal regions. Furthermore, a hybrid model combining an artificial neural network and a Bayesian network was developed for liquidity risk assessment in banking. A real-world case study was conducted to demonstrate its applicability and effectiveness [5]. In our previous work [7], a modeling scheme based on a Volterra system combined with a neural network was proposed for nonlinear discrete-time dynamic systems. To evaluate modeling performance, a discrete bilinear system and a nonlinear time-varying discrete system were considered. All numerical results demonstrated that the proposed method outperformed the standard back-propagation algorithm.

To address system-modeling or time-series-modeling problems, another key factor that must be considered is the selection of an appropriate updating mechanism for tuning the adjustable parameters within the developed model. In this study, the well-known particle swarm optimization (PSO) algorithm is adopted [12]. There are two advantages related to the PSO algorithm: real-valued representation and simple updating rules. It has been demonstrated to be an effective and efficient method for solving various engineering optimization problems, such as optimization of the stacking sequence of laminated composite materials [13], wireless network optimization in vertical handover scenarios [14], geophysical inverse problems [15], capacitated vehicle-routing problems [16], and heliostat field layout optimization [17], among others [18,19,20,21,22]. The effectiveness of the PSO algorithm has been validated through both numerical simulations and practical implementation.

This paper mainly focuses on time-series modeling. It is well known that a time series consists solely of a sequence of observable output data indexed according to time. Consequently, the structure of the corresponding mathematical model differs from that used in general system modeling, as no explicit input signals are involved. In the classical Volterra digital structure, the output is determined by the current input, past inputs, and their cross-product terms [23,24,25,26,27]. However, for time-series modeling, a modified Volterra digital filter is developed in which all input signals are replaced by the corresponding output signals. In addition, to enhance modeling capability, the proposed structure is further combined with a feedforward neural network, where the input vector consists of past output signals and their cross-product terms. Detailed descriptions are provided in the subsequent section.

The remainder of this paper is organized as follows. Section 2 presents the three model structures, including the classical Volterra digital filter, its modified version, and their integration with a feedforward neural network. Section 3 describes the PSO algorithm in detail and presents the complete design procedure for time-series modeling based on the proposed PSO-tuned neural network model. Section 4 considers two time-series modeling problems: chaotic time series and financial exchange rate series. Various experiments are conducted, including analyses of initial conditions, system orders, and population sizes. Finally, Section 5 presents conclusions and discusses possible future research directions.

2. The Proposed Neural Network Model

2.1. Classical Volterra Digital Filter

Digital filters are generally composed of two main categories: finite impulse response (FIR) and the infinite impulse response (IIR) filters. The output of the FIR is influenced only by the present and past input signals. As a result, such a system is always stable when the filter coefficients are assigned arbitrarily. Conversely, in the IIR structure, the output signal is dominated not only by the present and past inputs but also by its past output signals. The stability of an IIR filter must be guaranteed by carefully choosing a set of suitable filter coefficients. Basically, the classical Volterra digital filter is a nonlinear model that is an extended version of FIR filter, and it can be expressed via the following difference equation [23,26]:

$$\begin{gathered} y[n]=h[0]+\sum _{k=1}^{N}h\left[k\right]x[n-k+1]+\sum _{k_1=0}^{N-1}\sum _{k_2=k_1}^{N-1}h[k_1,\hspace{0.33em}{k}_2]x[n-k_1]x[n-k_2] \\ =h\left[0\right]+h\left[1\right]x\left[n\right]+h\left[2\right]x[n-1]+\dots +h\left[N\right]x[n-N+1] \\ +h[0,\hspace{0.33em}0]x^2\left[n\right]+h[0,\hspace{0.33em}1]x\left[n\right]x[n-1]+\dots +h[0,\hspace{0.33em}N-1]x\left[n\right]x[n-N+1] \\ +\dots +h[N-1,\hspace{0.33em}N-1]x^2[n-N+1], \end{gathered}$$

(1)

where x represents the input signal; y is the output signal; N represents the number of past inputs required, i.e., the system’s memory capacity; $h\left[k\right]$ stands for the filter coefficient corresponding to the present and past inputs; and $h[k_1,\hspace{0.33em}{k}_2]$ is also a filter coefficient, this time reflecting the quadratic property with respect to the cross-product terms of present and past inputs. It can easily be gleaned from Equation (1) that the Volterra digital filter is a nonlinear model and that its new output is influenced only by the input signals, not past output signals.

2.2. Modified Volterra Digital Filter

It is well known that a time series is a collection of output data correlated with time and that it does not involve input signals. The classical Volterra digital model mentioned above is unsuitable for solving the time-series modeling problem. Hence, based on the structure of Equation (1), in this paper, a modified Volterra digital filter is proposed, which is expressed as follows:

$$\begin{gathered} y[n]=h[0]+{\sum }_{k=1}^{N}h\left[k\right]y[n-k]+{\sum }_{k_1=1}^N{\sum }_{k_2=k_1}^{N}h[k_1,\hspace{0.33em}{k}_2]y[n-k_1]y[n-k_2] \\ =h\left[0\right]\cdot 1+h\left[1\right]y[n-1]+h\left[2\right]y[n-2]+\dots +h\left[N\right]y[n-N] \\ +h[1,\hspace{0.33em}1]y^2[n-1]+h[1,\hspace{0.33em}2]y[n-1]y[n-2]+\dots +h[1,\hspace{0.33em}N]y[n-1]y[n-N] \\ +\dots +h[N,\hspace{0.33em}N]y^2[n-N], \end{gathered}$$

(2)

where the past output signals completely substitute for the input signals of Equation (1). Such a modified Volterra version can also be regarded as a nonlinear autoregressive model in which the cross-product terms of past output signals are involved. In Equation (2), after a simple calculation, the number of filter coefficients can be obtained as follows:

$$L={\displaystyle \frac{(N+1)(N+2)}{2}}.$$

(3)

Moreover, in order to combine this model with the feedforward neural network, we collect all past outputs—and their cross-product terms—from Equation (2) as follows:

$$\begin{gathered} X=[x_1,\hspace{0.33em}{x}_2,\cdots ,x_L] \\ =[1,\hspace{0.33em}y[n-1],\cdots ,y[n-N],\hspace{0.33em}{y}^2[n-1],\hspace{0.33em}y[n-1\left]y\right[n-2],\cdots ,\hspace{0.33em}{y}^2[n-N\left]\right], \end{gathered}$$

(4)

where X represents the input vector for the neural network model.

2.3. Modified Volterra Neural Network Model

To enhance the modeling capacity for the time-series problem, a modified Volterra neural network model is developed in this study, where the vector X of Equation (4) is used as an input vector for the feedforward neural network. The proposed model, denoted as $FN{N}_{L-M-1}$, is schematically shown in Figure 1, where L is the number of input neurons (and is also given by Equation (3)); M denotes the number of neurons in the hidden layer that is assigned by the designer, with only one output neuron; $x_i$ is the network input signal, as given by Equation (4); $w\text{\_}x{h}_{ij}$ represents the weight connecting from the ith neuron of the input layer to the jth neuron of the hidden layer; $w\text{\_}h{y}_j$ is the weight from the jth neuron of the hidden layer to the single output neuron, all for $i=1,\hspace{0.33em}2,\dots ,L$, and $j=1,\hspace{0.33em}2,\dots ,M$; and $y_m$ is the final model output of the feedforward neural network.

In the hidden layer, the mathematical operation for each neuron can be described as follows:

$$net\text{\_}{h}_j={\sum }_{i=1}^{L}w\text{\_}x{h}_{ij}\cdot x_i-\theta \text{\_}{h}_j,$$

(5)

$$h_j=f(net\text{\_}{h}_j)={\displaystyle \frac{e^{net\text{\_}{h}_j}-e^{-net\text{\_}{h}_j}}{e^{net\text{\_}{h}_j}+e^{-net\text{\_}{h}_j}}},$$

(6)

where $net\text{\_}{h}_j$, $\theta \text{\_}{h}_j$, and $h_j$ for $j=1,\hspace{0.33em}2,\dots ,M$ represent the internal state, threshold, and output signal of the j-th hidden neuron, respectively, and $f(\cdot )$ is the so-called activation function; here, the hyperbolic tangent function is employed. In addition, only one output neuron is utilized in the proposed model, and the mathematical operations are described by Equations (7) and (8):

$$net\text{\_}y={\sum }_{j=1}^M{w}_{h{y}_j}\cdot h_j-{\theta }_y,$$

(7)

$$y_m=f\left(ne{t}_y\right)=ne{t}_y,$$

(8)

where $net\text{\_}y$ and $\theta \text{\_}y$ represent the internal state and the threshold of the output neuron, respectively, and $y_m$ is the final output of the proposed model, $FN{N}_{L-M-1}$, obtained via the linear activation function of Equation (8). To deal with system modeling, it is expected that the model output $y_m$ can approximate the actual series output $y_{series}$ for any instant by adjusting all the weights and thresholds within the neural network model. Under the architecture of $FN{N}_{L-M-1}$, the total of all the adjustable parameters can easily be calculated:

$$T=L\times M+M+M+1=M(L+2)+1.$$

(9)

It is clear that the total T is much more than the L in Equation (3) for the modified Volterra digital filter. In general, adding more adjustable parameters to a model always results in better modeling capacity. This is the main advantage of combining the modified Volterra digital filter with the feedforward neural network. Furthermore, the excellent PSO algorithm is adopted to design the model parameters in this study. Thus, we set

$$\begin{gathered} \Theta =[{\theta }_1,\hspace{0.33em}{\theta }_2,\dots ,{\theta }_T]=[w\text{\_}x{h}_{11},\dots ,w\text{\_}x{h}_{1M},\hspace{0.33em}w\text{\_}x{h}_{21},\dots ,w\text{\_}x{h}_{2M},\dots , \\ ,w\text{\_}x{h}_{L1},\dots ,w\text{\_}x{h}_{LM},\theta \text{\_}{h}_1,\dots ,\theta \text{\_}{h}_M,w\text{\_}h{y}_1,\dots ,w\text{\_}h{y}_M,\theta \text{\_}y], \end{gathered}$$

(10)

where $\Theta$ is the so-called particle of the algorithm, which consists of a collection of all the model parameters of $FN{N}_{L-M-1}$. Furthermore, a large number of particles form a population, and some evolving mechanisms are employed to achieve optimization.

3. Design Steps for Time-Series Modeling

The PSO algorithm was proposed by Kennedy and Eberhart in 1995 [12], and it is a popular and effective algorithm for solving different kinds of optimization problems. In the PSO algorithm, two important factors need to be considered: the individual best, pbest, for each particle and the global best, gbest, for the whole population. pbest is the best individual particle from the first iteration to the present, and gbest is the global best particle of the whole population until the present iteration. These two factors significantly guide the movement of all the particles toward the optimal or near-optimal system solution. Only two updating formulas are used to achieve optimization. One is the velocity-updating formula, and the other is the position formula, which can be described by Equations (11) and (12), respectively, for $i=1,\hspace{0.33em}2,\dots ,\hspace{0.33em}PS$ and $j=1,\hspace{0.33em}2,\dots ,\hspace{0.33em}T$, where PS represents population size:

$$v_{ij}(k+1)=w{v}_{ij}\left(k\right)+c_1{r}_1\left(pbes{t}_{ij}\right(k)-{\theta }_{ij}(k\left)\right)+c_2{r}_2\left(gbes{t}_j\right(k)-{\theta }_{ij}(k\left)\right),$$

(11)

$${\theta }_{ij}(k+1)={\theta }_{ij}\left(k\right)+v_{ij}(k+1),$$

(12)

where ${\theta }_{ij}\left(k\right)$, $pbes{t}_{ij}\left(k\right)$, and $gbes{t}_j\left(k\right)$ are the jth position components of the ith particle, the ith individual best, and the global best, respectively, in the kth iteration; $v_{ij}(k+1)$ and ${\theta }_{ij}(k+1)$ are the new jth velocity and position components of the ith particle; w is here called the inertia weight and is used to control the global and local search; $c_1$ and $c_2$ are two positive constants given by the designer; and $r_1$ and $r_2$ are two random numbers uniformly chosen from the interval [0, 1].

On the other hand, for time-series modeling, an objective function or a cost function of the system should be previously defined to evaluate the performance of each particle. Here, a simple mean square error (MSE) function was chosen, namely, Equation (13),

$$MSE={\displaystyle \frac{1}{NS}}{\sum }_{n=1}^{NS}{e}^2[n]={\displaystyle \frac{1}{NS}}{\sum }_{n=1}^{NS}{\left[y_{series}\left[n\right]-y_m\left[n\right]\right]}^2,$$

(13)

where NS represents the number of time-series data for modeling, $y_{series}\left[n\right]$ is the nth actual sample of time-series data, and $y_m\left[n\right]$ is the nth output sample of the modified Volterra neural network model, as described in Equation (8), all for $n=1,\hspace{0.33em}2,\dots ,\hspace{0.33em}NS$. It is expected that the MSE can be minimized via the tuning of the PSO algorithm; that is, the model output $y_m\left[n\right]$ approaches the actual output $y_{series}\left[n\right]$ for any instant n. This results in time-series modeling. As a summary, a design flowchart for time-series modeling is displayed in Figure 2 based on the modified neural network model with PSO tuning. Furthermore, the corresponding design steps are also listed.

• Required Data: A modified Voterra neural network model (as described previously), a sequence of known time-series data $y_{series}\left[n\right]$, the inertia weight w and positive constants $c_1$ and $c_2$ from Equation (11), the total of time-series data NS from Equation (13), the number of particles (population size) PS, and the allowable number of iterations G.
• Goal: The proposed modified Volterra neural network must successfully model a sequence of time-series data using PSO algorithm tuning.
• Step 1. Generate an initial population with PS particles that are produced from the interval $[-1,\hspace{0.33em}1]$ randomly.
• Step 2. Check the number of algorithm iterations. If the number is greater than G, then the algorithm must stop; otherwise, Step 3 is performed.
• Step 3. Evaluate the MSE of each particle using Equation (13), and based on the derived value, record the individual best, pbest, for each particle and the global best, gbest, for the whole population.
• Step 4. For each particle, employ the velocity- and position-updating formulas using Equations (11) and (12), respectively, to obtain new particle positions.
• Step 5. Go back to Step 2.

4. Some Simulations

To reveal the applicability and effectiveness of the developed neural network model with PSO tuning, two different kinds of time-series were simulated: chaotic series and financial exchange rate series. In the following, the fixed variables used in the algorithm are given by the number of neurons in the hidden layer ($M=10$), inertia weight ($w=0.8$), positive constants ($c_1=1.2$ and $c_2=1.2$) in Equation (11), and the allowable number of iterations ($G=2000$). The first experiment focuses on the influence of system order N on modeling performance, and the second experiment pertains to explaining the influence of the population size of the algorithm. For each case, 20 independent runs with different sets of initial conditions were performed to verify the robustness of the proposed scheme.

4.1. Chaotic Series Modeling

In this experiment, a simple chaotic system named the logistic map was employed to generate a sequence of chaotic data. The dynamic equation can be expressed using Equation (14) [28]:

$$y\left[n\right]=\mu \cdot y[n-1][1-y[n-1\left]\right],$$

(14)

where $0\le \mu \le 4$ is the control parameter of the chaotic system. When the control parameter $\mu =4$ and the initial value $y\left[0\right]\notin \{0,\hspace{0.33em}0.25,\hspace{0.33em}0.5,\hspace{0.33em}0.75,\hspace{0.33em}1\}$, the above equation is deterministic and displays chaotic dynamics. Furthermore, the output signal $y\left[n\right]$ is distributed in the range $(0,\hspace{0.33em}1)$ provided the initial value $y\left[0\right]\in (0,\hspace{0.33em}1)$. For the simulations, $y\left[0\right]=0.8$, and based on this value, a sequence of 100 chaotic data, i.e., $NS=100$, is generated for modeling. Two different cases are simulated: $N=5$ and $N=7$. It can easily be determined from Equations (3) and (9) that $L=21$ and $L=36$ and, further, $T=231$ and $T=381$, respectively. Obviously, the case where $N=7$ has many more model-adjustable parameters than the case where $N=5$. In general, modeling performance improves when there are more model parameters. In the simulation, the population size was set as follows: $PS=20$. After the proposed design method was employed, some simulation results were plotted, as shown in Figure 3 and Figure 4 and listed in

Table 1. MSE results for two different cases: $N=5$ and $N=7$ (* indicates the best solution among 20 different runs).

	N = 5 (L = 21 and T = 231)	N = 7 (L = 36 and T = 381)
Run 1	3.204 × 10−3	1.972 × 10−3
Run 2	3.742 × 10−3	3.432 × 10−3
Run 3	3.967 × 10−3	2.167 × 10−3
Run 4	3.489 × 10−3	2.788 × 10−3
Run 5	4.011 × 10−3	1.701 × 10−3
Run 6	3.923 × 10−3	1.573 × 10−3
Run 7	3.181 × 10−3	4.883 × 10−3
Run 8	4.503 × 10−3	3.706 × 10−3
Run 9	3.18 × 10−3	2.456 × 10−3
Run 10	2.815 × 10−3	2.32 × 10−3
Run 11	3.414 × 10−3	1.151 × 10−3
Run 12	3.326 × 10−3	1.537 × 10−3
Run 13	5.015 × 10−3	2.662 × 10−3
Run 14	4.802 × 10−3	1.736 × 10−3
Run 15	3.787 × 10−3	1.677 × 10−3
Run 16	3.669 × 10−3	1.541 × 10−3
Run 17	2.961 × 10−3	9.978 × 10−4 *
Run 18	4.049 × 10−3	4.782 × 10−3
Run 19	2.733 × 10−3 *	2.77 × 10−3
Run 20	4.177 × 10−3	2.284 × 10−3
Mean	3.697 × 10−3	2.407 × 10−3
Variance	3.73981 × 10−7	1.11854 × 10−6
Standard deviation	6.115 × 10−4	1.057 × 10−3
95% Confidence interval	[3.429 × 10−3, 3.965 × 10−3]	[1.943 × 10−3, 2.87 × 10−3]

, respectively. Table 1 provides a numerical comparison of 20 different runs for two cases: $N=5$ and $N=7$. Table 1 shows that the best modeling solution occurs in Run 19 for the case where $N=5$ and in Run 17 for $N=7$, and their MSE values are 2.735 × 10⁻³ and 9.978 × 10⁻⁴, respectively. Furthermore, the numerical means resulting from executing 20 different runs can eventually be derived as 3.697 × 10⁻³ and 2.407 × 10⁻³ for the cases where $N=5$ and $N=7$. These results clearly show that modeling performance improves when more adjustable parameters are given. Figure 3a and Figure 4b show comparisons between the real chaotic series output (dotted line) and model series output (solid line) with respect to the time n for the best simulation runs for $N=5$ and $N=7$, respectively. In addition, the performance index R squared (R²) for the modeling problem was determined: R² = 0.978860556 and R² = 0.992283987, respectively. The error signals with respect to time step n are plotted in Figure 3b and Figure 4b, respectively. As they show, both curves of the real and modeling outputs are almost identical. These results fully confirm the feasibility and superiority of the proposed design scheme.

4.2. Financial Exchange Rate Series Modeling

The second experiment involved modeling a sequence of physical financial exchange rate data corresponding to Japanese and Taiwanese currencies. All exchange rate series for the entire year of 2016 were collected, and there are 247 data corresponding to each day (taken from the Bank of Taiwan website: http://rate.bot.com.tw/). We sought to apply our method to model these time-series data. In this experiment, the influence of population size on modeling performance was examined by setting $PS=20$ and $PS=30$. The system order of $N=5$ was used for simulations. As in the above experiment, 20 independent runs with different initial conditions were performed for each case to verify robustness. All the numerical results obtained after the proposed method was applied are listed in

Table 2. MSE results for two different cases: $PS=20$ and $PS=30$ (* indicates the best solution among 20 different runs).

	$\mathit{P}\mathit{S}=20$	$\mathit{P}\mathit{S}=30$
Run 1	1.279 × 10−5	7.26 × 10−6
Run 2	1.706 × 10−5	5.14 × 10−6
Run 3	6.67 × 10−6	6.07 × 10−6
Run 4	5.41 × 10−6	6.15 × 10−6
Run 5	7.15 × 10−6	6.44 × 10−6
Run 6	7.96 × 10−6	6.24 × 10−6
Run 7	5.51 × 10−6	5.42 × 10−6
Run 8	9.08 × 10−6	6.77 × 10−6
Run 9	8.46 × 10−6	6.73 × 10−6
Run 10	6.32 × 10−6	7.7 × 10−6
Run 11	1.395 × 10−5	9.51 × 10−6
Run 12	5.12 × 10−6 *	1.034 × 10−5
Run 13	9.13 × 10−6	5.72 × 10−6
Run 14	6.56 × 10−6	6.15 × 10−6
Run 15	8.31 × 10−6	9.24 × 10−6
Run 16	5.34 × 10−6	4.92 × 10−6 *
Run 17	1.974 × 10−5	5.93 × 10−6
Run 18	6.76 × 10−6	5.07 × 10−6
Run 19	7.3 × 10−6	5.22 × 10−6
Run 20	7.54 × 10−6	5.61 × 10−6
Mean	8.808 × 10−6	6.581 × 10−6
Variance	1.529 × 10−11	2.237 × 10−12
Standard deviation	3.911 × 10−6	1.495 × 10−6
95% Confidence interval	[7.094 × 10−6, 1.052 × 10−5]	[5.925 × 10−6, 7.237 × 10−6]

, where the means of the derived MSE values are 8.808 × 10⁻⁶ and 6.581 × 10⁻⁶ for the cases $PS=20$ and $PS=30$, respectively. These results show that the modeling performance for $PS=30$ is slightly superior to that for $PS=20$. In addition, the best solution among 20 runs for the case where $PS=20$ occurs in Run 12 and in Run 16 for $PS=30$. Their MSE values are 5.12 × 10⁻⁶ and 4.92 × 10⁻⁶. Figure 5a further compares the physical exchange rate output and modeling output with respect to the whole year of 2016 for Run 12 for $PS=20$. In this case, the performance index R² was found to be 0.964350819. Figure 5a clearly shows that these two curves are very close. Also, the best modeling result for Run 16 for $PS=30$ is shown in Figure 6a, and the value of R² obtained was 0.965712804. In addition, Figure 5b and Figure 6b show their error signal curves. It can be concluded that the proposed scheme achieved a satisfactory outcome in modeling the exchange rate between currencies.

5. Conclusions

In this study, a novel modeling architecture for time-series data was developed. The proposed scheme modifies the classical Volterra digital filter to meet the requirements of time-series modeling. Based on this modified structure, a feedforward neural network model was constructed, where the input vector consists of past output values and their cross-product terms. This formulation introduces additional adjustable parameters, thereby enhancing modeling capability. An effective particle swarm optimization (PSO) algorithm was then employed to estimate the model parameters so that the network output could approximate the actual time-series output as closely as possible. In addition, the complete design procedure and a schematic flowchart were provided. To verify the applicability of the proposed method, two types of time-series modeling problems were considered: chaotic time series and financial exchange rate series. For each case, multiple independent runs were conducted to evaluate the robustness of the algorithm. Moreover, the effects of system order and population size on modeling performance were investigated. It can be concluded from the simulation results that the proposed neural network model combined with PSO tuning is effective for time-series modeling problems.