Optimized Multiple Regression Prediction Strategies with Applications

Abstract

As a classical statistical method, multiple regression is widely used for forecasting tasks in power, medicine, finance, and other fields. The rise of machine learning has led to the adoption of neural networks, particularly Long Short-Term Memory (LSTM) models, for handling complex forecasting problems, owing to their strong ability to capture temporal dependencies in sequential data. Nevertheless, the performance of LSTM models is highly sensitive to hyperparameter configuration. Traditional manual tuning methods suffer from inefficiency, excessive reliance on expert experience, and poor generalization. Aiming to address the challenges of complex hyperparameter spaces and the limitations of manual adjustment, an enhanced sparrow search algorithm (ISSA) with adaptive parameter configuration was developed for LSTM-based multivariate regression frameworks, where systematic optimization of hidden layer dimensionality, learning rate scheduling, and iterative training thresholds enhances its model generalization capability. In terms of SSA improvement, first, the population is initialized by the reverse learning strategy to increase the diversity of the population. Second, the mechanism for updating the positions of producer sparrows is improved, and different update formulas are selected based on the sizes of random numbers to avoid convergence to the origin and improve search flexibility. Then, the step factor is dynamically adjusted to improve the accuracy of the solution. To improve the algorithm’s global search capability and escape local optima, the sparrow search algorithm’s position update mechanism integrates Lévy flight for detection and early warning. Experimental evaluations using benchmark functions from the CEC2005 test set demonstrated that the ISSA outperforms PSO, the SSA, and other algorithms in optimization performance. Further validation with power load and real estate datasets revealed that the ISSA-LSTM model achieves superior prediction accuracy compared to existing approaches, achieving an RMSE of 83.102 and an $R^2$ of 0.550 during electric load forecasting and an RMSE of 18.822 and an $R^2$ of 0.522 during real estate price prediction. Future research will explore the integration of the ISSA with alternative neural architectures such as GRUs and Transformers to assess its flexibility and effectiveness across different sequence modeling paradigms.

1. Introduction

Optimization problems have long been a central research focus, and they arise in diverse practical domains, such as image classification, feature selection, autonomous driving, medical diagnosis, and energy management [1,2,3,4,5,6,7,8,9]. Solving complex optimization problems requires analyzing constraints and objectives and optimizing the objective function [10]. These problems are challenging due to multimodality, high dimensionality, and dynamic constraints. Traditional methods like gradient descent and Newton’s method rely on derivatives, are computationally expensive, and often get trapped in local optima, limiting their practical use. With the advancement of research, scholars have developed metaheuristic algorithms, which are inspired by intuitive experience or biological behavior patterns [11]. Metaheuristic optimization algorithms can be applied to optimize both symmetric and asymmetric engineering problems and are characterized by high computational efficiency, ease of implementation, flexibility, and simple frameworks. In recent years, a series of metaheuristics have been proposed, including particle swarm optimization (PSO) [12], ant colony optimization (ACO) [13], the whale optimization algorithm (WOA) [14], the grey wolf optimizer (GWO) [15], Harris hawk optimization (HHO) [16], differential evolution (DE) [17], the bat algorithm (BA) [18], ship rescue optimization (SRO) [19], the Rafflesia optimization algorithm [20], and the goose optimization algorithm (GOOSE) [21]. For example, PSO emulates bird flocking behavior, using particle velocity, position, and individual/global extrema to conduct a collaborative search in a solution space. The WOA innovatively incorporates a bubble-net hunting mechanism, where random or optimal search agents simulate humpback whale behavior to navigate solution spaces. GOOSE leverages the early-warning behavior and balance mechanisms of goose flocks, adaptively adjusting the search resolution and speed to enable precise exploration of multimodal solution spaces.

The sparrow search algorithm (SSA), proposed by Xue J and Shen B in 2020 [22], is a metaheuristic algorithm inspired by the foraging and anti-predation behaviors of sparrow populations. The SSA is characterized by a simple structure, few parameters, and high scalability. However, like other metaheuristic algorithms, it is prone to local optima and suffers from suboptimal optimization accuracy. Several improvements have been proposed to enhance the SSA’s performance. For example, Ouyang et al. [23] introduced a dynamic population partitioning method based on K-medoids clustering to balance local exploitation and global exploration. Su et al. [24] developed an adaptive hybrid SSA framework that incorporates differential evolution operators to improve population diversity and escape local optima. Yan et al. [25] proposed a multi-population collaborative search model based on K-means clustering, which avoids premature convergence while preserving global search capabilities. Huang et al. [26] suggested a chaotic optimization strategy using elite solution sets to guide population evolution. The effectiveness of these improvements has been demonstrated through benchmark testing, showing that the improved SSA algorithms outperform traditional methods in optimization tasks. Based on previous research, this paper proposes an improved sparrow search algorithm (ISSA). This algorithm generates the initial population using a reverse learning strategy to increase diversity, modifies the position update formula to prevent convergence to the origin, dynamically adjusts the step size to improve accuracy, and incorporates Lévy flight for enhanced global search capabilities. Simulations show that ISSA-LSTM outperforms other algorithms on the CEC2005 benchmark functions.

Multiple regression models analyze the relationships between multiple variables and a single response, providing better explanatory power than univariate methods by incorporating multiple factors [27,28]. This is particularly useful in time-series tasks, such as power load forecasting and real estate price prediction, where long-term dependencies and multi-factor interactions dominate the data dynamics [29,30]. However, traditional regression methods struggle with high-dimensional and non-stationary data. In contrast [31], deep learning models, such as LSTM networks, have revolutionized prediction tasks by autonomously extracting features from complex datasets. These models outperform traditional machine learning approaches, which rely on manual feature selection [32,33,34]. Recent research has combined metaheuristic optimization with LSTM networks to further enhance prediction accuracy. For instance, Sun et al. [35] proposed a PSO-LSTM hybrid to optimize hyperparameters, improving failure prediction accuracy. Lu et al. [36] demonstrated that differential evolution-optimized LSTMs outperform classical models in electricity forecasting. Pal et al. [37] showed that Bayesian-optimized shallow LSTMs achieve greater long-term prediction accuracy than deeper models. Furthermore, Cao et al. [38] integrated K-means clustering with IPSO to improve weather-resilient predictions. These advancements illustrate the potential of combining heuristic algorithms with LSTM networks to enhance predictive analytics.

This study introduces an ISSA to optimize LSTM hyperparameters for time-series prediction tasks, including hidden layer nodes, training epochs, and the learning rate, significantly improving prediction accuracy and reliability. Validation on two publicly available datasets—power load forecasting and real estate price prediction—shows that ISSA-LSTM outperforms other LSTM models optimized by alternative methods, with consistently lower error deviations. These results highlight the combined strength of the ISSA’s global search capabilities and LSTM’s sequential modeling.

The structure of this paper is as follows: Section 2 introduces the operational paradigms of the SSA and LSTM networks, providing the theoretical foundations for subsequent developments. Section 3 introduces the ISSA model, detailing its enhanced convergence mechanisms through a mathematical derivation. Section 4 validates the optimization superiority of the ISSA by benchmarking it against eight established metaheuristic algorithms using 10 test functions. In Section 5, an integrated ISSA-LSTM architecture is proposed and its performance is demonstrated through multivariate regression analysis on two publicly available datasets, outperforming eight state-of-the-art models. The conclusion summarizes the main contributions of this research and outlines three key research directions for advancing evolutionary algorithms.

2. Related Works

2.1. Sparrow Search Algorithm

The sparrow search algorithm (SSA) simulates the foraging behavior of sparrows to achieve population-based optimization through collaborative mechanisms. Its core principle involves dynamically dividing the population into discoverers and followers based on mutual observation and information exchange. Discoverers are responsible for locating high-quality solutions and guiding the swarm’s movement, while followers adjust their trajectories by observing the positions of the discoverers. Additionally, some individuals compete for better positions through a competitive mechanism, enhancing the overall allocation of resources. Compared to traditional swarm intelligence algorithms, the SSA exhibits stronger global search capabilities and faster convergence, making it well-suited for solving complex optimization problems.

The mathematical formulation of the SSA consists of solution vectors and corresponding fitness values, which are defined as follows:

$$X=\left[\begin{array}{llll}{x}_{1,1}\hfill & x_{1,2}\hfill & \dots \hfill & x_{1,d}\hfill \\ x_{2,1}\hfill & x_{2,2}\hfill & \dots \hfill & x_{2,d}\hfill \\ ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ x_{N,1}\hfill & x_{N,2}\hfill & \dots \hfill & x_{N,d}\hfill \end{array}\right]$$

(1)

$$F_x=\left[\begin{array}{c}f([x_{1,1},x_{1,2},\dots ,x_{1,d}])\\ f([x_{2,1},x_{2,2},\dots ,x_{2,d}])\\ ⋮\\ f([x_{N,1},x_{N,2},\dots ,x_{N,d}])\end{array}\right]$$

(2)

Here, $X$, $N$, and $d$ represent the individual position matrix, the number of individuals in the population, and the dimension of the population in the SSA, respectively. $x_{i,j}$ is the position of the i-th individual in the j-th dimension. $f$ is the fitness function of the SSA, and the fitness matrix ($F_x$) is an $N\times 1$ matrix.

As the population evolves, discoverer individuals with higher fitness guide the global search by exploring promising regions, while followers perform localized refinements within these areas. The SSA achieves behavioral coordination by dividing individuals into three roles, each governed by distinct position update strategies.

The discoverers have higher energy. With the progress of each iteration, the formula for updating their positions is as follows:

$$X_{i,j}^{t+1}=\left\{\begin{array}{c}{X}_{i,j}^t\cdot \exp ({\displaystyle \frac{-i}{\alpha \cdot t_{\max }}}),R_2<\mathrm{ST} \\ X_{i,j}^t+Q\cdot L,R_2\ge \mathrm{ST}\end{array}\right.$$

(3)

In this equation, $X_{i,j}^{t+1}$ represents the spatial coordinate of the $i^{th}$ sparrow in the $j^{th}$ dimension at the (t + 1)-th iteration; $\alpha \in [0,1]$ is a stochastic decay coefficient modulating the exploration rate; $t_{\max }$ denotes the termination criterion for the evolutionary iterations; $ST$ and $R_2$ denote the safety threshold and early-warning value, with their value ranges being [0.5, 1] and (0, 1], respectively. The term $Q\sim N$ represents a normally distributed random variable bounded within [−1, 1], and $L$ is a $1\times d$ matrix.

Followers observe the movements of discoverers to identify promising directions. Once a discoverer is found to occupy a region with higher fitness, followers adjust their positions accordingly and engage in competitive replacement. Successful followers replace inferior individuals; otherwise, they continue to monitor other discoverers. The follower update mechanism is mathematically described as follows:

$$X_{i,j}^{t+1}=\left\{\begin{array}{c}Q\cdot \exp ({\displaystyle \frac{X_{worst}^t-X_{i,j}^t}{i^2}}),i>{\displaystyle \frac{n}{2}}\\ X_p^{t+1}+\left|X_{i,j}^{t+1}-X_p^{t+1}\right|+L\cdot A^{+},i\le {\displaystyle \frac{n}{2}}\end{array}\right.$$

(4)

$$A^{+}=A^T{(A{A}^T)}^{-1}$$

(5)

In this formula, $X_{i,j}^{t+1}$ represents the spatial coordinate of the $i^{th}$ sparrow in the $j^{th}$ dimension at the (t + 1)-th iteration. $X_{worst}^t$ denotes the least-fit agent’s spatial configuration at iteration t; $i$ indexes the fitness hierarchy among the population members; and $X_p^{t+1}$ is the position of the individual with the best fitness at the $t+1$-th iteration. $A$ constitutes a binary coefficient matrix, and the instances of $A$ take values of either 1 or −1. The term $Q\sim N$ represents a normally distributed random variable bounded within [−1, 1], and $L$ is a $1\times d$ matrix.

The pseudoinverse $A^{+}=A^T{(A{A}^T)}^{-1}$ is used to calculate an update direction. To ensure the mathematical validity of this operation, the matrix $A\in R^{d\times d}$ is constructed as a full-rank binary matrix, with each element randomly assigned within {1, −1}. In this setting, $A{A}^T$ is symmetric and nonsingular with high probability, thus guaranteeing the existence of the inverse and the correctness of the pseudoinverse expression.

$$X_{i,j}^{t+1}=\left\{\begin{array}{c}{X}_{best}^t+\beta \left|X_{i,j}^i-X_{best}^t\right|,f_i>f_g\\ X_{i,j}^t+K\cdot \left({\displaystyle \frac{\left|X_{i,j}^{t+1}-X_{worst}^{t+1}\right|}{f_i-f_w+\epsilon }}\right), f_i=f_g \end{array}\right.$$

(6)

In this formula, $X_{i,j}^{t+1}$ represents the spatial coordinate of the $i^{th}$ sparrow in the $j^{th}$ dimension at the (t + 1)-th iteration. $X_{best}^t$ encodes the elite agent’s optimal spatial state, where $\beta$ and $K\sim u(0,1)$ are stochastic scaling factors. $\epsilon$ is negligible and has a constant value; its purpose is to prevent division by zero. $f_i$ denotes the fitness of the present individual, while $f_g$ (global best) and $f_w$ (global worst) are designations used to denote the maximum and minimum levels of fitness that can be achieved, respectively, by any given individual.

2.2. Long Short-Term Memory Neural Network

As an improved architecture of recurrent neural networks (RNNs), the LSTM model effectively mitigates the challenge of learning long-term dependencies in time-series prediction. This is achieved through a gating mechanism composed of input, forget, and output gates, which enables the selective retention and removal of historical information [39]. Compared to standard RNNs, LSTM significantly reduces the problems of gradient vanishing and explosion. The structure of the LSTM network is illustrated in Figure 1.

(1) The forget gate ($f_t$) controls which historical states in $C_{t-1}$ are discarded and transfers the remaining information to the current memory cell ($C_t$):

$$f_t=\sigma (W_f\cdot [h_{t-1},x_t]+b_f)$$

(7)

In this formula, $h_{t-1}$ denotes the hidden state at time t − 1 and $x_t$ is the input vector at t. $W_f$ (weight) and $b_f$ (bias) are critical parameters defining the forget gate. The value of $f_t\in [0,1]$ is generated by the sigmoid function. When $f_t=0$, all previous states in $C_{t-1}$ are erased; if $f_t=1$, $C_{t-1}$ is fully preserved.

(2) The input gate ($i_t$) determines how much new information from $x_t$ is stored in $C_t$:

$$i_t=\sigma (W_i\cdot [h_{t-1},x_t]+b_i)$$

(8)

Similar to the forget gate, $h_{t-1}$ represents the hidden state at the previous time step, and $x_t$ is the input vector at the current time step. $W_i$ (weight) and $b_i$ (bias) are specific to the input gate. The value calculated through the sigmoid activation function ($\sigma$) is within the interval [0, 1]. When $i_t=0$, the gate blocks all input, while when $i_t=1$ it allows complete storage.

(3) The candidate state (${\tilde{C}}_t$) is computed from $C_t$ and serves as an intermediate step to update $C_{t-1}$.

$${\tilde{C}}_t=\tanh (W_c\cdot [h_{t-1},x_t]+b_C)$$

(9)

Here, $W_c$ and $b_c$ are the weight matrix and bias parameter of the candidate state (${\tilde{C}}_t$), respectively. The value of ${\tilde{C}}_t$ calculated through the hyperbolic tangent activation function ($\tanh$) is within the interval [−1, 1].

(4) Memory Cell Update

The memory cell update rule integrates historical and novel states:

$$C_t=f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t$$

(10)

The terms $f_t$, $i_t$, and ${\tilde{C}}_t$ are derived from Equations (7)–(9).

(5) The output gate ($o_t$) regulates how much of ${\tilde{C}}_t$ is propagated to $h_t$ at the same time step, and it outputs the filtered information.

$$o_t=\sigma (W_o\cdot [h_{t-1},x_t]+b_o)$$

(11)

$$C_t=f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t$$

(12)

The gate parameters are defined by the weight matrix ($W_o$) and the bias vector ($b_o$), and $o_t\in [0,1]$ through sigmoidal activation. Equation (12) updates the memory cell by blending $C_{t-1}$ and ${\tilde{C}}_t$.

3. Improved Sparrow Search Algorithm Strategy

3.1. Oppositional Learning Strategy

Oppositional learning [40] enhances population diversity by performing mutation operations on the population after each iteration.

$$X_i^{\prime }=1\times (X_{\max }+X_{\min })-X_i$$

(13)

In this formula, $X_{\max }$ and $X_{\min }$ denote the upper and lower bounds, respectively. In the i-th iteration, $X_i$ represents the individual, whereas ${\overline{X}}_i$ signifies the generated adversarial individual. The population size is expanded to 2$M$ candidate solutions by merging the original individual ($X_i$) with the adversarial learning-generated individual (${\overline{X}}_i$). The optimal $M$ solutions are filtered based on fitness ranking to update the location, and the iterative loop continues to optimize the quality of the population.

3.2. Producer Anti-Origin Convergence Strategy

The performance of the SSA was evaluated in Reference [22] through a series of benchmark experiments. The results indicated that when the early-warning variable (R2) exceeds the safety threshold (ST), discoverers update their positions toward the current global optimum using random perturbations following a normal distribution. Conversely, if R2 < ST, the position update mechanism drives individuals to converge toward the origin, which limits the algorithm’s ability to handle optimization tasks with distant optima. As a result, while the SSA performs well on problems with solutions near the origin, it often exhibits insufficient accuracy when addressing complex landscapes where the global optimum is far away. To address this limitation, this study introduces an improved position update rule for discoverers, as shown in Equation (14). The proposed strategy mitigates premature convergence to the origin and enhances the algorithm’s global search capability across diverse optimization scenarios:

$$X_{i,j}^{t+1}=\left\{\begin{array}{c}{X}_{i,j}^t\cdot (1+Q),R_2<\mathrm{ST} \\ X_{i,j}^t+Q\cdot L,R_2\ge \mathrm{ST}\end{array}\right.$$

(14)

In this equation, $X_{i,j}^{t+1}$ represents the spatial coordinate of the $i^{th}$ sparrow in the $j^{th}$ dimension at the (t + 1)-th iteration; the term $Q\sim N$ represents a normally distributed random variable bounded within [−1, 1]; and $L$ is a $1\times d$ matrix.

3.3. Lévy Flight Strategy

Lévy flight [41], a non-Gaussian stochastic process with heavy-tailed step-size distributions, exhibits the dual characteristics of frequent small steps for local exploration and occasional large jumps to search globally [42]. The standard SSA tends to fall into local optima, limiting its effectiveness in global optimization tasks. To address this issue, Lévy flight is integrated into the position update mechanism (Equation (6)), enabling a balance between short-range exploitation and long-range exploration. As the algorithm nears the global optimum, Lévy steps enhance fine-grained search capabilities while preventing premature convergence. By utilizing the displacement between the current and best-known solutions, the improved SSA maintains robust local search performance while expanding its global exploration potential. This enhanced update rule is formalized in Equation (15):

$$X_{i,j}^{t+1}=\left\{\begin{array}{c}Levy(d)·X_{best}^t+\beta \left|X_{i,j}^i-Levy(d)\cdot X_{best}^t\right|,f_i>f_g\\ X_{i,j}^t+K\cdot \left({\displaystyle \frac{\left|X_{i,j}^{t+1}-X_{worst}^{t+1}\right|}{f_i-f_w+\epsilon }}\right), f_i=f_g \end{array}\right.$$

(15)

In this formula, d represents the spatial dimension. The Lévy calculation formulas are shown in Equations (16) and (17):

$$Levy(d)=0.01\cdot {\displaystyle \frac{r_1\cdot \sigma }{{\left|r_2\right|}^{\frac{1}{\beta }}}}$$

(16)

$$\sigma ={\left\{{\displaystyle \frac{\Gamma (1+\theta )\cdot \sin (\pi \cdot \theta /2)}{\Gamma [(1+\lambda )/2]\cdot \theta \cdot 2^{(1-\theta )/2}}}\right\}}^{1/\theta }$$

(17)

In these formulas, $\mathrm{\Gamma }$ represents the gamma function, $r_1$ and $r_2$ are random numbers within the interval [0, 1], and $\theta$ is a constant.

3.4. Step-Size Factor Adjustment Strategy

In the standard SSA, the step-length control parameters (β and K in Equation (6)) play a crucial role in balancing global exploration and local exploitation. However, due to their inherently random nature, the search process often suffers from poor utilization of the solution space and a tendency to fall into local optima. To overcome this issue, this study proposes an improved strategy where β and K are adaptively adjusted: larger values promote a broader global search, while smaller values support precise local refinement [43]. The modified formulations for β and K are detailed in Equations (18) and (19), respectively:

$$\beta =fitnes{s}_{best}-(fitnes{s}_{best}-fitnes{s}_{worst})·{({\displaystyle \frac{T-t}{T}})}^{1.5}$$

(18)

$$K=(fitnes{s}_{best}-fitnes{s}_{worst})·\exp (-20·\tan {({\displaystyle \frac{t}{T}})}^2)·(2·rand-1)$$

(19)

In these formulas, $fitnes{s}_{best}$ denotes the best fitness value and $fitnes{s}_{worst}$ is used to denote the worst fitness value, while T is used to denote the maximum number of iterations. Equation (18) shows that the improved step-length control parameters exhibit a nonlinear increasing trend. In early SSA iterations, high population diversity implies strong global exploration but weak local exploitation. Therefore, we set the control parameters to smaller values to enhance local exploitation. In later iterations, as sparrows converge to the current global optimum, the risk of premature convergence increases as the search space narrows. Larger control parameter values help avoid sparrows becoming trapped in local optima and facilitate further exploration. Equation (19) reveals biphasic modulation of the step-length factor (K): an ascending phase in the initial iterations promotes exhaustive search space exploration, while subsequent rapid attenuation expedites convergence. This dynamic adjustment of the step-length parameters balances the SSA’s global exploration and local exploitation, improving optimization accuracy while avoiding local optima.

3.5. The Pseudo-Code of the ISSA

To illustrate the improved sparrow search algorithm (ISSA), the pseudo-code is presented in Algorithm 1.

Algorithm 1. The pseudo-code of the ISSA.

Input: The population size (n), the number of discoverers (PD), the number of followers (SD), the early-warning value (R2), the safety threshold (ST), and the maximum number of iterations (T). Output: The location of a sparrow and its fitness value. 1: Initialize the population using the oppositional learning strategy. 2: Calculate the fitness value of each sparrow and determine the optimal and worst individuals in the current population. 3: While(t < T) 4:    for i = 1:PD 5:          Update the position of the i-th sparrow according to Equation (14) (update the positions of the discoverers) 6:    end for 7:    for i = PD + 1:n 8:          Update the position of the i-th sparrow according to Equation (4) (update the positions of the followers) 9:    end for 10:    for l = 1:SD 11:          Update the position of the l-th sparrow according to Equations (15)–(18) (update the positions of the scouts);

12:    end for 13:    Obtain the updated positions of all sparrows; 14:    If the fitness value corresponding to the new updated position is higher than that of the current position, update the current position; 15:    t = t + 1; 16: end while 17: Return the objective function value.

4. Case Study

4.1. Parameter Settings

Computational experiments were executed on a workstation with a 2.40 GHz Intel i5-1135G7 processor and 16 GB of memory running 64-bit Windows 10 Pro. The ISSA framework was developed using MATLAB R2022a.

To evaluate the effectiveness of the ISSA in solving complex optimization problems, ten benchmark functions from the CEC 2005 test suite were selected for simulation experiments. As detailed in

Table 1. Benchmark test functions.

Function	Function Name	Type	Range	Optimal Value
F1	Sphere	Unimodal	[−100, 100]	0
F2	Schwefel 1.2	Unimodal	[−100, 100]	0
F3	Schwefel 2.21	Unimodal	[−100, 100]	0
F4	Generalized Rosenbrock	Multimodal	[−30, 30]	0
F5	Quartic with Noise	Multimodal	[−1.28, 1.28]	0
F6	Schwefel 2.26	Multimodal	[−500, 500]	−12,569.5
F7	Shekel’s Foxholes	Multimodal	[−65.536, 65.536]	1
F8	Shekel’s Family(A)	Composite	[0, 10]	−10
F9	Shekel’s Family(B)	Composite	[0, 10]	−10
F10	Shekel’s Family(C)	Composite	[0, 10]	−10

, these functions are 30-dimensional and fall into three categories: unimodal (F1–F3), multimodal (F4–F7), and composite (F8–F10). Unimodal functions are used to assess the convergence rates of algorithms, multimodal functions test their robustness in navigating local optima, and composite functions examine the balance between exploration and exploitation in intricate problem landscapes.

4.2. Comparative Algorithm Analysis

The ISSA’s performance, evaluated through 30 independent trials per benchmark function, was benchmarked against the SSA, SSA1 (the SSA optimized with Lévy flight), PSO, the GWO, DE, a GA, and GOOSE. All algorithms were initialized with identical configurations: T = 500 (maximum of 500 iterations), N = 30 (30 population members), and dimensionality (d = 30). Three evaluation metrics—the optimal value, mean value, and standard deviation—were derived from the 10 benchmark functions. Given the shared population-based iterative framework, the theoretical time complexity of the ISSA, the SSA, and PSO can be uniformly expressed as O(N × D × T), where N = 30, d = 30, and T = 500. Although the ISSA incorporates enhancements such as reverse learning and Lévy flight, these operations only incur constant-time overhead and do not alter the asymptotic complexity. The simulation results are shown in

Table 2. Simulation results of algorithm testing (d = 30).

Function	Algorithm	Best	Mean	Std
F1	GWO	2.14 × 10−29	2.35 × 10−27	5.84 × 10−27
	PSO	−4.79 × 10−0	−3.02 × 10−1	2.67 × 100
	DE	3.98 × 10−4	1.54 × 101	5.72 × 101
	GOOSE	5.22 × 10−3	1.83 × 101	5.37 × 101
	GA	1.85 × 104	2.61 × 104	3.32 × 103
	SSA	0	1.54 × 10−67	8.38 × 10−67
	SSA1	0	1.42 × 10−70	7.79 × 10−70
	ISSA	0	0	0
F2	GWO	2.14 × 10−29	2.35 × 10−27	5.84 × 10−27
	PSO	−4.79 × 100	−3.02 × 10−1	2.67 × 100
	DE	3.98 × 10−4	1.54 × 101	5.72 × 101
	GOOSE	5.22 × 10−3	1.83 × 101	5.37 × 101
	GA	1.85 × 104	2.61 × 104	3.32 × 103
	SSA	−1.05 × 101	−9.45 × 100	2.20 × 100
	SSA1	0	1.42 × 10−70	7.79 × 10−70
	ISSA	0	0	0
F3	GWO	8.94 × 10−8	7.95 × 10−7	1.02 × 10−6
	PSO	−2.04 × 101	1.55 × 101	1.29 × 101
	DE	1.40 × 101	2.49 × 101	5.89 × 100
	GOOSE	1.18 × 10−1	2.28 × 101	2.16 × 101
	GA	6.22 × 101	7.04 × 101	3.91 × 100
	SSA	0	4.85 × 10−33	2.66 × 10−32
	SSA1	0	5.93 × 10−34	3.24 × 10−33
	ISSA	0	7.65 × 10−244	0
F4	GWO	2.58 × 101	2.70 × 101	6.61 × 10-1
	PSO	−1.47 × 101	1.47 × 100	9.04 × 100
	DE	1.13 × 102	2.77 × 103	7.00 × 103
	GOOSE	2.55 × 101	1.90 × 102	4.24 × 102
	GA	8.89 × 106	2.13 × 107	5.85 × 106
	SSA	4.84 × 10−9	4.88 × 10−5	8.02 × 10−5
	SSA1	1.05 × 10−8	2.20 × 10−4	4.02 × 10−4
	ISSA	5.92 × 10−14	7.16 × 10−6	3.35 × 10−5
F5	GWO	3.31 × 10−4	1.93 × 10−3	9.97 × 10−4
	PSO	−1.48 × 10−1	2.90 × 10−2	1.13 × 10−1
	DE	3.33 × 10−2	9.47 × 10−2	4.41 × 10−2
	GOOSE	4.90 × 10−2	1.24 × 10−1	5.70 × 10−2
	GA	3.39 × 100	1.05 × 101	3.95 × 100
	SSA	3.50 × 10−5	4.62 × 10−4	4.17 × 10−4
	SSA1	2.24 × 10−5	3.91 × 10−4	2.74 × 10−4
	ISSA	2.21 × 10−5	2.45 × 10−4	2.09 × 10−4
F6	GWO	−7.45 × 103	−5.96 × 103	9.88 × 102
	PSO	−5.00 × 102	2.04 × 101	2.56 × 102
	DE	−9.76 × 103	−7.36 × 103	1.29 × 103
	GOOSE	−8.18 × 103	−6.88 × 103	5.73 × 102
	GA	−1.18 × 104	−1.14 × 104	2.07 × 102
	SSA	−9.15 × 103	−7.90E × 103	7.88 × 102
	SSA1	−1.00 × 104	−8.59 × 103	6.82 × 102
	ISSA	−1.26 × 104	−1.22 × 104	6.68 × 102
F7	GWO	9.98 × 10−1	4.23 × 100	4.08 × 100
	PSO	−3.20 × 101	−3.20 × 101	6.88 × 100
	DE	9.98 × 10−1	9.98 × 10−1	0
	GOOSE	1.99 × 100	1.31 × 101	7.30 × 100
	GA	9.98 × 10−1	1.00 × 100	5.95 × 10−3
	SSA	9.98 × 10−1	8.20 × 100	5.60 × 100
	SSA1	9.98 × 10−1	9.82 × 100	4.87 × 100
	ISSA	9.98 × 10−1	7.50 × 100	5.43 × 100
F8	GWO	−1.02 × 101	−9.06 × 100	2.26 × 100
	PSO	4.00 × 100	5.73 × 100	1.39 × 100
	DE	−1.02 × 101	−8.65 × 100	2.83 × 100
	GOOSE	−1.02 × 101	−5.47 × 100	3.27 × 100
	GA	−4.18 × 100	−2.08 × 100	8.50 × 10−1
	SSA	−1.02 × 101	−8.28 × 100	2.50 × 100
	SSA1	−1.02 × 101	−9.12 × 100	2.07 × 100
	ISSA	−1.02 × 101	−1.00 × 101	4.29 × 10−1
F9	GWO	−1.04 × 101	−1.02 × 101	9.70 × 10−1
	PSO	4.00 × 100	4.77 × 100	1.25 × 100
	DE	−1.04 × 101	−1.01 × 101	1.39 × 100
	GOOSE	−1.04 × 101	−5.68 × 100	3.26 × 100
	GA	−3.94 × 100	−1.93 × 100	6.00 × 10−1
	SSA	−1.04 × 101	−9.52 × 100	2.01 × 100
	SSA1	−1.04 × 101	−9.68 × 100	1.83 × 100
	ISSA	−1.04 × 101	−1.02 × 101	8.80 × 10−1
F10	GWO	−1.05 × 101	−1.00 × 101	2.06 × 100
	PSO	2.01 × 100	4.57 × 100	1.58 × 100
	DE	−1.05 × 101	−1.04 × 101	9.79 × 10−1
	GOOSE	−1.05 × 101	−4.61 × 100	3.38 × 100
	GA	−3.79 × 100	−2.04 × 100	6.19 × 10−1
	SSA	−1.05 × 101	−9.45 × 100	2.20 × 100
	SSA1	−1.05 × 101	−9.61 × 100	2.04 × 100
	ISSA	−1.05 × 101	−1.05 × 101	4.49 × 10−2

.

Table 2 demonstrates that the ISSA, SSA1, and the SSA yielded the best results in solving the unimodal functions, F1, F2, and F3, outperforming traditional algorithms such as PSO, the GWO, DE, the GA, and GOOSE in terms of optimization accuracy. Notably, the ISSA achieved a mean and a standard deviation of zero, reflecting its exceptional consistency and robustness compared to the other methods.

In the case of the multimodal functions, the ISSA surpassed the SSA and SSA1 in optimization accuracy, achieving the best results for F4, F6, and F7. For F5, the ISSA yielded the solution closest to the theoretical optimum (0) among all tested algorithms. In terms of average performance, the ISSA demonstrated the highest accuracy, while the SSA ranked second. Regarding the standard deviation, the ISSA achieved superior consistency across most functions, except for F6 and F7.

Regarding the composite functions, F8–F10, all algorithms—except PSO, the GWO, DE, the GA, and GOOSE—approximated the optimal values to varying degrees. Among them, the ISSA achieved the lowest mean and standard deviation, signifying superior global search performance, robustness, and stability. These results confirm the ISSA’s competitive advantage across all three categories: unimodal, multimodal, and composite functions.

The convergence trajectories of the ISSA and the baseline algorithms for different benchmark functions are visualized in Figure 2. The x-axis represents the iteration number, while the y-axis denotes the best fitness value obtained. The figure clearly illustrates the ISSA’s faster convergence toward the global optimum.

Figure 2 illustrates the convergence trajectories for three function types: unimodal (F1–F3), multimodal (F4–F7), and composite (F8–F10).

Unimodal functions (F1–F3): The ISSA, SSA1, and the SSA showed superior global exploration performance compared to PSO, the GWO, DE, the GA, and GOOSE. Notably, the ISSA achieved rapid early-stage convergence, reaching near-optimal solutions faster, while PSO and the GWO converged more slowly in the initial iterations.

Multimodal functions (F4–F7): The ISSA excelled at avoiding local optima, benefiting from a robust global search mechanism. Its ability to balance exploration and exploitation effectively prevented early stagnation while maintaining solution accuracy.

Composite functions (F8–F10): The ISSA gradually converged toward superior solutions. Specifically, for F10, although PSO performed well initially, the ISSA demonstrated consistent convergence across the iterations and reliably approached the global optimum.

In summary, the ISSA achieves rapid convergence in unimodal tasks through efficient search dynamics. In multimodal problems, it reliably escapes local optima and locates the global optimum with high accuracy. In composite scenarios, the ISSA ensures stable convergence across iterations, showcasing strong adaptability and robustness in complex optimization settings. These strengths make the ISSA a valuable tool for solving high-dimensional predictive and optimization problems.

5. Multiple Regression Prediction Based on ISSA-LSTM

Multiple regression prediction plays a vital role across various domains by integrating multiple influencing factors to forecast outcomes. Optimization of LSTM neural networks via ISSA enhances prediction accuracy and reliability. However, as the number of influencing factors increases, discrepancies between predicted and actual results tend to grow. Therefore, improving the precision of multiple regression prediction has become a significant research focus in recent years.

In this section, the predictive performance of the ISSA-LSTM framework is comparatively evaluated against benchmark models. Empirical results demonstrate its superior accuracy in multivariate regression tasks.

5.1. Dataset Information

This study employed two diverse datasets from Kaggle, representing typical application domains for multivariate regression: an electricity load dataset and a real estate housing price dataset. The electricity load data comprised 625 hourly consumption records collected in June 2020 across three Indian cities (Mumbai, Delhi, and Chennai), featuring 12 variables, including temperature, humidity, and wind speed and direction, as well as temporal indicators like the date and time. Temporal features such as the time of day, the day of the week, and lagged load values were also extracted. Missing values were addressed using linear interpolation, and Min–Max normalization was applied.

The real estate dataset contained 1460 housing transaction records, each characterized by 12 attributes such as the number of bedrooms, the number of bathrooms, the living area, the year the house was built, and location-based parameters. House prices were normalized into units of thousands to facilitate interpretation.

The selection of these datasets ensured a comprehensive evaluation of the ISSA-LSTM model’s adaptability, robustness, and generalization in scenarios with both structured and sequential data.

5.2. ISSA-LSTM Load Forecasting Model

The initial hyperparameter settings of LSTM neural networks often result in suboptimal prediction performance [44]. To overcome this, the ISSA was introduced to dynamically optimize three critical hyperparameters: the number of hidden layer nodes, the number of training epochs, and the learning rate. Specifically, the learning rate controls convergence behavior—a rate that is too low slows training, while a rate that is too high risks entrapment in local optima. Meanwhile, the complexity of the hidden layer and the training duration directly impact predictive accuracy [45]. The ISSA-LSTM model was implemented as follows:

Step 1: Preprocess the data to define the inputs and outputs for the ISSA-LSTM regression model. Feature parameters are extracted from the housing price and electric load datasets as inputs, with corresponding target values as outputs. Due to differences in dimensionality and scale, normalization is applied before splitting the dataset into training and testing subsets at a 4:1 ratio.

Step 2: Configure ISSA parameters including the dimensionality, population size, maximum iterations, and early-warning thresholds. Initialize the sparrow population and establish the LSTM regression model along with the hyperparameter search ranges.

Step 3: Employ the ISSA to iteratively optimize the three hyperparameters: the learning rate, the number of hidden nodes, and the number of epochs. During optimization, experimental data train the model, and the test set RMSE is used as the objective metric. After T iterations, the algorithm converges to an optimal hyperparameter set that minimizes prediction error while balancing the computational cost.

Step 4: Deploy the optimized ISSA-LSTM model for multivariate regression prediction. Validate its performance by comparing the predicted outputs with the ground-truth values of the test set and quantify its accuracy through statistical error metrics to demonstrate improved predictive power and robustness.

5.3. Model Parameter Settings and Evaluation Indicators

This study employed a single-layer LSTM network architecture and selected “Adam” as the optimizer. The gradient clipping threshold was set to 1.0 to avert gradient explosion. All remaining network parameters adopted default configurations, with only the target hyperparameters undergoing optimization.

The basic parameter configurations for each algorithm in the LSTM neural network optimization are shown in

Table 3. Basic parameter settings of optimization algorithm.

Parameter Name	Parameter Value
Pop_size	20
Dim	3
Max_iter	200
HiddenUnits	[10, 200]
MaxEpochs	[30, 100]
InitialLearnRate	[0.001, 0.05]

. To ensure experimental fairness, key parameters were kept consistent across the algorithms. Based on the proposed ISSA-LSTM model and the comparative models, multiple regression prediction experiments were conducted on the two datasets described earlier.

The performance assessment metrics employed in this study were as follows:

RMSE quantifies the quadratic mean deviation between model predictions and ground-truth observations:

$$RMSE=\sqrt{{\displaystyle \sum _{i=’}^n{({\widehat{y}}_i-y_i)}^2}}$$

(20)

The Mean Absolute Error (MAE) is mathematically defined as the average of absolute differences between predicted values and corresponding actual measurements:

$$MAE={\displaystyle \sum _{i=1}^n\frac{({\widehat{y}}_i-y_i)}{n}}$$

(21)

For the Coefficient of Determination ($R^2$), the closer its value is to one, the more accurate the results:

$$R^2=1-{\displaystyle \frac{{{\displaystyle {\sum }_{i=1}^n(y_i-{\widehat{y}}_i)}}^2}{{{\displaystyle {\sum }_{i=1}^n(y_i-\overline{y})}}^2}}$$

(22)

5.4. Experimental Results and Analysis of Multiple Regression Prediction

This study employed the ISSA alongside other optimization algorithms to identify the optimal hyperparameter configurations for LSTM prediction models. Experimental validation was performed on models optimized by each algorithm using two datasets.

Figure 3 and Figure 4 illustrate the prediction curves of LSTM models optimized by different algorithms for the electric load and real estate price datasets, respectively. The ISSA-LSTM model demonstrated the closest fit to the actual data, exhibiting minimal deviation in both cases.

Table 4. Prediction errors and hyperparameters of different models (electric load dataset).

Models	MAE	Rmse	R2	HiddenUnits	LearningRate	MaxEpochs
LSTM	83.062	106.308	0.263	100	0.001	80
PSO-LSTM	73.457	101.468	0.329	76	0.00395	40
GWO-LSTM	81.440	101.973	0.322	16	0.00460	67
GOOSE-LSTM	73.234	100.448	0.342	81	0.001	72
DE-LSTM	71.079	96.059	0.399	41	0.0287	76
GA-LSTM	72.111	100.146	0.346	63	0.0468	51
SSA-LSTM	80.474	98.523	0.367	41	0.0495	87
SSA1-LSTM	67.818	90.563	0.465	145	0.0225	60
ISSA-LSTM	67.037	83.102	0.550	124	0.0385	90

and

Table 5. Prediction errors and hyperparameters of different models (real estate housing price dataset).

Models	MAE	Rmse	R2	HiddenUnits	LearningRate	MaxEpochs
LSTM	18.566	24.805	0.169	100	0.001	100
PSO-LSTM	16.748	22.631	0.41	85	0.0025	50
GWO-LSTM	14.294	20.801	0.416	100	0.00221	70
GOOSE-LSTM	16.035	24.245	0.206	47	0.05	84
DE-LSTM	15.394	23.129	0.278	99	0.0272	59
GA-LSTM	15.117	21.395	0.382	16	0.0430	76
SSA-LSTM	15.016	21.088	0.400	12	0.0334	67
SSA1-LSTM	14.911	20.765	0.418	149	0.0361	73
ISSA-LSTM	12.886	18.822	0.522	168	0.042	97

further confirm that ISSA-LSTM achieved the lowest prediction errors across all evaluated metrics.

Building on the critical role of hyperparameter optimization, comparative experiments revealed the superior fitting performance of the ISSA-LSTM model. Table 5 highlights that ISSA-LSTM attained the smallest average prediction errors, with significantly better MAE and RMSE values compared to SSA-LSTM, GWO-LSTM, and other models. Its forecasting deviation was also notably lower than that of the PGOA-LSTM framework [46], underscoring its enhanced prediction accuracy.

Table 5 reinforces these findings, showing that ISSA-LSTM consistently achieved the highest accuracy and the lowest MAE/RMSE values, confirming its robustness and effectiveness.

In summary, the ISSA-LSTM model exhibits outstanding prediction accuracy and a strong generalization ability in electric load and real estate price forecasting, offering a reliable solution for time-series prediction in these domains.

6. Conclusions

This study presented an ISSA framework and assessed its optimization performance against several metaheuristic algorithms through extensive benchmarking on 10 standard test functions. The results show that the ISSA achieved faster convergence on unimodal functions by swiftly approaching the optimal solution. For multimodal functions, it effectively balanced global exploration and local exploitation, improving the search for optimal solutions and reducing the risk of local optima. In composite function optimization, the ISSA demonstrated high stability and efficiency, consistently converging to optimal solutions across multiple trials. By integrating the ISSA with the LSTM neural network, we developed the ISSA-LSTM model for multivariate regression prediction, where the ISSA adaptively optimizes three critical hyperparameters: the number of training epochs, the hidden layer size, and the learning rate. Experimental validations on two datasets revealed that ISSA-LSTM significantly outperformed competing models in prediction accuracy, confirming its effectiveness and superiority in multivariate regression tasks. The model’s enhanced predictive performance arises from the synergy between the ISSA’s strong global optimization capabilities and LSTM’s proficiency in modeling sequential data. The ISSA improves hyperparameter search through reverse learning-based population initialization, adaptive step-size control, and Lévy flight, enabling broader exploration of the parameter space and mitigating premature convergence. This leads to parameter configurations better aligned with data characteristics, allowing the LSTM to capture nonlinear relationships and long-term dependencies more effectively, thereby boosting generalization and prediction stability.

Despite promising results, there remains potential for improvement. This work focused on optimizing three hyperparameters: the number of hidden layer nodes, the number of training epochs, and the learning rate. Nonetheless, other aspects such as the number of layers, the connectivity patterns, and the forget gate thresholds also influence accuracy. Future research will employ the ISSA to further optimize these elements and comprehensively enhance the LSTM model.