An Enhanced ABC Algorithm with Hybrid Initialization and Stagnation-Guided Search for Parameter-Efficient Text Summarization

Abstract

The digital transformation of oil and gas pipeline networks has generated substantial volumes of unstructured maintenance documentation from communication systems, creating an urgent need for automated summarization to improve operational efficiency. However, domain-specific text summarization for pipeline communication maintenance remains challenging due to scarce labeled data and the high computational cost of fine-tuning large pretrained models. Parameter-efficient fine-tuning alleviates this issue, but its effectiveness strongly depends on appropriate hyperparameter selection. This paper proposes a unified framework that combines weight-decomposed low-rank adaptation with an enhanced Artificial Bee Colony algorithm for automated hyperparameter optimization. The enhanced algorithm addresses two specific limitations of the standard Artificial Bee Colony algorithm: uninformed random initialization that ignores promising regions, and premature abandonment of stagnated solutions that discards partially useful search directions. These two components represent principled design choices, each targeting a distinct bottleneck in applying swarm intelligence search to high-dimensional mixed-type hyperparameter spaces. The method introduces a hybrid initialization strategy to exploit prior knowledge and a stagnation-guided local search mechanism to refine stagnated solutions instead of discarding them, achieving a better balance between exploration and exploitation. Experimental results on a public Chinese summarization benchmark and an industrial oil and gas pipeline communication maintenance corpus show that the proposed approach consistently outperforms full fine-tuning, manually tuned parameter-efficient methods, and several evolutionary optimization baselines in terms of ROUGE metrics. The automated search introduces modest additional computational overhead compared to manual tuning while eliminating expert-dependent hyperparameter configuration and achieving consistent performance gains across both datasets. Overall, the proposed framework provides an efficient and robust solution for adapting large language models to specialized summarization tasks in the context of pipeline communication system maintenance.

1. Introduction

The ongoing digital transformation of the oil and gas pipeline industry has created an urgent demand for automated document processing technologies. The National Pipeline Network Group manages large-scale pipeline infrastructure spanning the country, with extensive communication network systems supporting operational monitoring and control. Modern pipeline communication system maintenance activities continuously generate large volumes of unstructured textual data, including operation logs, alarm records, inspection reports, and technical maintenance narratives. These documents typically contain detailed procedural descriptions, network topology information, equipment parameters, and domain-specific terminology, resulting in high information density but excessive length. As a consequence, manually converting such documents into standardized work orders or maintenance summaries for communication operation and maintenance management systems is not only time-consuming but also highly prone to human errors. Therefore, automated document summarization has become a critical enabling technology for improving the accuracy, efficiency, and reliability of pipeline communication system maintenance workflows, supporting the strategic transformation from manual to digital operation and maintenance, and offering substantial practical value for national integrated pipeline network management.

Recent progress in abstractive summarization has been driven by Transformer-based models such as BART [1], which combine bidirectional encoders with autoregressive decoders to simultaneously capture contextual semantics and generate coherent summaries. BART has demonstrated robust performance across a range of generation tasks, including summarization, by learning to reconstruct corrupted input text using the Transformer architecture. Subsequent work has advanced the quality of abstractive summaries through improved content planning and semantic coherence. For example, Narayan et al. [2] introduce learned entity prompts to ground the generation process, showing that intermediate planning mechanisms can improve entity specificity and reduce hallucination in generated summaries. In parallel, recent approaches [3] explore hierarchical attention and graph representations to better model local and global discourse structures in long documents, yielding enhanced coverage of salient information for scientific summarization. Although these studies achieve notable improvements in generation quality, they predominantly rely on full fine-tuning and extensive manual hyperparameter search, resulting in high computational cost and inefficient tuning processes that limit their practical deployment in industrial settings. This limitation has motivated the development of parameter-efficient fine-tuning methods.

Parameter-efficient fine-tuning methods effectively address the computational overhead of full model adaptation by substantially reducing the number of trainable parameters. Among these techniques, Low-Rank Adaptation (LoRA) [4] has emerged as a widely adopted approach for efficiently fine-tuning pretrained Transformer models by injecting trainable low-rank matrices into attention layers while freezing the original pretrained weights. Building on this foundation, Weight-Decomposed Low-Rank Adaptation (DoRA) [5] further enhances parameter-efficient fine-tuning by decomposing pretrained weights into magnitude and directional components, and applying LoRA updates to direction while directly optimizing magnitude, thereby more closely emulating the learning dynamics of full fine-tuning. These methods dramatically reduce the number of trainable parameters compared to full fine-tuning, significantly lowering memory and computation requirements. However, the performance of these methods remains highly sensitive to hyperparameter configurations, such as low-rank dimension and scaling factors, which can substantially affect adaptation quality. In practice, hyperparameter selection still largely depends on manual tuning or grid search, leading to inefficient exploration and difficulties in identifying optimal configurations in high-dimensional search spaces. This limitation underscores the need for automated and efficient hyperparameter optimization strategies in parameter-efficient fine-tuning frameworks.

Evolutionary and swarm intelligence algorithms have been widely applied to hyperparameter optimization in machine learning due to their ability to efficiently explore large search spaces without requiring gradient information. Representative methods include Genetic Algorithm (GA) [6], Particle Swarm Optimization (PSO) [7], Differential Evolution (DE) [8], and several nature-inspired algorithms such as Firefly Algorithm (FA) [9], Bat Algorithm (BA) [10], and Grey Wolf Optimizer (GWO) [11]. Among these meta-heuristics, the Artificial Bee Colony (ABC) algorithm [12], inspired by the foraging behavior of honey bee swarms, has attracted significant research interest for both continuous and discrete optimization problems due to its simple structure and competitive performance. However, the standard ABC algorithm has two key limitations. First, purely random initialization of the population ignores prior knowledge, which may result in suboptimal solutions and inefficient exploration of the search space. Second, when a candidate solution fails to improve beyond a predefined limit, it is directly abandoned and replaced by a random solution, which may discard potentially valuable search directions and hinder optimization performance. In the PEFT hyperparameter optimization setting, where each evaluation is costly, these two weaknesses are particularly harmful: random initialization wastes early evaluations on poor configurations, and premature abandonment of stagnated solutions discards partially useful search information before it can be refined. Recent advances in automated search problems have demonstrated the effectiveness of incorporating learned guidance mechanisms to navigate high-dimensional search spaces. For instance, attention-based mechanisms have been successfully applied to neural architecture search [13,14,15], while learned importance guidance has enhanced population-based feature selection [16,17]. These limitations of the standard ABC algorithm and the demonstrated effectiveness of knowledge-guided strategies motivate the enhancements to population initialization and stagnation handling proposed in this work.

To address the limitations discussed above, this paper proposes a parameter-efficient abstractive summarization framework that integrates DoRA-based fine-tuning with an enhanced Artificial Bee Colony (ABC) optimization strategy. The proposed framework employs DoRA to achieve efficient domain adaptation for text summarization while using an improved ABC algorithm to automate hyperparameter search. To overcome the two inherent limitations of the standard ABC algorithm, we introduce two core improvement strategies: hybrid population initialization and stagnation-guided local search. The core insight is that domain knowledge about effective hyperparameter regions, which is available from prior fine-tuning experience, can substantially reduce wasted evaluations during early search, and that stagnated solutions in PEFT hyperparameter search often retain partial structural information that is worth refining rather than discarding. These two mechanisms together improve search quality without increasing the evaluation budget, which is critical when each evaluation requires a full model training run. The main contributions of this work are summarized as follows:

• We propose a hybrid framework that integrates DoRA parameter-efficient fine-tuning with an enhanced ABC algorithm for automated hyperparameter optimization in text summarization.
• We introduce a hybrid initialization strategy that combines empirical solutions from domain knowledge. This strategy improves solution quality by exploiting prior knowledge while maintaining global exploration capability.
• We develop a stagnation-guided local search mechanism that performs guided perturbation around the global best solution when a candidate exceeds the stagnation threshold but remains below the abandonment limit. This mechanism preserves valuable search information and enhances the algorithm’s ability to escape local optima.

While this work focuses on pipeline communication maintenance summarization, the proposed framework is general in design and can be applied to other domain-specific NLP tasks.

2. Related Work

2.1. Abstractive Text Summarization

Abstractive text summarization aims to generate concise summaries that paraphrase and reorganize salient content rather than directly extracting sentences from the source text. Early neural approaches modeled summarization as a sequence-to-sequence generation task using recurrent encoder–decoder architectures with attention mechanisms [18,19]. The emergence of Transformer-based architectures further advanced the field by enabling efficient global dependency modeling, substantially improving fluency and semantic coverage in generated summaries [20].

Subsequent research has focused on enhancing generation quality through more sophisticated modeling strategies. Reinforcement learning-based methods directly optimize evaluation metrics such as ROUGE to improve informativeness and reduce exposure bias [21]. Structure-aware approaches incorporate discourse signals or graph representations to capture document-level relationships beyond linear token sequences [22]. Planning-based and content-grounded models introduce intermediate representations or entity-aware prompts to improve factual consistency and reduce hallucination [2]. While these methods have achieved notable performance gains, they predominantly assume full fine-tuning of large pretrained models and rely on manually selected hyperparameters. As a result, their computational cost and tuning complexity limit applicability in resource-constrained or industrial environments. This limitation motivates research on more efficient adaptation and automated optimization strategies.

2.2. Parameter-Efficient Fine-Tuning

Parameter-efficient fine-tuning methods have been developed to reduce the computational and memory requirements of adapting large pretrained language models to downstream tasks. Adapter-based approaches insert lightweight task-specific modules into Transformer layers while keeping the original parameters frozen [23]. Prompt-tuning and prefix-tuning methods optimize a small set of continuous vectors that condition model behavior without modifying the backbone weights [24]. Among these techniques, Low-Rank Adaptation (LoRA) has gained widespread adoption by introducing trainable low-rank matrices into attention projections, achieving competitive performance with minimal additional parameters [4].

Building upon LoRA, Weight-Decomposed Low-Rank Adaptation (DoRA) further decomposes pretrained weights into magnitude and directional components [5]. DoRA applies low-rank updates to the directional component while directly optimizing the magnitude vector. This decomposition better approximates the learning dynamics of full fine-tuning while maintaining parameter efficiency. Despite their effectiveness, these parameter-efficient methods remain highly sensitive to hyperparameter configurations, including the low-rank dimension, scaling factors, learning rates, and dropout rates. Existing approaches typically rely on manual tuning or grid search strategies, which are inefficient and often fail to identify optimal configurations in high-dimensional hyperparameter spaces. This limitation motivates the need for automated hyperparameter optimization methods specifically designed for parameter-efficient fine-tuning frameworks.

2.3. Swarm Intelligence for Hyperparameter Optimization

Swarm intelligence and evolutionary algorithms have been widely employed for hyperparameter optimization due to their gradient-free nature and strong global search capability. Classical strategies such as grid search and random search [25] are straightforward but scale poorly to high-dimensional and mixed-type search spaces. Bayesian optimization methods such as Tree-structured Parzen Estimator (TPE) [26] and BOHB [27] build probabilistic surrogate models over the search history and achieve strong performance in continuous spaces. However, they require additional modeling assumptions and are less straightforward to apply when the search space contains both discrete and continuous variables. In PEFT, the low-rank dimension and scaling factor take values from a finite ordered set and interact with continuous variables such as learning rate and dropout rate. Population-based swarm intelligence methods handle such mixed spaces natively through their perturbation mechanisms and do not require a surrogate model, making them more directly applicable to this setting. Representative approaches include Genetic Algorithm [6], Particle Swarm Optimization [7], and Differential Evolution [8]. In addition, nature-inspired meta-heuristics such as Firefly Algorithm [9], Bat Algorithm [10], and Grey Wolf Optimizer [11] have demonstrated effectiveness in navigating complex, multimodal search spaces.

Among these methods, the Artificial Bee Colony (ABC) algorithm has attracted sustained attention due to its simple structure, limited control parameters, and competitive performance in mixed continuous–discrete optimization problems [12]. Several ABC variants have been proposed to enhance exploration or convergence speed through modified neighborhood search rules or diversity-enhancing mechanisms [28]. Among recent advances, the Adaptive Exploration ABC (AEABC) [29] introduces distance-based neighbor selection in the employed-bee phase and combines fitness with proximity to the global best for onlooker-bee selection, improving the exploration–exploitation balance. The Evolutionary ABC (EABC) [30] incorporates a DE-inspired adaptive evolutionary scaling factor that decays over iterations to shift the search from exploration to exploitation, and replaces the standard random scout reset with Gaussian perturbation centred on the global best solution to preserve structural information during reinitialization. However, two limitations remain insufficiently addressed in existing studies. First, standard ABC employs purely random population initialization, disregarding domain knowledge or empirical priors that could significantly improve early-stage search efficiency. Second, stagnated solutions are typically abandoned and replaced by random candidates, which may prematurely discard partially informative search directions. These limitations motivate the improved initialization and stagnation-handling strategies proposed in this work.

3. Problem Formulation

This work addresses hyperparameter optimization for parameter-efficient text summarization. We formulate text summarization as a supervised learning problem where a model learns to generate concise summaries from input documents.

Let $\mathcal{X}=\{x_1,x_2,\dots ,x_n\}$ denote an input document and $\mathcal{Y}=\{y_1,y_2,\dots ,y_m\}$ denote the target summary, where $m\ll n$. The model parameters $\mathit{\theta }$ are trained by minimizing the cross-entropy loss:

$${\mathcal{L}}_{\mathrm{CE}}\left(\mathit{\theta }\right)=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{t=1}^{m^{\left(i\right)}}logp\left(y_t^{\left(i\right)}\mid y_{1:t-1}^{\left(i\right)},{\mathcal{X}}^{\left(i\right)};\mathit{\theta }\right),$$

(1)

where N is the number of training samples and $m^{\left(i\right)}$ is the length of the i-th summary.

The hyperparameter optimization problem seeks the optimal configuration ${\mathit{\lambda }}^{\ast }$ that minimizes the validation performance:

$${\mathit{\lambda }}^{\ast }=arg\underset{\lambda \in \Lambda }{min}\mathcal{J}\left({\mathit{\theta }}^{\ast }\left(\mathit{\lambda }\right)\right),$$

(2)

where ${\mathit{\theta }}^{\ast }\left(\mathit{\lambda }\right)$ denotes the trained model parameters under hyperparameter configuration $\mathit{\lambda }$, $\mathcal{J}$ is the validation metric, and $\Lambda$ is the hyperparameter space. In our work, $\mathcal{J}$ is the validation loss or negative ROUGE-1 score, and the hyperparameters $\lambda$ include continuous variables (i.e., learning rate $\eta$, weight decay ${\lambda }_w$, dropout rate d) and discrete variables (i.e., low-rank dimension r, scaling factor $\alpha$). Specifically, the continuous hyperparameters are bounded as $\eta \in [8\times {10}^{-5},5\times {10}^{-4}]$, ${\lambda }_w\in [0.01,0.05]$, and $d\in [0.0,0.2]$. The discrete hyperparameters are constrained to $r\in \{8,16,32,64\}$ and $\alpha \in \{4,8,16,32,64,128\}$. Three additional hyperparameters, namely batch size, gradient accumulation steps, and warmup ratio, are fixed at 16, 2, and 0.01, respectively.

4. Method

The overall architecture of the proposed framework is illustrated in Figure 1. The framework consists of two main components: (1) DoRA-based parameter-efficient fine-tuning for domain adaptation, and (2) an enhanced ABC algorithm for automated hyperparameter optimization with hybrid initialization and stagnation-guided local search strategies.

4.1. Parameter-Efficient Fine-Tuning

4.1.1. Baseline Model

We adopt BART [1] as the baseline architecture for text summarization. BART is a sequence-to-sequence Transformer model consisting of a bidirectional encoder and an autoregressive decoder. The encoder uses multi-head self-attention to process the input document and capture contextual representations, while the decoder generates the summary token by token using both masked self-attention and cross-attention to the encoder outputs. This architecture is particularly effective for abstractive summarization as it combines bidirectional understanding of the source text with fluent autoregressive generation. We apply parameter-efficient fine-tuning methods to adapt the pretrained BART model to our domain-specific maintenance report summarization task.

4.1.2. Weight-Decomposed Low-Rank Adaptation

Standard parameter-efficient fine-tuning methods such as LoRA [4] reduce the number of trainable parameters by injecting low-rank matrices into the pretrained network and freezing the original weights during downstream adaptation. Specifically, given a pretrained weight matrix $W_0\in {\mathbb{R}}^{d\times k}$, LoRA introduces two low-rank matrices $A\in {\mathbb{R}}^{r\times k}$ and $B\in {\mathbb{R}}^{d\times r}$ with $r\ll min(d,k)$ to approximate the weight update $\Delta W$:

$$\Delta W\approx BA,$$

(3)

and the adapted weight becomes

$$W^{\prime }=W_0+\Delta W=W_0+BA.$$

(4)

This formulation significantly reduces the number of trainable parameters from $d\times k$ (full fine-tuning) to $r(d+k)$, thereby lowering computational cost.

Although LoRA effectively reduces trainable parameters, it updates magnitude and directional changes proportionally within the weight matrix, which can limit its learning flexibility relative to full fine-tuning. To address this limitation, we adopt Weight-Decomposed Low-Rank Adaptation (DoRA) as a more expressive parameter-efficient fine-tuning method [5]. The core idea of DoRA is to explicitly separate the pretrained weight matrix into independent magnitude and directional components before applying low-rank adaptation. Formally, let the pretrained weight be decomposed as

$$W_0=m·{\displaystyle \frac{V}{{\parallel V\parallel }_c}},$$

(5)

where $V\in {\mathbb{R}}^{d\times k}$ is the directional component and ${\parallel V\parallel }_c$ denotes the column-wise norm of V. The magnitude vector $m\in {\mathbb{R}}^{1\times k}$ captures per-column scaling factors initialized from $W_0$ and is treated as trainable. Under this decomposition, V encodes the unit directional structure of $W_0$ while m scales each directional vector. For directional adaptation, DoRA introduces low-rank updates $\Delta V$ via $A\in {\mathbb{R}}^{r\times k}$ and $B\in {\mathbb{R}}^{d\times r}$ with

$$\Delta V=BA,$$

(6)

and the adapted weight becomes

$$W^{\prime }=m·{\displaystyle \frac{V+\Delta V}{{\parallel V+\Delta V\parallel }_c}}.$$

(7)

This decomposition enables independent optimization of magnitude and direction, thereby enhancing learning flexibility and more closely emulating the learning dynamics of full fine-tuning.

4.2. Enhanced ABC Algorithm

As illustrated in Figure 1, the Enhanced ABC algorithm serves as the hyperparameter optimizer in the proposed framework. Each candidate solution in the ABC population represents a hyperparameter configuration $\lambda =\{r,\alpha ,\eta ,{\lambda }_w,d\}$, consisting of the low-rank dimension r, scaling factor $\alpha$, learning rate $\eta$, weight decay ${\lambda }_w$, and dropout rate d. For each candidate, BART is fine-tuned with DoRA under the corresponding configuration, and the validation ROUGE-1 score is returned to ABC as the fitness value to drive population updates. This loop repeats until the stopping criterion is met, after which the best configuration is used for final training. The complete procedure is summarized in Algorithm 1.

Algorithm 1 Enhanced ABC Algorithm for Hyperparameter Optimization

Require: Colony size N, stagnation threshold ratio ${\rho }_{\tau }$, abandonment limit L, maximum evaluations T, search bounds $[l,u]$ Ensure: Best hyperparameter configuration $X_{\mathrm{gbest}}$   1: Generate initial population via hybrid strategy (Equations (11)–(13))   2: Evaluate fitness $f\left(X_i\right)$ for all $i=1,\dots ,N$   3: Set ${\mathrm{trial}}_i\leftarrow 0$ for all i   4: Compute stagnation threshold $\tau \leftarrow \lfloor {\rho }_{\tau }·L\rfloor$   5: while number of evaluations $<T$ do   6:    Employed Bee Phase:   7:    for each solution $X_i$ do   8:      Generate candidate $V_i$ via Equation (14)   9:      if $f\left(V_i\right)\ge f\left(X_i\right)$ then 10:         $X_i\leftarrow V_i$, ${\mathrm{trial}}_i\leftarrow 0$ 11:      else 12:         ${\mathrm{trial}}_i\leftarrow {\mathrm{trial}}_i+1$ 13:      end if 14:    end for 15:    Onlooker Bee Phase: 16:    Select solutions with probability $p_i=f\left(X_i\right)/{\sum }_{j}f\left(X_j\right)$ (Equation (9)) 17:    Apply same neighborhood search and greedy update as employed bee phase 18:    Scout Bee Phase with Stagnation-Guided Local Search: 19:    for each solution $X_i$ do 20:      if $\tau \le {\mathrm{trial}}_i<L$ then 21:         Generate $V_i=X_{\mathrm{gbest}}+\eta ·(X_i-X_{\mathrm{gbest}})+\varphi ·(u-l)$      (Equation (15)) 22:         where $\eta \sim \mathcal{U}(0,1)$ is a random scaling factor, $\varphi ·(u-l)$ denotes perturbation per Equations (17)–(22) 23:         if $f\left(V_i\right)\ge f\left(X_i\right)$ then 24:           $X_i\leftarrow V_i$, ${\mathrm{trial}}_i\leftarrow 0$ 25:         end if 26:      else if ${\mathrm{trial}}_i\ge L$ then 27:         Reset $X_i\sim \mathcal{U}(l,u)$ (Equation (16)), ${\mathrm{trial}}_i\leftarrow 0$ 28:      end if 29:    end for 30:    Update $X_{\mathrm{gbest}}\leftarrow arg{max}_{X\in \mathcal{X}}f\left(X\right)$ 31: end while 32: return  $X_{\mathrm{gbest}}$

4.2.1. Standard ABC

The ABC algorithm [31] is a swarm intelligence optimization technique inspired by the foraging behavior of honey bee colonies. In ABC, each candidate solution is represented as a food source. Then, the population of $SN$ solutions $\mathbf{X}=\{X_1,X_2,\dots ,X_{SN}\}$ is initialized within the search space ${\mathbb{R}}^D$, where D denotes the problem dimensionality. The algorithm operates through three cooperative phases that simulate distinct foraging behaviors.

Employed Bee Phase: Each employed bee i explores the neighborhood of its current solution $X_i$ to generate a candidate $V_i$. For a randomly selected dimension $k\in \{1,\dots ,D\}$ and a neighbor index $j\ne i$, the neighborhood search is performed as

$$v_{i,k}=x_{i,k}+{\varphi }_{i,k}\times (x_{i,k}-x_{j,k}),$$

(8)

where ${\varphi }_{i,k}\sim \mathcal{U}(-1,1)$ controls the perturbation magnitude. A greedy selection rule is then applied to retain the better solution between $V_i$ and $X_i$.

Onlooker Bee Phase: After employed bees share their results, onlooker bees probabilistically select food sources based on their fitness values. The selection probability for solution $X_i$ is computed as

$$p_i={\displaystyle \frac{f\left(X_i\right)}{{\sum }_{j=1}^{SN}f\left(X_j\right)}},$$

(9)

which biases the search toward high-quality regions. Onlooker bees apply the same neighborhood search mechanism as employed bees and perform greedy selection.

Scout Bee Phase: Solutions that fail to improve after a predefined number of consecutive trials (the limit parameter) are considered stagnated and are reinitialized randomly:

$$X_i\leftarrow X_{min}+\xi ·(X_{max}-X_{min}),$$

(10)

where $\xi \sim \mathcal{U}(0,1)$, and $X_{min}$ and $X_{max}$ define the search space boundaries.

The standard ABC algorithm has demonstrated effectiveness across various optimization problems due to its simple structure and balanced exploration–exploitation mechanism. However, two aspects limit its performance in hyperparameter optimization tasks. First, random initialization ignores prior knowledge of promising regions. Second, the direct abandonment of stagnated solutions in the scout phase may discard partially useful search directions before sufficient local refinement. These observations motivate the enhancements proposed in the following sections.

4.2.2. Hybrid Initialization Strategy

In the standard ABC algorithm, the initial population of candidate solutions is generated purely at random within the hyperparameter search space. While this ensures broad exploration, it ignores prior knowledge of promising regions and may result in less efficient search trajectories. To address this limitation, we propose a hybrid initialization strategy that partitions the population into three components: empirical solutions, perturbation-based local samples, and random exploratory samples.

Let N denote the total colony size. We partition the population into three groups according to

$$N=N_e+N_p+N_r,$$

(11)

where $N_e$ is the number of empirical solutions directly injected based on domain knowledge, $N_p$ is the number of perturbation-based samples generated around empirical solutions, and $N_r$ is the number of purely random solutions to maintain diversity.

To control the balance between local exploitation of expert knowledge and population diversity, we introduce a perturbation ratio $\rho \in (0,1)$ that determines the proportion of perturbation-based samples. Given $N_e$ expert solutions, the number of perturbation samples is set as $\lceil \rho ·N\rceil$, with the remaining $N-N_e-\lceil \rho ·N\rceil$ solutions initialized randomly. This ratio is treated as a tunable parameter and its sensitivity is analyzed in Section 5.3.4.

Empirical Solutions: Empirical solutions are collected from prior experience or heuristics reflecting high-quality hyperparameter configurations. These configurations provide strong initial reference points in the hyperparameter search space $\Lambda$. We directly include these $N_e$ solutions in the initial ABC population to serve as high-quality seeds that guide subsequent exploration.

Perturbation-Based Local Samples: For each empirical solution $E_i$, we generate perturbation-based samples by adding small random deviations to exploit the neighborhood of promising configurations:

$$P_{i,j}=E_i+{\delta }_{i,j},{\delta }_{i,j}\sim \mathcal{U}(-ϵ,ϵ),$$

(12)

where $ϵ$ is the perturbation strength controlling the deviation magnitude. For discrete hyperparameters such as low-rank dimensions, probability-based selection mechanisms are employed instead of direct perturbation (see Section 4.2.4 for details). This local sampling strategy refines the search around empirical solutions while introducing controlled variability.

Random Exploratory Samples: To preserve population diversity, the remaining $N_r$ solutions are uniformly sampled across the search space $\Lambda \subseteq {\mathbb{R}}^D$ within the bounds $[L,U]$:

$$R_k\sim \mathcal{U}(L,U).$$

(13)

The initial ABC population is formed by combining all three components, providing a balance between high-quality guidance from expert knowledge, local refinement through perturbation, and exploratory diversity via random sampling. This hybrid initialization strategy effectively embeds prior information into the search while retaining ABC’s innate ability to explore the hyperparameter space broadly, leading to better optimization performance compared to completely random initialization.

4.2.3. Stagnation-Guided Local Search

In the standard ABC algorithm, when a candidate solution $X_i$ has not improved for a predefined number of consecutive trials L, it is directly abandoned and replaced by a random solution. Although this promotes exploration, such direct random reset may discard partially valuable search information, thereby hindering optimization performance in high-dimensional hyperparameter spaces.

To mitigate the premature loss of informative search directions, we propose a stagnation-guided local search mechanism that enhances exploitation around promising regions while retaining the algorithm’s exploration capability. Instead of immediately abandoning stagnated solutions, the mechanism performs guided search around the current global best solution $X_{\mathrm{gbest}}$, allowing the algorithm to leverage accumulated global information before resorting to random re-initialization.

The proposed mechanism operates in three stages based on the stagnation level of each candidate solution. Let ${\mathrm{trial}}_i$ denote the number of consecutive iterations during which solution $X_i$ fails to improve. The mechanism introduces a stagnation threshold $\tau$ that triggers guided local search, while retaining the conventional abandonment limit L from standard ABC. To ensure that $\tau$ scales consistently with the abandonment budget, it is defined as $\tau =\lfloor {\rho }_{\tau }·L\rfloor$, where ${\rho }_{\tau }\in (0,1)$ is the stagnation threshold ratio. The sensitivity of ${\rho }_{\tau }$ is empirically analyzed in Section 5.3.4.

Normal Search (${\mathrm{trial}}_i<\tau$). When ${\mathrm{trial}}_i<\tau$, the solution is in a normal search state and is updated using the standard employed-bee and onlooker-bee operators of ABC. For the employed-bee phase, a new candidate $V_i$ is generated by

$$V_{i,k}=X_{i,k}+{\phi }_{i,k}·(X_{i,k}-X_{j,k}),$$

(14)

where $j\in \{1,2,\dots ,N\}\setminus \left\{i\right\}$ is a randomly selected solution different from i, k is a randomly chosen dimension, and ${\phi }_{i,k}\sim \mathcal{U}(-1,1)$ controls the perturbation magnitude.

Stagnation-Guided Search ($\tau \le {\mathrm{trial}}_i<L$). When the stagnation counter satisfies $\tau \le {\mathrm{trial}}_i<L$, the search switches to a guided local exploration phase. Instead of discarding the solution, $X_i$ is perturbed around $X_{\mathrm{gbest}}$ to exploit promising regions while preserving directional information.

Specifically, for a stagnating solution $X_i$, we generate a local neighbor $V_i$ defined by

$$V_i=X_{\mathrm{gbest}}+\eta ·(X_i-X_{\mathrm{gbest}})+\varphi ·(\mathbf{u}-\mathbf{l}),$$

(15)

where $X_{\mathrm{gbest}}=arg{max}_{X\in \mathbf{X}}f\left(X\right)$ denotes the global best solution in the current population, $\eta \sim \mathcal{U}(0,1)$ is a scaling factor controlling the contribution from $X_i$, $\varphi ·(\mathbf{u}-\mathbf{l})$ provides small random perturbations for local exploration which is sampled according to the selected distribution type as defined in Section 4.2.4, and $\mathbf{l},\mathbf{u}\in {\mathbb{R}}^D$ are the lower and upper bounds of the search space. For discrete hyperparameters, probability-based selection mechanisms are employed analogously to Section 4.2.4.

Random Reset (${\mathrm{trial}}_i\ge L$). Only when ${\mathrm{trial}}_i\ge L$ does the algorithm follow the original ABC abandonment rule, where $X_i$ is reinitialized uniformly at random across the search space:

$$X_i\sim \mathcal{U}(\mathbf{l},\mathbf{u}).$$

(16)

To illustrate the behavior of the proposed stagnation-guided local search mechanism, Figure 2 shows the population distribution of the Enhanced ABC at iteration 100 on the two-dimensional Rastrigin function, a standard multimodal benchmark with a known global optimum at the origin. The majority of bees are concentrated near the global optimum, reflecting the exploitation effect of guided perturbation in Equation (15). At the same time, a small number of bees remain distributed in distant regions due to the random reset in Equation (16), preserving global exploration capability. This distribution confirms that the proposed stagnation-guided mechanism effectively balances local exploitation and global exploration.

By introducing this intermediate stagnation-aware search stage, the proposed mechanism prevents premature random resets and enhances local refinement near the global best solution. This guided exploitation of accumulated search information accelerates convergence toward high-quality hyperparameter configurations while maintaining the algorithm’s ability to escape suboptimal regions through eventual random re-initialization when necessary.

4.2.4. Perturbation Generation

The hybrid initialization strategy and the stagnation-guided local search rely on perturbation mechanisms to explore the hyperparameter space. Since hyperparameters include both continuous and discrete types, we design appropriate perturbation schemes for each parameter type and employ three perturbation distributions to accommodate different search characteristics.

Perturbation Distributions: Three perturbation distributions are employed, each offering distinct exploration properties.

Gaussian distribution facilitates smooth local exploration, with perturbations concentrated around zero, making it suitable for fine-grained local search in promising regions. Its probability density function is given by

$$f_{\mathrm{G}}(x;\sigma )={\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}exp\left(-{\displaystyle \frac{x^2}{2{\sigma }^2}}\right).$$

(17)

The Cauchy distribution, characterized by heavy tails, is more likely to generate large perturbations compared to the Gaussian distribution, thereby helping to escape local optima. Its probability density function is

$$f_{\mathrm{C}}(x;\gamma )={\displaystyle \frac{1}{\pi \gamma }}·{\displaystyle \frac{1}{1+{(x/\gamma )}^2}}.$$

(18)

The Logistic map generates a deterministic yet unpredictable sequence based on the chaotic system

$$z_{n+1}=4z_n(1-z_n),z_0\in (0,1),$$

(19)

which balances exploration randomness with reproducibility.

Continuous Hyperparameters: For continuous hyperparameters, perturbations are sampled from the selected distribution and applied additively. Let $l_b$ and $u_b$ denote the lower and upper bounds of a parameter, and let $\alpha \in (0,1)$ denote the perturbation strength. The perturbed value is computed as

$$x^{\prime }=x+\alpha ·(u_b-l_b)·\xi ,$$

(20)

where $\xi$ is sampled according to the chosen distribution type: $\xi \sim \mathcal{N}(0,1)$ for Gaussian perturbation, $\xi \sim \mathrm{Cauchy}(0,1)$ for Cauchy perturbation, and $\xi =2z_n-1$ for chaotic perturbation. Here, $z_n\in (0,1)$ is the Logistic map output at iteration n, and the linear transformation $2z_n-1$ maps the chaotic sequence to the interval $(-1,1)$ to ensure zero-centered perturbations. The perturbed value is constrained to $[l_b,u_b]$.

Discrete Hyperparameters: For discrete hyperparameters taking values from a finite ordered set $C=\{c_1,c_2,\dots ,c_n\}$, direct additive perturbation is not applicable. Instead, we adopt a probability-based neighbor selection mechanism that assigns selection probabilities centered on the current value, with higher probabilities for nearby values. Let $\mathrm{idx}$ denote the index of the current value, and let $d_j=|j-\mathrm{idx}|$ be the distance from the current index to candidate $c_j$. The selection probability is defined as

$$P\left(c_j\right)={\displaystyle \frac{w\left(d_j\right)}{{\sum }_{k=1}^{n}w\left(d_k\right)}},$$

(21)

where the weight function $w\left(d\right)$ depends on the perturbation type. For Gaussian and Cauchy perturbations, the weight functions are derived from their corresponding probability density shapes:

$$w_{\mathrm{G}}(d;\sigma )=exp\left(-{\displaystyle \frac{d^2}{2{\sigma }^2}}\right),w_{\mathrm{C}}(d;\gamma )={\displaystyle \frac{1}{1+{(d/\gamma )}^2}},$$

(22)

where $\sigma >0$ and $\gamma >0$ control the concentration of probability around the current index. Since the Logistic map generates a single deterministic value at each iteration rather than a probability distribution over discrete candidates, the chaos-based perturbation employs a different strategy: the chaotic value is used to compute a shifted center index, around which a Gaussian-weighted probability distribution is then applied for selection.

This perturbation framework enables adaptive hyperparameter space exploration by providing three complementary distribution types. Gaussian perturbations support efficient local refinement in smooth search landscapes, Cauchy perturbations facilitate escape from local optima in multimodal problems through heavy-tailed sampling, and chaos-based perturbations offer deterministic yet diverse exploration. The framework allows for flexible selection of perturbation types based on problem characteristics, enhancing the algorithm’s applicability across diverse hyperparameter optimization scenarios.

5. Experiments

5.1. Dataset

Experiments are conducted on the NLPCC 2017 dataset [32], a widely used benchmark for Chinese abstractive summarization that contains 50,000 training pairs and 2000 test samples of news articles with human-written summaries averaging about 45 Chinese characters. The dataset covers diverse topics, including technology, finance, sports, and social news, and is suitable for evaluating long-form, real-world summarization performance. To assess domain adaptability, the proposed framework is further evaluated on a proprietary oil and gas pipeline communication corpus consisting of communication network maintenance reports. Compared with general news text, this corpus exhibits stronger domain specificity, providing a realistic setting to examine the effectiveness of parameter-efficient adaptation and automated hyperparameter optimization in industrial applications.

The industrial corpus was preprocessed through a standard pipeline before evaluation. Raw maintenance reports were exported from the communication operation and maintenance management system and converted to plain text. Format normalization was applied to remove encoding errors, special symbols, and redundant whitespace, followed by length-based filtering and near-duplicate removal. To comply with the confidentiality requirements of the pipeline network operator, sensitive information including personnel identifiers, equipment serial numbers, and geographic location references was de-identified prior to evaluation.

5.2. Evaluation Metrics

To comprehensively evaluate the quality of generated summaries, we adopt three standard automatic evaluation metrics widely used in text summarization research, namely ROUGE-1, ROUGE-2, and ROUGE-L [33], along with BERTScore-F1 [34] as a complementary semantic similarity metric. These metrics measure the overlap or semantic similarity between system-generated summaries and human-written reference summaries from different perspectives, reflecting content coverage, structural consistency, and semantic quality.

ROUGE-1 measures the overlap of unigrams between the generated summary and the reference summary. It mainly reflects the adequacy of key information coverage and is commonly used to evaluate whether important content units are preserved.

ROUGE-2 computes the overlap of bigrams, capturing local word-order information and phrase-level consistency. Compared with ROUGE-1, it provides a stricter assessment of fluency and coherence.

ROUGE-L is based on the longest common subsequence between the generated and reference summaries. It evaluates sentence-level structure similarity while allowing flexible word gaps, making it suitable for assessing overall summary organization.

BERTScore-F1 evaluates summary quality using contextual token embeddings from a pretrained language model, capturing semantic equivalence that surface-level overlap metrics may not fully reflect.

5.3. Results and Discussion

5.3.1. Overall Performance Comparison

To validate the effectiveness of the proposed method, we compare it with four categories of baseline approaches on the NLPCC-2017 dataset. The first category includes traditional methods, namely the pretrained BART-Base without fine-tuning and BART with full fine-tuning. The second category consists of parameter-efficient fine-tuning methods with manually tuned hyperparameters, including LoRA [4] and DoRA [5]. The third category includes classical hyperparameter search methods, namely Grid Search and Random Search [25], which serve as widely adopted non-evolutionary baselines for hyperparameter optimization. The fourth category comprises metaheuristic hyperparameter optimization methods combined with DoRA, including Particle Swarm Optimization (PSO) [7], Differential Evolution (DE) [8], Genetic Algorithm (GA) [35], Firefly Algorithm (FA) [9], Bat Algorithm (BA) [10], Grey Wolf Optimizer (GWO) [11], standard ABC [12], Evolutionary ABC (EABC) [30], a recent variant that incorporates a DE-inspired adaptive evolutionary scaling factor and Gaussian-guided scout reinitialization, and Adaptive Exploration ABC (AEABC) [29], a recently proposed ABC variant with distance-based adaptive exploration mechanisms.

Table 1. Overall performance comparison on NLPCC-2017 dataset.

Method	ROUGE-1	ROUGE-2	ROUGE-L	BERTScore-F1
Traditional Methods
BART-Base	0.0913	0.0146	0.0890	0.7007
BART + Full Fine-Tuning	0.5559	0.3756	0.4894	0.7933
Parameter-Efficient Methods
BART + LoRA (Manual)	0.5458	0.3705	0.4833	0.7712
BART + DoRA (Manual)	0.5526	0.3788	0.4860	0.7884
Classical Search Methods
BART + DoRA + Grid Search	0.5533	0.3755	0.4937	0.7902
BART + DoRA + Random Search	0.5561	0.3761	0.4893	0.7891
Metaheuristic Optimization Methods
BART + DoRA + PSO	0.5578	0.3782	0.4928	0.7955
BART + DoRA + FA	0.5557	0.3747	0.4888	0.7918
BART + DoRA + BA	0.5551	0.3731	0.4871	0.7906
BART + DoRA + GWO	0.5576	0.3803	0.4910	0.7964
BART + DoRA + DE	0.5556	0.3762	0.4901	0.7923
BART + DoRA + GA	0.5560	0.3773	0.4893	0.7927
BART + DoRA + ABC (Standard)	0.5576	0.3792	0.4950	0.8015
BART + DoRA + EABC	0.5579	0.3819	0.4991	0.8142
BART + DoRA + AEABC	0.5591	0.3805	0.4953	0.8106
BART + DoRA + Enhanced ABC (Ours)	0.5645	0.3868	0.5003	0.8211

PSO: Particle Swarm Optimization; FA: Firefly Algorithm; BA: Bat Algorithm; GWO: Grey Wolf Optimizer; DE: Differential Evolution; GA: Genetic Algorithm; ABC: Artificial Bee Colony; AEABC: Adaptive Exploration ABC [29]; EABC: Evolutionary ABC [30].

presents the performance comparison of all methods.

Table 1 demonstrates that the proposed Enhanced ABC achieves the best performance across all evaluation metrics. The pretrained BART-Base without fine-tuning yields extremely low ROUGE scores. This is because the model is pre-trained with a denoising objective rather than for text summarization, and tends to copy input passages rather than generating concise summaries. Its relatively higher BERTScore reflects the semantic relevance retained in the copied content. Full fine-tuning substantially improves performance to 0.5559 in ROUGE-1, confirming the necessity of domain adaptation. For parameter-efficient methods, LoRA [4] with manual hyperparameter tuning achieves competitive results but remains below full fine-tuning. DoRA [5] further outperforms LoRA on all metrics, validating the effectiveness of weight decomposition in low-rank adaptation. However, both LoRA and DoRA with manual tuning still underperform full fine-tuning, indicating that suboptimal hyperparameter configurations limit their adaptation quality. For classical search methods, Grid Search achieves a ROUGE-L of 0.4937 but obtains lower ROUGE-1 and ROUGE-2 scores compared to most metaheuristic optimizers, indicating that its exhaustive yet coarse-grained search over a predefined grid struggles to identify fine-grained optimal configurations. Random Search yields a ROUGE-1 of 0.5561, which is comparable to several metaheuristic methods but still lower than the best-performing optimizers, confirming that purely stochastic sampling without guided search mechanisms has limited effectiveness in this hyperparameter space. All swarm intelligence and evolutionary optimization methods surpass manual tuning, demonstrating the value of efficient hyperparameter search in high-dimensional spaces. Among these optimizers, standard ABC achieves competitive results with the highest ROUGE-L of 0.4950 among baselines. EABC [30], which enhances the scout phase with guided reinitialization and an adaptive evolutionary scaling factor, achieves competitive results but still underperforms our proposed method across all metrics. AEABC [29], a recent ABC variant incorporating adaptive exploration strategies, achieves slightly higher ROUGE-1 (0.5591) and ROUGE-2 (0.3805) than standard ABC, demonstrating that improved exploration mechanisms can benefit hyperparameter search. However, the improvement is modest compared to our proposed method. Our proposed Enhanced ABC further improves upon standard ABC, achieving ROUGE-1 of 0.5645, ROUGE-2 of 0.3868, and ROUGE-L of 0.5003. This improvement validates that the hybrid initialization strategy provides effective starting points by leveraging prior knowledge, and the stagnation-guided local search successfully preserves valuable search directions to enhance solution quality. The consistent advantage of Enhanced ABC in BERTScore-F1 (0.8211) further confirms that the improvements extend beyond n-gram overlap to deeper semantic alignment.

5.3.2. Ablation Study

To verify the contribution of each proposed component, we conduct ablation experiments analyzing the cumulative effects of incrementally adding hybrid initialization and stagnation-guided local search to standard ABC.

Table 2. Ablation study on search strategies.

Configuration	ROUGE-1	ROUGE-2	ROUGE-L
ABC (Standard)	0.5576	0.3792	0.4950
+ Hybrid Initialization	0.5608	0.3825	0.4963
+ Stagnation-Guided Local Search	0.5645	0.3868	0.5003

presents the results of incrementally adding the two proposed enhancements to standard ABC. Each component contributes positively to the overall performance. Adding hybrid initialization to standard ABC improves ROUGE-1 from 0.5576 to 0.5608, demonstrating that incorporating expert solutions and perturbation-based sampling provides a better starting point for the optimization process. Further adding stagnation-guided local search yields additional improvement, achieving ROUGE-1 of 0.5645, ROUGE-2 of 0.3868, and ROUGE-L of 0.5003. This enhancement validates that guiding stagnated solutions toward the global best effectively preserves valuable search information and improves final solution quality. The cumulative improvement from standard ABC to the full configuration is 1.24% in ROUGE-1, confirming that both components contribute complementarily to the enhanced performance.

Figure 3 shows the best-so-far ROUGE-1 score across function evaluations for the three ablation configurations, providing empirical convergence analysis. To reduce computational cost, the convergence experiment is conducted on a randomly sampled subset of the training data, and the final performance ranking across configurations is consistent with the full-data results in Table 2. ABC with hybrid initialization reaches a score of 0.374 within the first 10 evaluations, compared to 0.362 for Standard ABC, confirming that expert-seeded initialization provides better starting solutions and accelerates early-stage convergence. The full Enhanced ABC achieves further improvement around evaluation 100, reaching a final score of 0.397, while the other two configurations plateau at 0.383. This late-stage gain demonstrates that the stagnation-guided local search successfully escapes local optima that hybrid initialization alone cannot overcome. All three curves increase monotonically and stabilize well within the evaluation budget, confirming convergence stability consistent with Standard ABC.

5.3.3. Perturbation Type Analysis

To investigate how different perturbation distributions affect optimization performance in our method, this section first visualizes the sampling characteristics of three perturbation types (i.e., Gaussian, Cauchy, and Chaos) and then evaluates all the combinations of initialization and search perturbations.

Distribution Characteristics Visualization. To illustrate the sampling behavior of each perturbation type, we conduct a visualization experiment on both continuous and discrete parameters. For continuous parameters, values are normalized to the range [0, 1] with an assumed optimal value of 0.5. For discrete parameters, a candidate set {4, 8, 16, 32, 48, 64} is used with 32 as the optimal value. Each perturbation type generates 1000 samples with a fixed scale parameter of 0.1. Figure 4 presents the distribution characteristics of the three perturbation types. Gaussian perturbation generates samples highly concentrated around the optimal value for both continuous and discrete parameters, supporting fine-grained exploitation in promising regions. Cauchy perturbation maintains similar concentration but exhibits heavier tails, preserving non-negligible probability at distant candidates and enabling occasional large jumps to escape local optima. Chaos perturbation, based on the Logistic map, produces a multi-modal distribution for continuous parameters and a more uniform selection probability across discrete candidates, providing broader coverage when the search space structure is unknown.

Performance Comparison of Perturbation Combinations. Based on the above distribution analysis, we evaluate all the combinations of initialization and search perturbation types.

Table 3. Performance comparison of different perturbation type combinations.

Init Type	Search Type	ROUGE-1	ROUGE-2	ROUGE-L
Gaussian	Gaussian	0.5645	0.3868	0.5003
Gaussian	Cauchy	0.5590	0.3779	0.4947
Gaussian	Chaos	0.5577	0.3766	0.4913
Cauchy	Gaussian	0.5554	0.3742	0.4906
Cauchy	Cauchy	0.5547	0.3722	0.4983
Cauchy	Chaos	0.5575	0.3779	0.4948
Chaos	Gaussian	0.5588	0.3786	0.4946
Chaos	Cauchy	0.5595	0.3784	0.4949
Chaos	Chaos	0.5544	0.3742	0.4890

presents the results on the NLPCC-2017 dataset. The Gaussian–Gaussian combination achieves the best performance with ROUGE-1 of 0.5645, ROUGE-2 of 0.3868, and ROUGE-L of 0.5003. This result can be attributed to the quality of expert solutions used in our experiments. When high-quality empirical configurations are available, the concentrated sampling of Gaussian perturbation effectively exploits these promising regions rather than dispersing search effort through wide exploration. Combinations with Gaussian initialization consistently achieve competitive results regardless of the search perturbation type, with ROUGE-1 scores ranging from 0.5577 to 0.5645. In contrast, when Cauchy or Chaos is used for initialization, the heavier tails or uniform coverage may introduce solutions far from expert configurations, reducing the benefit of prior knowledge. It should be noted that these alternative perturbation types may prove more effective in scenarios where expert solutions are unavailable or of lower quality, as their broader exploration could help discover promising regions independently. Based on these findings, we adopt the Gaussian–Gaussian configuration as the default setting for the experiments in this paper.

5.3.4. Parameter Sensitivity Analysis

This section analyzes the sensitivity of four key parameters: perturbation scale for initialization and local search phases, stagnation threshold ratio that controls when to trigger local search, and expert sampling ratio that determines the proportion of perturbation-based samples in the initial population.

Table 4. Parameter sensitivity analysis demonstrating the impact of key algorithmic configurations on ROUGE scores.

Parameter	Value	ROUGE-1	ROUGE-2	ROUGE-L
Local Search Perturbation Scale	0.05	0.5613	0.3812	0.4977
	0.1	0.5645	0.3868	0.5003
	0.2	0.5600	0.3799	0.4961
Initialization Perturbation Scale	0.05	0.5571	0.3778	0.4930
	0.1	0.5645	0.3868	0.5003
	0.2	0.5582	0.3798	0.4954
Stagnation Threshold Ratio	0.1	0.5554	0.3741	0.4903
	0.15	0.5645	0.3868	0.5003
	0.3	0.5587	0.3781	0.4932
Expert Sampling Ratio	0.4	0.5541	0.3726	0.4888
	0.6	0.5645	0.3868	0.5003
	0.8	0.5588	0.3790	0.4947

presents the results with each parameter varied while others are fixed at their optimal values. For both initialization and local search phases, a perturbation scale of 0.1 achieves optimal performance in our setting. This scale provides sufficient exploration range while maintaining proximity to reference solutions, which aligns with our Gaussian perturbation strategy that emphasizes local refinement around expert configurations. The stagnation threshold ratio of 0.15 proves most effective for triggering the local search mechanism, allowing adequate standard ABC exploration before intervening with guided search around the global best. For the expert sampling ratio, a value of 0.6 achieves the best balance between leveraging expert knowledge through perturbation-based samples and maintaining population diversity through random initialization. Across all tested parameter ranges, the performance variation remains relatively small, with ROUGE-1 scores ranging from 0.5541 to 0.5645. This stability indicates that the proposed method is robust to parameter settings within reasonable ranges, reducing the need for extensive tuning in practical applications.

5.3.5. Computational Cost Analysis

To evaluate the practical efficiency of the proposed framework, we measure the computational cost of the hyperparameter search phase for all optimizer-based methods on a single NVIDIA GeForce RTX 3090 GPU (24 GB). For fair comparison, all swarm intelligence-based optimization algorithms use the same population size of 10 and the same number of iterations, For fair comparison, all optimization methods use the same population size of $N=10$ and the same total function evaluation budget of approximately 200. $T=10$ is the number of ABC iterations. For algorithms that perform N evaluations per iteration (PSO, DE, GA, GWO, BA), this corresponds to 20 iterations. Grid Search and Random Search also evaluate 200 configurations. ABC family algorithms may slightly exceed 200 evaluations because the scout bee phase generates additional function evaluations when stagnated solutions are reinitialized or locally refined. This section reports the search-phase overhead.

Table 5. Computational cost comparison on a single NVIDIA RTX 3090 GPU (24 GB).

Method	Trainable Params	Trials	Search (min)	Peak Mem. (GB)	ROUGE-1
Traditional Methods
BART + Full FT	140.19 M	—	—	9.91	0.5559
BART + LoRA (Manual)	3.24 M	—	—	9.45	0.5458
BART + DoRA (Manual)	3.3 5M	—	—	13.50	0.5526
Classical Search Methods
BART + DoRA + Grid Search	3.35 M	200	76.4	9.71	0.5533
BART + DoRA + Random Search	3.35 M	200	78.0	9.70	0.5561
Metaheuristic Optimization Methods
BART + DoRA + PSO	3.35 M	200	78.4	9.70	0.5578
BART + DoRA + FA	3.35 M	200	79.4	9.69	0.5557
BART + DoRA + BA	3.35 M	200	74.2	9.69	0.5551
BART + DoRA + GWO	3.35 M	200	70.5	9.69	0.5576
BART + DoRA + DE	3.35 M	200	80.1	9.70	0.5556
BART + DoRA + GA	3.35 M	200	80.0	9.70	0.5560
BART + DoRA + ABC	3.35 M	213	81.0	9.70	0.5576
BART + DoRA + EABC	3.35 M	212	74.6	9.70	0.5579
BART + DoRA + AEABC	3.35 M	243	95.3	9.70	0.5591
BART + DoRA + Enhanced ABC (Ours)	3.35 M	212	81.1	9.70	0.5645

presents the computational cost of all methods, organized by trainable parameters, actual trials, search time, peak GPU memory, and ROUGE-1. Since all optimizer-based methods share the same final DoRA training phase (approximately 207 min, 3.45 GPU hours), the search time and peak memory columns isolate the overhead introduced by each optimization strategy, and ROUGE-1 is included to facilitate direct comparison of efficiency against performance. From a parameter efficiency perspective, DoRA-based methods reduce trainable parameters from 140.19 M (full fine-tuning) to 3.35 M, achieving competitive or superior ROUGE-1 when combined with automated optimization. For the search-phase overhead, the per-trial evaluation cost is dominated by model training rather than optimizer computation, so methods with the same 200-trial budget exhibit similar search times regardless of the optimization strategy. AEABC triggers substantially more scout bee evaluations (243 trials) due to its adaptive exploration mechanism, leading to the highest search cost of 95.3 min, yet its ROUGE-1 (0.5591) remains lower than that of the proposed Enhanced ABC (0.5645). The proposed Enhanced ABC completes the search in 81.1 min with 212 trials, comparable to standard ABC (81.0 min, 213 trials), indicating that the hybrid initialization and stagnation-guided local search do not introduce additional computational overhead. Considering both search and training phases, the total pipeline cost of Enhanced ABC is approximately 4.81 GPU hours, adding 39% overhead compared to DoRA with manual tuning (3.46 GPU hours) while achieving the best ROUGE-1 among all methods and eliminating the need for manual hyperparameter selection. All optimizer-based methods operate within a peak GPU memory of 9.70 GB during the search phase, well below the 13.50 GB required during the full DoRA training phase, confirming that the optimization pipeline does not impose additional memory constraints.

5.3.6. Faithfulness Analysis

This section provides a qualitative analysis to evaluate the faithfulness of the generated summaries on the Chinese news dataset. Five annotators with NLP background independently annotated 30 randomly sampled test outputs. All disagreements were resolved by majority vote. Following the standard hallucination taxonomy [36], two error types were defined. Intrinsic hallucination occurs when the generated content contradicts the source document. Extrinsic hallucination occurs when the generated content cannot be verified from the source.

Table 6. Human evaluation results on 30 randomly sampled test outputs.

Error Type	Occurrence
Intrinsic Hallucination	2/30
Extrinsic Hallucination	1/30
No Hallucination	27/30

summarizes the evaluation results.

The hallucination rate is low overall, with 90% of the outputs containing no hallucination errors. Among the observed errors, one intrinsic hallucination case involves a subtle but critical entity-level alteration. The model changes a collective noun phrase referring to a group of people into a singular individual, fundamentally misrepresenting the scale of the reported event. Additionally, an instance of extrinsic hallucination manifests as the model introducing an unverified temporal marker absent from the source document. This suggests that the model occasionally defaults to its pre-trained parametric knowledge rather than strictly grounding its output in the provided context. Overall, the generation behavior is highly extractive, consistent with the high ROUGE-1 score of 0.5617 on the test set.

5.3.7. Case Study

The industrial corpus was preprocessed through a standard pipeline before evaluation. Raw maintenance reports were exported from the communication operation and maintenance management system and converted to plain text format. Format normalization was then applied to remove encoding errors, special symbols, and redundant whitespace. Documents were filtered by length to exclude records that were too short to contain substantive content or too long for effective processing. Near-duplicate records were removed to avoid redundancy in evaluation. To comply with the data confidentiality requirements of the pipeline network operator, sensitive information including personnel identifiers, equipment serial numbers, and specific geographic location references was de-identified by replacing with category-level placeholders prior to evaluation. Figure 5 illustrates a representative example of the proposed framework applied to real communication maintenance documentation from the oil and gas pipeline network. The left panel presents the original technical reports containing extensive procedural narratives, geographic descriptions, and material specifications distributed across multiple verbose paragraphs. The right panel displays the corresponding generated summaries, which automatically condense and reorganize essential information into concise work orders comprising objectives, operational scope, and construction locations. The model demonstrates strong capability in distinguishing critical technical content from redundant descriptive passages, accurately preserving domain-specific terminology, quantitative specifications, and spatial parameters while achieving substantial length reduction. This summarization format aligns well with practical work order requirements in industrial maintenance workflows, validating the applicability of integrating parameter-efficient fine-tuning with automated hyperparameter optimization for specialized technical domains.

As shown in Figure 6, both methods correctly retain the precise cable length, the number of relay segments, and the fault description (i.e., 4 fiber breaks and high-attenuation points). However, the manually tuned baseline summary omits two technically important pieces of information: the number of branch cables and the quarterly test trigger. In pipeline communication maintenance workflows, branch cable count directly determines the scope of repair work orders, since branch cables are independent access structures that require separate inspection and repair assignments. Omitting this information may cause work orders to overlook affected branches. The quarterly test trigger distinguishes planned maintenance from emergency response, which affects work order priority classification in practice. The proposed method retains both items, showing that automated hyperparameter optimization yields better summarization performance compared to manual tuning.

6. Conclusions

This paper presents an efficient and robust framework for domain-specific text summarization based on parameter-efficient fine-tuning and automated hyperparameter optimization. By integrating weight-decomposed low-rank adaptation with an enhanced Artificial Bee Colony algorithm, the proposed method addresses the dual challenges of high computational cost and hyperparameter sensitivity when adapting large pretrained models to specialized domains.

The enhanced optimization strategy introduces a hybrid initialization mechanism and a stagnation-guided local search scheme, which improve solution quality by leveraging prior knowledge while preserving useful search directions. Experimental results on a public Chinese summarization benchmark and a real-world oil and gas pipeline communication corpus demonstrate consistent improvements over full fine-tuning, manually tuned parameter-efficient methods, and several representative evolutionary optimizers. These results confirm that effective hyperparameter search plays a critical role in unlocking the full potential of parameter-efficient adaptation.

Nevertheless, several limitations of the current framework should be noted. First, experiments are conducted on Chinese-language corpora only, which limits conclusions about cross-lingual generalizability. Second, the current evaluation budget is feasible for BART-scale models but may become a practical bottleneck when scaling to larger language models with higher per-trial training cost. Future work will explore extending the proposed framework to larger language models and additional sequence generation tasks, as well as incorporating resource-aware optimization and distributed search strategies to further enhance scalability and practical applicability in industrial environments. Evaluation on corpora from other specialized application domains and additional languages, along with statistical significance testing, is also planned to further validate the transferability and robustness of the proposed framework.