Deep Lifelong Learning Optimization Algorithm in Dense Region Fusion

Abstract

Deep lifelong learning models can learn new information continuously while minimizing the impact on previously acquired knowledge, and thus adapt to changing data. However, existing optimization approaches for deep lifelong learning cannot simultaneously satisfy the following conditions: unrestricted learning of new data, no use of old data, and no increase in model parameters. To address this problem, a deep lifelong learning optimization algorithm based on dense region fusion (DLLO-DRF) is proposed. This algorithm first obtains models for each stage of lifelong learning, and divides the model parameters for each stage into multiple regions based on the parameter values. Then, based on the dispersion of the parameter distribution, dense regions are dynamically obtained from the divided regions, and the parameters of the dense regions are averaged and fused to optimize the model. Finally, extensive experiments are conducted on the self-labeled transmission line defect dataset, and the results show that DLLO-DRF has the best performance among various comparative algorithms.

1. Introduction

Deep lifelong learning has gained significant attention in recent years [1,2,3]. It requires multiple stages and iterations, and generates multiple corresponding models in each stage. These models at different stages contain varying depths of knowledge. Utilizing this knowledge in an integrated manner can enhance lifelong learning model performance. However, since lifelong learning involves continuously acquiring data, it is impractical for all data to be obtained in a single training session. Moreover, in different training stages, the previously acquired knowledge is affected due to the random update of parameters, which subsequently leads to the decrease in the overall performance of the lifelong learning model.

Another factor leading to lower performance of the model is the sense of fragmentation introduced during the staged training of deep lifelong learning. This fragmentation hampers the overall parameter update process and prevents the trained model from reaching its optimal performance, even when using the same final dataset. Therefore, addressing these challenges remains a focus of current research on deep lifelong learning.

Currently, research on deep lifelong learning has focused on optimizing neural networks through three main approaches. The first approach involves restraining the number and magnitude of neuron parameter updates by introducing various regularization terms into the loss function. For example, the classic EWC [4] method penalizes changes to important parameters in subsequent task training through elastic weight merging. LwF [5], LFL [6], and DMC [7] use the output of the previous model as a soft label for the previous task to reduce forgetting and knowledge transfer. The second approach aims to enable neural networks to review previously learned knowledge through data replay. This is achieved by employing a standard first-in-first-out (FIFO) buffer and selectively storing old knowledge in long-term memory. The third approach involves expanding the network’s capacity to retain more knowledge. This can be accomplished by employing redundant parameters in large deep networks, allowing them to accommodate the learning of new tasks while preserving previously acquired knowledge.

These above approaches offer the advantage of not requiring explicit consideration of previously trained models when new data arrives. However, these approaches also present several challenges. Firstly, they often entail imposing restrictions on the learning of new data during model training. Secondly, they rely on using old data to “review” the model, which can be cumbersome. Lastly, expanding the model’s parameter count to retain more knowledge may lead to increased complexity. Inspired by the theory of model integration technology, we propose an optimization method for deep lifelong learning based on dense region fusion (DLLO-DRF). This method aims to enhance the performance of the final lifelong learning model by effectively integrating the model parameters from each stage of lifelong learning [8,9,10]. By searching for model weight parameters that possess the capability to express data at all stages of lifelong learning, we can achieve significant improvements in the performance of lifelong learning models.

This paper makes the following key contributions:

• A deep lifelong learning optimization method, DLLO-DRF, is proposed, and its corresponding objective function is designed. This method analyzes the differences in the distribution of model parameters in different stages of lifelong learning, and then dynamically integrates these model parameters to obtain weight parameters that effectively capture the overall data patterns.
• Dense region fusion is introduced into the objective function of DLLO-DRF, and the corresponding algorithm is designed.
• Comparative experiments are conducted using self-labeled transmission line defect datasets, and with various deep object detection algorithms. The results show that DLLO-DRF effectively improves the performance of deep neural network models under lifelong learning.

The following sections are organized as follows: Section 2 provides a review of existing works that are relevant to the subject matter discussed in this paper. Section 3 presents the materials and methods in detail, including the objective function and inference process of DLLO-DRF, and experimental material. Section 4 focuses on presenting the obtained results in detail. Finally, Section 5 concludes the paper by summarizing the key findings and discussing the experimental results, as well as drawing overall conclusions based on the research conducted.

2. Related Work

2.1. Deep Lifelong Learning

The concept of lifelong learning [11] was first proposed by Thrun and Mitchell in 1995. In the past few years, the rapid growth and popularity of deep learning has encouraged many researchers to focus on the challenge of continual learning using deep structures [12]. Deep lifelong learning is driven by several key motivations. One is to address the problem of catastrophic forgetting while learning a series of tasks [13], i.e., gradually learn each new task within the same neural network without causing the neural network to forget the model learned from past tasks. The second is how to utilize previously learned knowledge to help better learn new tasks. At present, the optimization methods of deep lifelong learning mainly include methods based on regularization, methods based on knowledge replay, and methods based on parameter isolation.

Based on regularization methods, the regularization term is applied to the loss function to avoid over-updating the parameters of the new model, which in turn prevents the model from forgetting the old knowledge. Lee et al. [14] proposed incremental moment matching, which employs transfer learning techniques such as controlling the posterior parameter search space using L2 norms of old and new parameters to mitigate the forgetting problem. Feng et al. [15] introduced elastic response distillation, an incremental distillation method based on elastic response. This approach focuses on learning the response elastically from the tail class or regression head of the network, evaluating the prediction quality, and selecting valuable responses through elastic response selection strategies. Liu et al. [16] utilized network reparameterization to generate an information matrix based on the previous model training results prior to training a new class in the network. During this training, the gradient direction of parameter updates is adjusted by referencing the information matrix.

Based on knowledge replay, different approaches have been proposed to address the challenge of catastrophic forgetting and leverage previously learned knowledge. Mo Jianwen et al. [17] constructed pseudosamples through a generative model to train them together with new data, effectively constraining the over-updating of parameters and reducing the impact of new data on old knowledge. Isele et al. [18] selectively stored old knowledge in long-term memory using a standard first-in-first-out buffer. They aimed to find the maximum state space coverage by matching data distribution during training, which helps prevent catastrophic forgetting. Inspired by the generation characteristics of the hippocampus as the short-term memory system in the primate brain, Shin et al. [19] proposed a deep generation replay framework. This framework employs a cooperative dual model architecture consisting of a deep generative model and task solutions. By leveraging these two models, the training data of the old task can be easily sampled and effectively fused with the training data of the new task, facilitating knowledge transfer.

Based on parameter isolation methods, Mallya et al. [20] proposed a method to adapt a single deep neural network to multiple tasks without affecting the performance of already-learned tasks, inspired by network quantization and pruning. This method achieves good performance for each task by adding binary masks to the neurons of the original network and assigning different weights to different tasks. Roy et al. [21] proposed an adaptive hierarchical network structure composed of deep convolutional neural networks, which can grow and learn with the emergence of new data. The network grows in a tree-like manner to accommodate new data categories, while also having the ability to distinguish between previously trained categories. Aljundi et al. [22] combined an autoencoder with an expert network to introduce a set of autoencoders for each new task to learn the representation of the current task. During testing, the test samples are automatically forwarded to the relevant expert network.

In addition, there are many lifelong learning optimization methods specific to different application domains of deep neural networks. Chen et al. [23] proposed a meta-learning method that enables the model to adapt to unseen knowledge while overcoming the forgetting problem. This method uses base class samples to train the meta-learner, thus providing better initialization weights, and then fine-tunes the model to mitigate the forgetting problem by retaining the weights associated with the base class. Ramakrishnan et al. [24] proposed a selection mechanism for labeled objects in new classes for the characteristics of two-stage target detection network models. Then, they designed a relationship-guided loss function based on this model, aiming to preserve the relationship of the proposed selection between the base network and the new network trained on other classes. Yan et al. [25] proposed a two-stage learning method that uses dynamically scalable representations for more efficient incremental modeling. The previously learned representation is frozen in each incremental step, and additional feature dimensions of the new feature extractor are used to perform the expansion.

However, existing approaches for deep lifelong learning have some constraints, such as having a greater impact on early tasks, increasing the cost of training, and producing a very large model.

2.2. Deep Model Integration

Deep model integration provides more robust and accurate models by effectively fusing the parameters of multiple models without increasing computational complexity during inference. This makes deep model integration an ideal approach for practical applications that require high accuracy and low inference time. Up to now, there have been two common approaches to obtain the final inference model. One is to train multiple identical models with different training data and then average the weights of the multiple training models. The other is to average the weights of the models over different iterations.

Tarvainen et al. [8] introduced a time-series-based method for averaging model weights that can enhance testing accuracy and utilize fewer labels for training compared to conventional approaches. Huang et al. [26] proposed a technique for fusing multiple neural networks without added training costs by optimizing multiple local minima on the path and employing them as parameters for target neural networks. Izmailov et al. [27] suggested a random weight averaging parameter update approach that offers superior generalization compared to traditional training. Garipov et al. [28] asserted that optimal complex loss function values are connected by simple curves, leading to the development of fast geometric fusion for efficient integrated model training. Frankle et al. [29] analyzed neural networks’ optimization towards linearly connected minimum values under varying data conditions, discovering that stability and prediction accuracy can be improved through random data selection. Matena [9] introduced “Fisher merging,” an alternative merging process utilizing Laplacian approximation that enhances model performance. Von et al. [10] showed improvement in random gradient descent by integrating specific weights and spatially averaging weight space. Wortsman et al. [30] proposed a fine-tuning method that accounts for distribution changes without additional computational costs during the inference stage. Kaddour et al. [31] explored a flat minimum optimizer for training neural networks, merging average and minimax methods to obtain a flatter solution. Neyshabur et al. [32] observed that averaging weight parameters of different instances could share information and improve model robustness and accuracy. In the context of fine-tuning pre-training, Wortsman et al. [33] recommended using the average of multiple model weight fine-tuning events under various hyperparameter configurations, which typically enhances model accuracy and robustness.

The benefits of deep model integration are not limited to improving the overall performance. It can also provide greater model interpretability and help to mitigate the risks associated with model bias or overfitting. Therefore, deep model integration is a promising approach for the future of deep learning.

3. Materials and Methods

3.1. Overall Structure of DLLO-DRF

To address the limitations and challenges associated with various deep lifelong learning algorithms, we introduce DLLO-DRF as a model integration approach. The core concept of DLLO-DRF entails that models generated in each stage of lifelong learning should embody enhanced knowledge concerning the tasks within that stage. For the neural network model, the knowledge is represented by the learned neuron parameters. Thus, by effectively integrating the model parameters from each stage, the overall performance of the deep lifelong learning model can be improved. The complete structure of DLLO-DRF is illustrated in Figure 1.

In DLLO-DRF, the dataset is partitioned and trained separately. The first dataset $D_1$ is used for the training of the first stage to obtain Model 1. During each subsequent stage, the dataset for the current stage and the previously trained model are employed to train the present stage’s model. Consequently, DLLO-DRF effectively derives trained models from multiple stages. Then, by selecting and fusing the same network parameter modules from different stages, the fused network parameter modules are obtained. Finally, the fused model is obtained by combining other network modules.

For convenience, important notations are listed in

Table 1. Important notations used in this paper.

Notation	Description
D	Initial training dataset
$d_n$	The training dataset used in the $n^{th}$ stage in lifelong learning
$mode{l}^{\left(n\right)}$	The model in the $n^{th}$ stage in lifelong learning
$x_i^{\left(k\right)}$	The parameter at position i of $mode{l}^{\left(k\right)}$
$X_i$	The parameter set encompassing all stage models at position i
j	The number of regions that divided
$x_{max}$	The maximum value in $X_i$
$x_{min}$	The minimum value in $X_i$
$X_{dense}$	The dense region
$x_i^{fusion}$	The parameter at the $i^{th}$ position in the fusion model
$\left|X\right|$	The number of elements in the set X
$size$	The size of region

.

3.2. Algorithm Principle in DLLO-DRF

DLLO-DRF integrates parameters that are obtained from multiple stages of lifelong learning. The algorithm consists of two parts. The first part is the obtaining of models for each stage of lifelong learning, and the second part is the dense region fusion on the network model parameters. This section provides a detailed introduction to these two parts separately.

3.2.1. Lifelong Learning Paradigm Based on Model Fine-Tuning

A lifelong learning algorithm that is based on the model fine-tuning approach requires the condition that when new information is learned, only previous knowledge is tuned, instead of relearning all the information. This can significantly reduce computational complexity and allow the model to learn new information faster. The fine-tuning approach can take full advantage of the powerful generalization capabilities of deep learning neural networks, and can avoid the need to design new models and train from the beginning, converging faster and with better results. This offers many advantages, such as the model having a high initial performance before training, gaining faster improvement of the model during training, and having better convergence than the obtained model. This lifelong learning paradigm is universal, and it can be applied to all deep learning algorithms. The models trained through this paradigm also provide support for fusion. The lifelong learning paradigm based on model fine-tuning can be summarized as using data from the current stage of lifelong learning tasks, and using the task model of the previous stage as a pre-training model to train the task model of the current stage. The following paragraph provides a definition of the model-fine-tuning-based lifelong learning paradigm.

The initial training dataset, noted as D, is divided into n parts: $D=\{d_1,d_2,\dots ,d_n\}$. The training dataset used in the $n^{th}$ stage in lifelong learning is defined as $D^{\left(n\right)}=d_n$. $\mathrm{\Gamma }\{A,B\}$ represents the model obtained by employing training dataset A to train pre-trained model B using algorithm $\mathrm{\Gamma }$. $Mode{l}^{\left(n\right)}$ represents the model obtained in the $n^{th}$ stage, and is defined as follows:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill Mode{l}^{\left(n\right)}=\left\{\begin{array}{cc}\mathrm{\Gamma }\{D^{\left(n\right)},Mode{l}^{\left(other\right)}\},\hfill & \hfill n=1\\ \mathrm{\Gamma }\{D^{\left(n\right)},Mode{l}^{(n-1)}\},\hfill & \hfill n\ge 2,\end{array}\right.\end{array}\end{array}$$

(1)

where $Mode{l}^{\left(other\right)}$ represents other pre-trained models that can be selected in the first stage.

3.2.2. Dense Region Fusion

In order to observe the distribution of parameters under lifelong learning, we conducted 10 stages of training on the Cascade R-CNN object detection algorithm using the model fine-tuning lifelong learning training paradigm. Figure 2 shows some parameters in the model changing with the task stage, and it can be observed that these parameters are often concentrated in one of the regions. The region with the highest concentration of parameters is referred to as the dense region. For neural networks, these aggregated parameters are more valuable to the model itself, as they often carry the most important knowledge within the model.

Based on the above findings, dense region fusion divides the parameters of models from multiple task stages of lifelong learning into regions at the same location, and takes the average of all parameters in the region with the densest parameter distribution as the parameters of the fusion model. Therefore, the model fused in this way can maximize the information of all stages of lifelong learning tasks. The main steps and related formulas of dense region fusion are introduced in the following paragraphs.

Through the training method defined in Section 3.2.1, we can obtain the model for each stages: $Model=mode{l}^{\left(1\right)},mode{l}^{\left(2\right)},\dots ,mode{l}^{\left(n\right)}$. For deep neural network models, the parameter dimensions correspond to the size of the convolutional kernel. Typically, the convolutional kernel size varies across different layers. To enable the fusion process, it is crucial to map the parameter vectors in the network onto a one-dimensional space, as the convolutional kernel size varies across layers. The dimension mapping algorithm is illustrated in Algorithm 1. Through dimension mapping, the parameter at position i of the $k^{th}$ stage model is represented as $x_i^{\left(k\right)}$. Consequently, the parameter set encompassing all stage models at position i is $X_i=x_i^{\left(1\right)},x_i^{\left(2\right)},\dots ,x_i^{\left(n\right)}$. In order to identify the most densely distributed region in $X_i$, it is essential to partition $X_i$ into distinct regions, with each region’s size determined by the parameter range of $X_i$, computed using Equation (2).

$$size=\frac{x_{max}-x_{min}}{j},$$

(2)

where j is the number of regions that are divided, and $x_{max}$ and $x_{min}$ are the maximum and minimum values in $X_i$, respectively.

Algorithm 1: Dimension Mapping

Input: $Model=\{mode{l}^{\left(1\right)},mode{l}^{\left(2\right)},\dots ,mode{l}^{\left(n\right)}\}$: The set of the trained model. Output: $Model$: Model set with weight parameters reduced to one dimension. $Shape$: Original model parameter dimension. 1 Obtain the parameter dimension $shape$ in $Model;$ 2 For each model, $mode{l}^{\left(i\right)}$, traverse all parameters based on their dimensions and merge them into a one-dimensional array; 3 For each model, $mode{l}^{\left(i\right)}$, replace all the parameters to an array obtained in the last step.

The region division implementation is depicted in Algorithm 2. Through region division, the j subset of $X_i$ parameter space for all models at position i can be acquired, denoted as $X_{i1},X_{i2},\dots ,X_{ij}$. The objective of dense region fusion is to identify the subset with the highest number of parameters within the $j^{th}$ subsets, namely, the dense region $X_{dense}$. To circumvent the uncertainty in dense region selection when parameter quantities are relatively evenly distributed across different regions, which could compromise the fusion model’s robustness, the dense region $X_{dense}$ is determined based on $X_{i1},X_{i2},\dots ,X_{ij}$. The parameter quantity’s variance $s^2$ is calculated for each subset, and when $s^2$ falls below a predefined threshold, the original parameter set $X_i$ is directly treated as the dense region. The calculation of $X_{dense}$ is carried out using Equation (3).

$$X_{dense}=\left\{\begin{array}{cc}{\displaystyle X_i}\hfill & s^2<V\hfill \\ {\displaystyle arg\underset{X_{ij}\in X_i}{max}\left|X_{ij}\right|}\hfill & s^2\ge V,\hfill \end{array}\right.$$

(3)

where $\left|X\right|$ represents the number of elements of X, and V is the set threshold of variance.

Algorithm 2: Region Division

Finally, the mean value of the parameters in $X_{dense}$ is used as the parameter at the ith position in the fusion model $x_i^{fusion}$, and it can be calculated using Equation (4).

$$x_i^{fusion}=\frac{1}{|X_{dense}|}\sum _{x\in X_{dense}}x,$$

(4)

where $\left|X\right|$ represents the number of elements in the set X.

We also offer an approach for directly identifying dense regions. This approach involves constructing a network model that employs short- and long-term neural networks to learn the boundaries of dense regions within a given parameter space, with the model being referred to as a dense region divider. The  structure of the region divider is illustrated in Figure 3.

The input for the network consists of the parameters of the $n^{th}$ stage model, followed by a layer of LSTM units with input and output dimensions of $(n,100)$. Subsequently, another layer of LSTM units with input and output dimensions of $(100,100)$ is employed. Finally, a fully connected layer with input and output dimensions of $(100,2)$ is added. This connected layer’s two outputs represent the upper and lower bounds of the learned dense region. By utilizing this region divider, dense regions can be obtained as an alternative to employing Algorithm 2 and Equation (3).

Based on the principles and steps of the above algorithms, an algorithm for dense region fusion has been developed, which is presented in Algorithm 3.

Algorithm 3: Dense Region Fusion

3.2.3. Process of DLLO-DRF

Based on the structure and principles of DLLO-DRF, we give the detailed process of DLLO-DRF in Figure 4. Initially, DLLO-DRF divides the training set and proceeds to train according to the lifelong learning paradigm defined in Section 3.2.1. Subsequently, the parameters of models across all stages are obtained at the overlapping position of the fusion module, followed by the division of these parameters into regions. Finally, the mean value of all parameters within the densest region is utilized as the parameters of the fusion module at the corresponding positions. This step is iterated until all the average values of parameters across every position within the module are fused.

3.2.4. Complexity Analysis

In DLLO-DRF, primary variables include m (number of training stages) and n (number of fusion parameters). It contains two components: lifelong learning training and dense region fusion. The time complexity of obtaining parameters and partitioning them is $O\left(m\right)$ and $O\left(n\right)$ for a single model, resulting in a total of $O\left(mn\right)$. Spatial complexity, dependent on model number and size, is also $O\left(mn\right)$.

3.3. Experimental Material

3.3.1. Dataset

In order to evaluate the performance of DLLO-DRF, a self-labeled transmission line defect detection dataset is utilized, which includes a bird’s nest foreign object dataset, a cement pole damage dataset, an anti-vibration hammer slip dataset, and an insulator self-explosion dataset. These four datasets consist of 385, 384, 385, and 376 images, respectively. Due to the varying degrees and damage methods, defect characteristics may differ within the same defect type, causing uneven data distribution. Figure 5 presents two images from the bird’s nest foreign object dataset, wherein the left image exhibits a bird’s nest and the right one a honeycomb. Despite their distinct types and characteristics, they both pertain to the bird’s nest foreign object category. Lifelong learning training methods are preferable since the model must handle data within the same category but with varying features in the future. Consequently, employing these four datasets in experiments on the proposed algorithm is more persuasive.

The detailed information of the transmission line defect dataset is shown in

Table 2. The detailed information of four defect datasets.

Dataset	Total	Training Set	Validation Set
Bird’s nest foreign body	385	300	85
Damaged cement pole	384	300	84
Shockproof hammer slipping	385	300	85
Insulator self-explosion	376	300	76

.

3.3.2. Evaluation Metrics

In the field of object detection, the most commonly used evaluation metrics include Intersection over Union, accuracy, precision, recall, $AP$ value, $AR$ value, and $mAP$ value. This section provides a detailed introduction to these evaluation metrics.

Intersection over Union ($IoU$) can be used to determine the degree of overlap between two images. In the field of object detection, the degree of overlap between two boxes can be determined. The calculation of $IoU$ is shown in Equation (5).

$$IoU=\frac{area(B_p\cap B_{gt})}{area(B_p\cup B_{gt})},$$

(5)

where $B_{gt}$ represents the border of the actual object in the image, $B_p$ represents the bounding box predicted by the detection algorithm, and $IoU$ is calculated based on these two values, which can measure the accuracy of the bounding box predicted by the detection model. In object detection, prediction boxes with $IoU>0.5$ are generally classified as positive samples, while the rest are classified as negative samples.

Moreover, the correct prediction of positive samples as the number of positive samples is called true ($TP$), the correct prediction of negative samples as the number of negative samples is called true negative ($TN$), the incorrect prediction of negative samples as the number of positive samples is called false positive ($FP$), and the incorrect prediction of positive samples as the number of negative samples is called false negative ($FN$). The accuracy, precision, and recall are calculated as follows:

$$Acc=\frac{TP+TN}{TP+FN+FP+TN}.$$

(6)

$$Pre=\frac{TP}{TP+FP}.$$

(7)

$$Recall=\frac{TP}{TP+FN}.$$

(8)

By combining accuracy and recall, a PR curve can be drawn, with recall as the horizontal axis and accuracy as the vertical axis. The larger the area between the PR curve and the axes, the better the performance of the detection model.

The average precision ($AP$) represents the area enclosed by the PR curve and the coordinate axis, which can be calculated using Equation (9).

$$AP={\int }_0^{1}P\left(r\right)dr.$$

(9)

$AP$ is a relatively important evaluation metric, which is generally calculated based on different $IoU$ thresholds for $AP$ under various thresholds. For example, $A{P}_{50}$ and $A{P}_{75}$ represent the measured values of $AP$ at $IoU$ thresholds of 0.5 and 0.75, respectively. Furthermore, it is possible to calculate all $AP$ values separately with an $IoU$ step of 0.05 from $A{P}_{50}$ to $A{P}_{95}$, and then take the average value and record it as $A{P}_{50:95}$. This value can fully evaluate the detection effect from more perspectives.

The average recall ($AR$) is the average of all recalls for $IoU$ from 0.5 to 1.0, and its calculation formula is shown in Equation (10):

$$AR=2{\int }_{0.5}^{1}recall\left(o\right)do.$$

(10)

The mean average precision ($mAP$) of categories is the average value of $AP$s under each category. In the field of object detection, it may be necessary to simultaneously detect multiple categories of objects, and $mAP$ can effectively evaluate the effectiveness of a model in predicting multiple categories of targets. The calculation of $mAP$ is shown in Equation (11).

$$mAP=\frac{{\sum }_{i=1}^{K}A{P}_i}{K},$$

(11)

where K is the number of categories detected, and $A{P}_i$ represents the $AP$ value under the ith category. If there is only one detection category, then $mAP$ is equal to $AP$.

3.3.3. Experimental Setting

The experimental hardware device used a host of the Ubuntu 22.04 LTS operating system, and was equipped with an AMD Ryzen 5600x CPU, NVIDIA TITAN X graphics card, and 64 GB of memory. The software was programmed using Python as the programming language, and Pycharm was used to write the code for the deep learning framework, which was based on Pytorch.

For the settings of the basic algorithm, the datasets used are the four transmission line defect datasets mentioned in Section 3.3.1. The deep neural network model used is the object detection model, which includes three single-stage object detection algorithms, ATSS [34], DDOD [35], and GFL [36], and six two-stage object detection algorithms, which are Cascade R-CNN [37], Faster R-CNN [38], Dynamic R-CNN [39], Grid R-CNN [40], Libra R-CNN [41], and Sparse R-CNN [42]. The feature extraction network adopts a 50-layer residual network ResNet50 [43] and a feature pyramid network FPN. All models have a batch size of 2 and a learning rate of 0.02.

The DLLO-DRF algorithm was set up by dividing the training set into 10 parts and training each algorithm model for 10 stages using the lifelong learning method defined in Section 3.2.1. The FPN module parameters were selected for dense region fusion in the basic algorithm, with a region division number of 3 and a variance threshold of 2.

Furthermore, $AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$ were selected as the evaluation metrics. The $AP$ metric is consistent with the one used in the COCO dataset [44], which refers to $A{P}_{50:95}$, as mentioned in the summary.

3.3.4. Friedman Test

The first step of the Friedman test method involves sorting the algorithm performance from good to poor based on the evaluation metrics in each dataset, and assigning order values of 1, 2, 3, etc. If the algorithm performance is the same, the order values are divided equally. Once the order values for each algorithm have been assigned, the average order value is calculated across all datasets. The second step involves calculating the test statistics required for the test, according to Equations (12) and (13).

$${\tau }_{{\chi }^2}=\frac{12N}{k(k+1)}\left(\sum _{i=1}^k{r}_i^2-\frac{k{(k+1)}^2}{4}\right),$$

(12)

$${\tau }_F=\frac{(N-1){\tau }_{{\chi }^2}}{N(k-1)-{\tau }_{{\chi }^2}},$$

(13)

where k represents the number of algorithms, N represents the number of datasets, $r_i$ is the average order value of the $i^{th}$ algorithm, and the final calculated ${\tau }_F$ follows the F distribution with degrees of freedom of $k-1$ and $(k-1)(N-1)$.

The significance $\alpha$ is often set to 0.05 in hypothesis testing. The original assumption is that there is no significant difference in the performance of each algorithm. When the calculated $\rho$ is greater than $\alpha$, the original assumption is accepted, that is, there is no significant difference in the performance of each algorithm. When the calculated $\rho$ is less than $\alpha$, the original assumption is rejected, meaning that there are significant differences in the performance of each algorithm.

4. Results and Analysis

4.1. Experiment Effect of Lifelong Learning Training

For datasets with small differences in data distribution, a good lifelong learning model can make the model better, as more data becomes available. For optimization algorithms, a good lifetime learning pattern can also provide more effective information for subsequent fusions. To verify the effectiveness of the lifetime learning approach for the defective dataset mentioned in Section 3.2.1, eight object detection algorithms are selected in this section, and trained on the impact hammer sliding dataset according to the experimental setup described in Section 3.3.3.

Table 3. The $AP$ index of each algorithm at different training stages under the sliding dataset of the shock hammer.

Algorithm	1th	2th	3th	4th	5th	6th	7th	8th	9th	10th
ATSS	15.3	21.3	24.3	24	25.6	27.5	26	27.2	27.1	27.7
DDOD	22.7	25.4	31	30.9	31.2	30.3	31.8	31.3	31.9	33
GFL	17	22.5	27.1	26.4	26.7	28.3	28.8	28.8	30.7	29.4
Cascade R-CNN	25.8	37.1	39.5	39.6	39.7	38.4	38.8	40	39.9	39.9
Faster R-CNN	23.8	35.3	37.6	38.9	38.4	36.9	38.3	37.8	40.9	41.4
Dynamic R-CNN	27.5	32.5	35.4	38	38.8	36	38.1	37.8	37.3	37.7
Grid R-CNN	18.2	33.8	36.6	37.2	40.3	37.6	38.6	40.5	40.6	40.2
Libra R-CNN	23.1	30.7	32.6	35.5	34.9	34.3	35.3	36	36.2	35.7

,

Table 4. The $A{P}_{50}$ index of each algorithm at different training stages under the sliding dataset of the shock hammer.

Algorithm	1th	2th	3th	4th	5th	6th	7th	8th	9th	10th
ATSS	48.7	58.1	61.3	61.2	60.6	64.3	61.7	63.2	63.4	64
DDOD	52.4	57.7	66.7	65.9	65.9	63.3	65.5	65.4	64.8	66.6
GFL	51.2	54.6	61.3	62.6	62.5	62	64.2	63	65.5	63.8
Cascade R-CNN	57.5	63.9	65.6	63.4	61	58.4	60	60.7	61.2	61.9
Faster R-CNN	52.1	62.1	63.8	65.7	63.9	61.9	63.4	61.9	65.8	66.9
Dynamic R-CNN	51.9	60.3	61.2	62	62.4	59.9	60.7	60.4	59.9	60.2
Grid R-CNN	42.6	65.2	66.1	64.9	68.9	62.3	62.6	66.8	64	63.7
Libra R-CNN	49.2	57.3	58.2	61.4	60.8	62.4	61.1	64.8	61.1	61.2

,

Table 5. The $A{P}_{75}$ index of each algorithm at different training stages under the sliding dataset of the shock hammer.

Algorithm	1th	2th	3th	4th	5th	6th	7th	8th	9th	10th
ATSS	6	9.9	12.2	10.5	17.1	16.7	17.3	18.1	18	19.3
DDOD	14.3	23.6	27.2	28.4	29.7	29.3	31.6	27.3	29.7	31.5
GFL	6.1	12.2	18.1	19	17.2	19.4	18.7	20.5	24.8	20.1
Cascade R-CNN	23.3	36.6	45.2	45.5	47.4	44.3	45.4	45.4	46.7	48.2
Faster R-CNN	20.3	35.2	38.3	42.7	44.7	43.3	43.6	44.4	44.2	46.2
Dynamic R-CNN	28.1	33.2	33	40.2	42.7	40	42.7	43.4	42.1	42.4
Grid R-CNN	11.8	32.7	38.9	43.9	43.4	39.8	45.6	43.4	45.9	44.4
Libra R-CNN	20.7	32.6	32	37	37.7	34.3	37.1	37.6	38.5	38

and

Table 6. The $AR$ index of each algorithm at different training stages under the sliding dataset of the shock hammer.

Algorithm	1th	2th	3th	4th	5th	6th	7th	8th	9th	10th
ATSS	33.3	36.7	37.9	37.2	37.4	38.5	37.4	38	38.3	37.8
DDOD	36.1	36.8	39.3	38.8	39	37.3	39	39.3	39.5	40.6
GFL	34.4	36.9	40	41.1	41.3	40.5	41.4	40.1	42.1	40.3
Cascade R-CNN	35.4	44.6	46.1	45.4	45.1	43.2	43	44.6	44.5	44.6
Faster R-CNN	35.9	43	44.1	46.2	44.7	44.2	44.7	45.1	46.1	47.3
Dynamic R-CNN	38.5	40.3	43	46.1	46.4	43.6	45.1	46	44.6	45.7
Grid R-CNN	30.7	41.3	43	43.2	45.7	43.8	44.6	46.8	46.8	46.9
Libra R-CNN	38.6	42.6	44.3	45.9	46.2	45.4	46.2	46.2	46.9	46.3

relate to $AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$, respectively. These four metrics demonstrate the performance variation of each algorithm at each stage of lifelong learning.

The results indicate that the overall performance of the eight object detection algorithms used in the experiment shows an upward trend with the increase in lifelong learning stage across all four metrics. To better illustrate the changes in the model’s overall performance, we plotted line charts for each algorithm’s performance under the four metrics ($AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$) with the training stage, as shown in Figure 6. The results show that all metrics rapidly increased in the first three stages, and the growth rate slowed down in the last seven stages, while the overall trend remained upward. These observations confirm that the lifelong learning training paradigm defined in Section 3.2.1 can effectively demonstrate the performance of various algorithms on the transmission line defect dataset.

4.2. Longitudinal Comparative Experiments on Basic Algorithms

4.2.1. Comparison of DLLO-DRF Detection Algorithms Used and Unused in the Bird’s Nest Foreign Object Dataset

Table 7. Comparison of DLLO-DRF algorithms used and unused in the bird’s nest foreign object dataset. The better results are indicated using bold and underline.

Algorithm	Using DLLO-DRF				Without DLLO-DRF
	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$
ATSS	26.5	66	12.5	39.8	26.8	65.7	11.6	37.7
DDOD	30.1	67.2	16.6	42.4	30.5	66.8	16.2	42.8
GFL	19.3	46.9	11.6	36.6	19.5	46.4	10.6	33.2
Cascade R-CNN	30.2	70.1	14.4	37.6	29.7	70	12.9	37.7
Faster R-CNN	31.1	73	14.2	38.1	30.4	72.9	14.2	38
Dynamic R-CNN	27.7	68.8	11.5	36.5	27.9	67.1	10.8	37.2
Grid R-CNN	29.8	65.2	23.3	38.4	29.5	64.7	23.6	38
Libra R-CNN	26.6	62.9	13.2	37.2	26.8	64.2	12.3	36.9
Sparse R-CNN	25.4	65.2	11.5	40.9	25.5	65.5	11	40.8
Average	27.4	65	14.3	38.6	27.4	64.8	13.7	38

indicates that using DLLO-DRF improved the average values of the metrics for each algorithm in the bird’s nest foreign object dataset, with improvements observed in $A{P}_{50}$, $A{P}_{75}$, and $AR$.

4.2.2. Comparison of DLLO-DRF Detection Algorithms Used and Unused in the Cement Pole Damage Dataset

Table 8. Comparison of DLLO-DRF algorithms used and unused in the cement pole damage dataset. The better results are indicated using bold and underline.

Algorithm	Using DLLO-DRF				Without DLLO-DRF
	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$
ATSS	13.4	37.6	9.3	28.2	13.6	37.1	9.2	29
DDOD	15.6	47.2	12.2	21.2	15.4	47	12.1	21.1
GFL	21.9	44.8	18.7	30.7	21.8	44.9	18.1	30.3
Cascade R-CNN	16.9	35.4	15.2	21.8	16.8	34.2	15.6	21.4
Faster R-CNN	17	33.5	16.5	22.7	16.8	33.4	18.4	23
Dynamic R-CNN	13.4	26	10.5	20.1	13.2	25.6	9.4	19.9
Grid R-CNN	16	37.8	9.1	22.6	15.9	36.2	10.4	22.5
Libra R-CNN	18.8	45.6	13.7	24.1	18	45.7	13.2	23.6
Sparse R-CNN	18.3	47.7	5	26	18.4	47.8	4.9	25.6
Average	16.8	39.5	12.2	24.2	16.7	39.1	12.4	24

shows that using DLLO-DRF improved the average values of the metrics for each algorithm in the cement pole damage dataset, with improvements observed in $AP$, $A{P}_{50}$, and $AR$.

4.2.3. Comparison of DLLO-DRF Detection Algorithms Used and Unused in the Shockproof Hammer Slip Dataset

Table 9. Comparison of DLLO-DRF algorithms used and unused in the shockproof hammer slip dataset. The better results are indicated using bold and underline.

Algorithm	Using DLLO-DRF				Without DLLO-DRF
	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$
ATSS	28.2	64	19.4	38	27.7	64	19.3	37.8
DDOD	33.2	66.8	31.7	40.8	33	66.6	31.5	40.6
GFL	29.8	64.2	20.1	41.1	29.4	63.8	20.1	40.3
Cascade R-CNN	39.6	62	48.3	44.4	39.9	61.9	48.2	44.6
Faster R-CNN	40.9	67.3	46.5	46.7	41.4	66.9	46.2	47.3
Dynamic R-CNN	38	60.6	42.6	46	37.7	60.2	42.4	45.7
Grid R-CNN	40.2	64.4	43.6	47	40.2	63.7	44.4	46.9
Libra R-CNN	35.9	61.1	38.2	46.5	35.7	61.2	38	46.3
Sparse R-CNN	18.3	40.3	14.6	43.5	18.5	40	14.9	43
Average	33.8	61.2	33.9	43.8	33.7	60.9	33.9	43.6

demonstrates that using DLLO-DRF improved the average values of the metrics for each algorithm in the shockproof hammer slip dataset, with improvements observed in $AP$, $A{P}_{50}$, and $AR$.

4.2.4. Comparison of DLLO-DRF Detection Algorithms Used and Unused in the Insulator Self-Explosion Dataset

Table 10. Comparison of DLLO-DRF algorithms used and unused in the insulator self-explosion dataset. The better results are indicated using bold and underline.

Algorithm	Using DLLO-DRF				Without DLLO-DRF
	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$	$\mathit{AP}$	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{AP}}_{\mathbf{75}}$	$\mathit{AR}$
ATSS	13.1	41.8	3.6	34.8	13	41.6	3.6	34.5
DDOD	19.6	53.9	8.3	39.5	19.4	53	6.7	38.8
GFL	16.3	46	5.5	37.8	16.2	45.6	5.5	37.7
Cascade R-CNN	29.5	67.3	19.9	43.7	28.9	65.9	19.6	43.7
Faster R-CNN	16.1	45.3	7.9	31.6	16	44.9	7.9	31.7
Dynamic R-CNN	23.6	61.2	13.4	37.2	23.2	60.6	13.6	37.4
Grid R-CNN	27.6	60.9	19.7	46	27.5	60.4	20.1	46.2
Libra R-CNN	21.1	52.6	14.2	37.2	20.3	51.1	13.7	37
Sparse R-CNN	20.8	49.4	15.3	37.5	20.7	49.4	15.2	37.3
Average	20.9	53.2	12	38.4	20.6	52.5	11.8	38.3

illustrates that using DLLO-DRF improved the average values of the metrics for each algorithm in the insulator self-explosion dataset, with improvements observed in $AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$.

4.3. Horizontal Comparison Experiment with Other Algorithms

4.3.1. Comparison between DLLO-DRF and Other Algorithms in the Bird’s Nest Foreign Object Dataset

As shown in

Table 11. Comparison of DLLO-DRF and other fusion methods in the bird’s nest foreign object dataset. The better results are indicated using bold and underline.

Object Algorithm	Fusion Method	$\mathit{AP}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	$\mathit{AR}$
ATSS	Uniform soup	25.7	63.5	11.6	37.5
	Greedy soup	26	64.4	12.3	37.5
	DLLO-DRF	26.5	66	12.5	39.8
DDOD	Uniform soup	28	64.6	14.6	40.7
	Greedy soup	29.3	65.8	16.3	41.5
	DLLO-DRF	30.1	67.2	16.6	42.4
GFL	Uniform soup	17.3	39.9	10.8	35.4
	Greedy soup	17.8	41.6	10.5	36
	DLLO-DRF	19.3	46.9	11.6	36.6
Cascade R-CNN	Uniform soup	28.9	67.9	11.9	36.6
	Greedy soup	29.3	68.2	13.1	37
	DLLO-DRF	30.2	70.1	14.4	37.6
Faster R-CNN	Uniform soup	27.9	71.3	12.3	36
	Greedy soup	28.6	71.9	11.9	36.9
	DLLO-DRF	31.1	73	14.2	38.1
Dynamic R-CNN	Uniform soup	26.6	66.5	12.1	36.8
	Greedy soup	26.9	66.5	10.7	36.9
	DLLO-DRF	27.7	68.8	11.5	36.5
Grid R-CNN	Uniform soup	28.8	64.5	17.5	36.4
	Greedy soup	28.5	64.5	14.2	37.6
	DLLO-DRF	29.8	65.2	23.3	38.4
Libra R-CNN	Uniform soup	25.7	62.3	11.5	35.4
	Greedy soup	26.7	63.8	11.7	36.6
	DLLO-DRF	26.6	62.9	13.2	37.2
Sparse R-CNN	Uniform soup	25.1	67	9.5	39.7
	Greedy soup	24.1	66.1	6.9	38.9
	DLLO-DRF	25.4	65.2	11.5	40.9

, DLLO-DRF ranks first for three or more metrics among seven detection algorithms. For the $AP$ metric, DLLO-DRF ranks first among eight detection algorithms. For the $A{P}_{50}$ metric, DLLO-DRF ranks first among seven detection algorithms. For the $A{P}_{75}$ metric, DLLO-DRF ranks first among eight detection algorithms. For the $AR$ metric, DLLO-DRF ranks first among eight detection algorithms. In contrast, for Uniform soup, there is only one algorithm ranking first for the $A{P}_{50}$ and $A{P}_{75}$ metrics, and for Greedy soup, there is only one algorithm ranking first for the $AP$, $A{P}_{50}$, and $AR$ metrics.

4.3.2. Comparison between DLLO-DRF and Other Algorithms in the Cement Pole Damage Dataset

As shown in

Table 12. Comparison of DLLO-DRF and other fusion methods in the cement pole damage dataset. The better results are indicated using bold and underline.

Object Algorithm	Fusion Method	$\mathit{AP}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	$\mathit{AR}$
ATSS	Uniform soup	13.4	37.5	9.2	28.3
	Greedy soup	13.3	37.2	9.2	28.2
	DLLO-DRF	13.4	37.6	9.3	28.2
DDOD	Uniform soup	15.2	45.9	12.3	20.4
	Greedy soup	15.3	46.5	12.3	20.8
	DLLO-DRF	15.6	47.2	12.2	21.2
GFL	Uniform soup	21.4	42.2	21.7	29.2
	Greedy soup	21.4	42.5	22.1	28.9
	DLLO-DRF	21.9	44.8	18.7	30.7
Cascade R-CNN	Uniform soup	17.5	34.9	16.5	21.8
	Greedy soup	17.2	34.7	16	21.7
	DLLO-DRF	16.9	35.4	15.2	21.8
Faster R-CNN	Uniform soup	15.8	30.4	15	21.6
	Greedy soup	16.2	31.9	14.8	22.3
	DLLO-DRF	17	33.5	16.5	22.7
Dynamic R-CNN	Uniform soup	12.8	25.4	9.9	19.3
	Greedy soup	12.4	25.1	9.4	18.4
	DLLO-DRF	13.4	26	10.5	20.1
Grid R-CNN	Uniform soup	15.5	36	10.5	22.2
	Greedy soup	15.7	36	10.4	22.4
	DLLO-DRF	16	37.8	9.1	22.6
Libra R-CNN	Uniform soup	18.7	45.2	14	23.7
	Greedy soup	18.1	45.4	13.5	23.6
	DLLO-DRF	18.2	45.3	13.3	23.9
Sparse R-CNN	Uniform soup	17.5	46.7	5.5	25.3
	Greedy soup	17.2	45.8	3.7	24.8
	DLLO-DRF	18.3	47.7	5	26

, DLLO-DRF ranks first for three or more metrics among seven detection algorithms. For the $AP$ metric, DLLO-DRF ranks first among eight detection algorithms. For the $A{P}_{50}$ metric, DLLO-DRF ranks first among eight detection algorithms. For the $A{P}_{75}$ metric, DLLO-DRF ranks first among three detection algorithms. For the $AR$ metric, DLLO-DRF ranks first among eight detection algorithms. In contrast, for Uniform soup, there are two, five, and two algorithms ranking first for the $AP$, $A{P}_{75}$, and $AR$ metrics, respectively, and for Greedy soup, there are one and two algorithms ranking first for the $AP$, $A{P}_{50}$, and $AR$ metrics, respectively.

4.3.3. Comparison between DLLO-DRF and Other Algorithms in the Shockproof Hammer Slip Dataset

As shown in

Table 13. Comparison of DLLO-DRF and other fusion methods in the shockproof hammer slip dataset. The better results are indicated using bold and underline.

Object Algorithm	Fusion Method	$\mathit{AP}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	$\mathit{AR}$
ATSS	Uniform soup	27.9	63.8	18.3	37.6
	Greedy soup	27.8	63.9	17.4	37.8
	DLLO-DRF	28.2	64	19.4	38
DDOD	Uniform soup	31.8	64.1	29.7	39.6
	Greedy soup	32.3	65.8	30.1	40.1
	DLLO-DRF	33.2	66.8	31.7	40.8
GFL	Uniform soup	29	63.9	19.4	40.6
	Greedy soup	29.1	63.8	19.2	40.9
	DLLO-DRF	29.8	64.2	20.1	41.1
Cascade R-CNN	Uniform soup	39.4	61.1	48.1	44.3
	Greedy soup	39.9	61.8	48.6	44.8
	DLLO-DRF	39.6	62	48.3	44.4
Faster R-CNN	Uniform soup	40.5	66.5	46.1	45.8
	Greedy soup	40.5	66.4	46.8	45.7
	DLLO-DRF	40.9	67.3	46.5	46.7
Dynamic R-CNN	Uniform soup	38.5	60.2	43.6	46
	Greedy soup	38.8	60.4	43.7	46.1
	DLLO-DRF	38	60.6	42.6	46
Grid R-CNN	Uniform soup	40.3	64.4	44.7	46.9
	Greedy soup	40.7	64.4	45.6	47.4
	DLLO-DRF	40.2	64.4	43.6	47
Libra R-CNN	Uniform soup	35.3	60	37.7	46.2
	Greedy soup	35.3	60.4	37.7	46.2
	DLLO-DRF	35.9	61.1	38.2	46.5
Sparse R-CNN	Uniform soup	17.4	38.7	13.8	42
	Greedy soup	17.6	38.9	14	42.1
	DLLO-DRF	18.3	40.3	14.6	43.5

, DLLO-DRF ranks first for three or more metrics among six detection algorithms. For the $AP$ metric, DLLO-DRF ranks first among six detection algorithms. For the $A{P}_{50}$ metric, DLLO-DRF ranks first among nine detection algorithms. For the $A{P}_{75}$ metric, DLLO-DRF ranks first among five detection algorithms. For the $AR$ metric, DLLO-DRF ranks first among six detection algorithms. In contrast, for Uniform soup, there is only one algorithm ranking first for the $A{P}_{50}$ metric, and for Greedy soup, there are three, one, four, and three algorithms ranking first for the $AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$ metrics, respectively.

4.3.4. Comparison between DLLO-DRF and Other Algorithms in the Insulator Self-Explosion Dataset

As shown in

Table 14. Comparison of DLLO-DRF and other fusion methods in the insulator self-explosion dataset. The better results are indicated using bold and underline.

Object Algorithm	Fusion Method	$\mathit{AP}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	$\mathit{AR}$
ATSS	Uniform soup	12.5	41.7	3.2	34.5
	Greedy soup	12.6	41.5	3.2	34.6
	DLLO-DRF	13.1	41.8	3.6	34.8
DDOD	Uniform soup	19.8	55	8	39.1
	Greedy soup	19.8	55	8.5	38.8
	DLLO-DRF	19.6	53.9	8.3	39.5
GFL	Uniform soup	16.6	48.6	6.8	38.5
	Greedy soup	16.7	48.4	6.8	38.2
	DLLO-DRF	16.3	46	5.5	37.8
Cascade R-CNN	Uniform soup	29.1	66.1	21.9	42.5
	Greedy soup	28.8	65.3	22.5	42.5
	DLLO-DRF	29.5	67.3	19.9	43.7
Faster R-CNN	Uniform soup	14.5	41.3	8.6	30.4
	Greedy soup	15	42.9	9	31.3
	DLLO-DRF	16.1	45.3	7.9	31.6
Dynamic R-CNN	Uniform soup	22.4	59.9	13.3	36.4
	Greedy soup	21.6	59.9	9.6	35.3
	DLLO-DRF	23.6	61.2	13.4	37.2
Grid R-CNN	Uniform soup	26.8	59.1	20.9	46.7
	Greedy soup	27	60.1	19.1	46.2
	DLLO-DRF	27.6	60.9	19.7	46
Libra R-CNN	Uniform soup	20.2	50	13.4	35.5
	Greedy soup	20.9	49.8	15.1	35.8
	DLLO-DRF	21.1	52.6	14.2	37.2
Sparse R-CNN	Uniform soup	20	48.9	12.2	36.7
	Greedy soup	20.1	48.2	13.3	36.2
	DLLO-DRF	20.8	49.4	15.3	37.5

, DLLO-DRF ranks first for three or more metrics among six detection algorithms. For the $AP$ metric, DLLO-DRF ranks first among seven detection algorithms. For the $A{P}_{50}$ metric, DLLO-DRF ranks first among seven detection algorithms. For the $A{P}_{75}$ metric, DLLO-DRF ranks first among three detection algorithms. For the $AR$ metric, DLLO-DRF ranks first among seven detection algorithms. In contrast, for Uniform soup, there are one, two, three, and two algorithms ranking first for the $AP$, $A{P}_{50}$, $A{P}_{75}$, and $AR$ metrics, respectively, and for Greedy soup, there are one, one, and four algorithms ranking first for the $AP$, $A{P}_{50}$, and $A{P}_{75}$ metrics, respectively.

In conclusion, DLLO-DRF outperforms Uniform soup and Greedy soup on all four datasets, and Greedy soup performs better than Uniform soup. The model faces significant parameter update deviations when encountering new data at each training stage under the training conditions based on fine-tuning lifelong learning. DLLO-DRF strikes a balance between fundamental parameters to identify and compute parameters that are appropriate for the entire lifelong learning process, while Uniform soup and Greedy soup integrate all basic parameters, resulting in a weaker expression ability of the integrated model for all data in the entire lifelong learning process. As Greedy soup aims to identify the best metrics, and uses a greedy approach to identify suitable subsets of basic models for fusion, it is more suitable for lifelong learning as a dynamic training method compared to Uniform soup.

4.4. Friedman Test Results

To investigate the differences between the three abovementioned algorithms, the Friedman test method was used to verify the experiment in Section 4.3. Here, the $AP$ metric is selected for testing all datasets and basic object detection algorithms. The experiment involved three algorithms and used four datasets, and ${\tau }_F$ followed the F distribution with degrees of freedom of 2 and 6. According to the $F(2,6)$ distribution, the $\rho$ values for ATSS, DDOD, GFL, Cascade R-CNN, Fast R-CNN, Dynamic R-CNN, Grid R-CNN, Libra R-CNN, and Sparse R-CNN are 0.0293, 0.0293, 0.0293, 0.0421, 0.0293, 0.0293, 0.0293, 0.0293, and 0.0388, respectively, with significance less than 0.05. As a result, we must reject the initial assumption that there is no significant difference in the performance of each algorithm. Therefore, there are significant differences in the performance of the algorithms participating in the experiment. Based on the conclusion of the experiment in Section 4.3, it is concluded that DLLO-DRF has the best performance compared to the comparative algorithms.

5. Conclusions

This paper presents DLLO-DRF, a deep lifelong learning optimization algorithm based on dense region fusion. In this algorithm, the parameters at each stage in lifelong learning are partitioned into regions based on their values. DLLO-DRF identifies dense regions through parameter dispersion analysis, and optimizes the model by averaging and fusing the parameters. In Section 4, extensive experiments on the self-labeled transmission line defect dataset and various deep object detection algorithms demonstrate that the proposed DLLO-DRF effectively improves the performance of deep neural network models under lifelong learning. This significantly contributes to the development of deep lifelong learning by offering an optimization method that simultaneously allows for the learning of new data without using old data, and without increasing the model parameters.

Additionally, as deep lifelong learning is a rapidly growing and evolving field, there are many areas in which DLLO-DRF can be further improved and developed. Further research is required to evaluate the effectiveness of DLLO-DRF on diverse and large-scale datasets. It is also important to explore ways of adapting DLLO-DRF to optimize complex, changing deep neural network models, and make them scalable for large-scale lifelong learning scenarios.

Overall, the proposed DLLO-DRF offers a promising approach for optimizing lifelong learning in deep neural networks, and opens up new avenues for future research and development.