Object Detection, Segmentation and Categorization in Artificial Intelligence

In the field of computer vision, three basic tasks are particularly important: object detection [1], segmentation [2] and categorization [3]. These tasks form the basis for understanding and interpreting visual data and are widely used in key areas such as autonomous driving, medical imaging, and surveillance. They not only promote the progress of computer vision technology, but also provide solid technical support for various application scenarios, enabling machines to interpret and analyze visual information more accurately. The continuous development of these technologies improves the performance and efficiency of the machine systems and promotes their widespread implementation in all walks of life [4].

Traditional object detection, segmentation and categorization algorithms have been instrumental to the early development of computer vision [5,6,7]. However, with the rise of deep learning methods, the shortcomings of traditional methods have become clear. Most traditional methods require a great deal of domain knowledge to design and select features, and the generalization ability of these features is limited, making it difficult to apply these methods to large-scale datasets.

Recently, artificial intelligence and deep learning have flourished, and a large number of outstanding artificial intelligence algorithms have emerged for use in object detection, segmentation, and categorization tasks [8,9,10,11,12,13,14]. Compared with traditional methods, these algorithms usually have significant advantages such as automatic feature learning, end-to-end training, high precision and robustness, ability to process massive data, flexibility to adapt to multiple tasks and efficient feature expression.

Moreover, artificial intelligence algorithms often achieve excellent real-time performance, especially in applications such as real-time monitoring and autonomous driving [15]. The fast reasoning ability of artificial intelligence methods enables the system to identify and respond to changes and events in the environment, such as traffic signs, pedestrians and obstacles, in a timely manner, ensuring the safety and stability of the system.

Indoor scenes are important urban spaces, and logos are very important for the accurate operation of mobile robots in indoor environment. Therefore, in Contribution 1, a logos detection method named MobileNetV2-YOLOv4-UP is proposed. This method combines unsupervised learning and few-shot learning to obtain a potential feature representation of the logo by pre-training the autoencoder on a public unlabeled logo dataset, and then incorporating a small number of labeled indoor scene logo datasets into the training to update the weight of the logos detection network. The related research on logo detection, few-shot learning and unsupervised learning is also reviewed in this paper.

In recent years, multi-objective and multi-task evolutionary algorithms have attracted a great deal of attention [16]. Evolutionary algorithms can tackle complex nonlinear optimization problems and find the global optimal solution; they can also optimize multiple targets at the same time and find a solution to balance each target through the Pareto optimal solution set. In light of this, a multi-objective automatic clustering algorithm based on evolutionary multi-task optimization is introduced (Contribution 2). It introduces multi-task learning in multi-objective optimization, realizes knowledge sharing among different clustering tasks, and improves the performance and efficiency of clustering.

A novel method called causal semantic representation learning with domain intervention (CSRDN) is proposed (Contribution 3). Based on the data generation process in the natural environment, it uses the structural causal model (SCM) to construct an inclusive causal directed acyclic graph (DAG) in order to simulate domain generalization tasks. Proxy domain variables are introduced into the causality diagram to explain the limitations of generalization performance and simulate domain shift. Meanwhile, the domain change is regarded as an intervention, the confounding effect is eliminated by controlling the proxy domain variables, and the unstable bias information flow is blocked. Finally, counterfactual reasoning is used to generate counterfactual representations that are consistent with the original input semantics but are influenced by different domains to improve the generalization ability of deep learning models in unknown environments.

Unlike natural images, hyperspectral images (HSIs) typically contain tens to hundreds of bands. Each band corresponds to a specific wavelength range that captures the detailed spectral properties of an object. Endmember extraction plays an important role in HSI analysis by extracting representative spectra to facilitate more accuracy when performing feature classification, mineral identification, agricultural monitoring, environmental monitoring, and urban planning tasks. An endmember extraction method based on evolutionary competitive multi-task optimization is proposed (Contribution 4), which considers the problem of endmember extraction with different numbers of endmembers as a set of optimization tasks and assumes that there is a competitive relationship between these tasks. Online resource allocation is then used to allocate appropriate computational resources to these tasks. Finally, experiments are conducted on simulated and real hyperspectral datasets to verify the effectiveness of the proposed method.

A composite infrared dim and small target detection model, USES-Net, is proposed (Contribution 5). Based on a typical encoder–decoder structure, it introduces the self-attention mechanism of Swin Transformer to extract potential features related to dim and small targets with a larger sensory field, and designs a bottom-up cross-layer feature fusion module (AFM), which can reconstruct the acquired feature information on different scales of the target feature information. The model effectively improves the detection accuracy.

In the field of rotated object detection, existing regression- and classification-based methods suffer from multiple-solution issues and boundary issues in predicting rotation angles, which limit the upper bound of detector performance. To address these issues, an improved rotation object detection method is introduced (Contribution 6), which proposes a method based on the circular Gaussian distribution to predict angles. This method converts the marked angles into a discrete circular Gaussian distribution, covering a minimum positive period, allowing the model to predict distribution parameters instead of directly regressing or classifying angles. The experimental results on multiple public datasets show that the proposed method is effective and achieves a superior performance.

In order to improve the performance of underwater object detection, an improved model YOLOv5-CFDSDSE based on YOLOv5s is proposed (Contribution 7). This model combines C3 and FasterNet structures to improve model accuracy, reduce the number of parameters, and improve scale perception, spatial perception, and task perception, enhancing the model’s detection ability for multi scale and multi class objects. In addition, the small object detection (SD) layer has been optimized to effectively combine the feature information of different scales, retain more detailed information, and enhance the detection ability of small objects. Numerous experiments have shown that YOLOv5-CFDSDSE achieves an excellent performance in underwater object detection.

A database specially designed for autonomous vehicle was introduced (Contribution 8). Based on the KITTI visual benchmark suite, the proposed database contains 7481 scenes, each of which has a corresponding top-view image, label file, and segmentation map. These segmentation maps are designed specifically for environmental perception tasks of autonomous vehicles, including road, background, car, truck, and truck segmentation. Detailed methods for creating databases were introduced, including explanations of how to convert 3D point clouds into 2D top-view images, how to generate detection labels on a plane, and how to implement neural networks to generate segmentation maps for each scene. The road segmentation performance indicators in the database include an F1 score of 95.77, average accuracy (AP) of 92.54, accuracy (ACC) of 97.53, accuracy (PRE) of 94.34, and recall rate (REC) of 97.25. These indicators demonstrate the superior performance of the database in road segmentation tasks. This study also included a review of research related to top-view segmentation of autonomous vehicles, including semantic segmentation using LiDAR sensor data, as well as existing research databases on autonomous vehicles.

In order to accurately detect and identify different types of aircraft in high-resolution remote sensing images, a novel algorithm Fine-grained S²ANet (FS²ANet) is proposed (Contribution 9). It is based on the improved Single-shot Alignment Network (S²ANet) and uses data augmentation techniques to address the imbalanced number of aircraft instances in the dataset. This model uses ResNet101 as the backbone network and combines the space-to-depth (SPD) module to improve the feature pyramid network (FPN) structure, constructing the FPN-SPD module. An aircraft fine feature focusing module (AF³M) was constructed at the detection head of the network to reduce the loss of fine-grained information during feature extraction and improve the network’s ability to extract small aircraft features. Finally, the challenges that need to be addressed in future work were pointed out, including applications in different remote sensing scenarios, as well as designing algorithms with higher-resolution inputs and faster speeds.

A frame segmentation method based on neural network training is proposed for detecting abnormal eye-blinking events in female drivers in Saudi Arabia (Contribution 10). This method segments the eye area through variational pixel extraction and identifies different frames using texture changes. The model adopts a two-layer convolutional neural network. The first layer recognizes the differences between continuous and discrete frames, while the second layer is responsible for segmenting the eye area based on texture changes. Finally, experimental analysis was conducted using the Niqab dataset to verify that the proposed method improves the accuracy of eye-blink detection and reduces detection time and errors.

To address the challenge of small sample underwater noise data, a typical underwater noise generation model (UNGM) was established (Contribution 11) and increased noise samples by generating noise with specific kurtosis, simulating the spectral and statistical characteristics of actual noise. In addition, underwater noise classification models based on UNGM and convolutional neural networks (CNN) have been developed. UNGM-CNN effectively solves the problem of small sample underwater noise classification and demonstrates the potential application of combining physical principles and machine learning techniques in the field of marine acoustics.

A multi-channel multi-location attention mechanism is proposed for a multi-modal phased transfer side-scan sonar target recognition model (Contribution 12). At the model level, different stages of the backbone network are trained using SAR and ImageNet datasets to improve the adaptability to side scan sonar data. Additionally, an attention mechanism network (AMN) and recognition network are combined, using side-scan sonar data to learn network parameters and obtaining important features of the target through different channel attention factors at different locations. The experiment shows that the proposed model has better accuracy and robustness in side-scan sonar target recognition.