Robot Grasp Detection with Loss-Guided Collaborative Attention Mechanism and Multi-Scale Feature Fusion

Abstract

Grasp detection serves as the fundamental element for achieving successful grasping in robotic systems. The encoder–decoder structure has become widely adopted as the foundational architecture for grasp detection networks due to its inherent advantages of speed and accuracy. However, traditional network structures fail to effectively extract the essential features required for accurate grasping poses and neglect to eliminate the checkerboard artifacts caused by inversion convolution during decoding. Aiming at overcoming these challenges, we propose a novel generative grasp detection network (LGAR-Net2). A transposed convolution layer is employed to replace the bilinear interpolation layer in the decoder to remove the issue of uneven overlapping and consequently eliminate checkerboard artifacts. In addition, a loss-guided collaborative attention block (LGCA), which combines attention blocks with spatial pyramid blocks to enhance the attention to important regions of the image, is constructed to enhance the accuracy of information extraction. Validated on the Cornell public dataset using RGB images as the input, LGAR-Net2 achieves an accuracy of 97.7%, an improvement of 1.1% over the baseline network, and processes a single RGB image in just 15 ms.

1. Introduction

With the widespread implementation of robots in various fields, grasping technology, which plays a crucial role in controlling robots, has gained significant prominence. Currently, the majority of grasp detection methods still heavily rely on established object models and contextual information, rendering it inadequate for tackling the challenging task of recognizing complex unknown objects. Deep learning-based methods [1] have made significant progress, along with the continuous development of deep learning technology, in the field of grasp detection in recent years. Jiang et al. [2] first proposed a grasp representation based on 2D-oriented grasp boxes, optimizing the expression of grasp poses and enabling direct grasp detection through object detection. Since then, researchers have focused on improving the accuracy and efficiency of grasp detection. Lenz et al. [3] employed a two-stage cascaded network for grasp pose detection, achieving an accuracy of 73.9% on the Cornell grasp dataset. Due to the extensive number of network parameters, real-time performance was compromised, and generalization capabilities were limited. Redmon et al. [4] trained an end-to-end network using deeper networks with input RGD (red, green, depth) data to obtain grasp detection results directly, achieving 88.0% accuracy on the Cornell dataset. Morrison et al. [5] proposed a generative grasp convolutional neural network that utilizes depth images as input and directly outputs the precise grasp location, angle, and jaw width, thereby enabling real-time object grasping by robotic arms. Kumra et al. [6] proposed GRCNN, a convolutional neural network capable of processing multi-modal inputs such as RGB, RGD, and RGB-D data. By employing regression techniques for grasp pose prediction, the model achieved an impressive accuracy rate of 96.6% on the Cornell grasp dataset.

To enhance real-time performance and accuracy in the grasp detection process, this paper proposes LGAR-Net2, a grasp network based on a loss-guided collaborative attention mechanism. RGB input is utilized to generate grasp detection results with identical dimensions as the input image, achieving pixel-level detection precision. In the coding stage, multi-layer step convolution is employed for deep semantic extraction preparation. The semantic extraction stage incorporates a convolution structure that combines residual architecture and a loss-guided collaborative attention mechanism to enhance network robustness in detecting irregular objects. In the decoding stage, a bilinear interpolation layer replaces the inverse transpose convolution layer to ensure smoother pixel representation.

The contributions of this paper are summarized as follows:

• We propose a novel grasp detection network that integrates a loss-guided collaborative attention mechanism with multi-scale feature fusion to enhance the network’s focus on object grasp regions.
• In order to address the checkerboard artifacts in the upper sampling layer and improve the grasping accuracy, an effective convolutional decoding block is proposed.
• The grasping network LGAR-Net2 built with the proposed module achieved 97.7% performance in both open datasets and simulated robot grasping. And an ROS simulation environment is established for the real-time prediction of grasping poses using LGAR-Net2, achieving a processing speed of 15 ms per piece with an accuracy rate of 93.3%.

2. Related Works

2.1. Grasp Detection Using Deep Learning

The empirical method and analytical method are commonly employed for grasping detection. The analytical approach relies on a known object model and utilizes precise mathematical modeling to calculate the grasping pose [7]. While it boasts a rich theoretical foundation, the real scene’s grasping object model and the mathematical model between the manipulator and the object are more intricate, leading to extensive calculations. On the other hand, the empirical method determines the grasping pose based on the approximate position and shape of the object, thus circumventing the complex modeling process. Method [8], which is based on target recognition, initially identifies the region of interest (ROI) and subsequently determines the grasping pose using a regression approach. Lenz et al. [3] employed a neural network as a classifier for detecting rectangles, achieving a detection accuracy of 75.6% on the Cornell dataset, thereby enhancing the precision of the grasping pose estimation. Pinto et al. [9] utilized convolutional neural networks (CNNs) to accurately predict the grasping posture of objects, leading to a significant improvement in the network’s overall generalization performance.

2.2. Encoder–Decoder Regression

The encoder–decoder architecture utilizes the encoder module to extract feature and the decoder module to reconstruct the learned feature information into the output of the original image size. This design is employed in U-Net [10], Seg-Net [11], and other networks to achieve automatic and efficient pixel-level output. Mahajan et al. [12] proposed a vector quantization variational autoencoder architecture for feature extraction in a semi-supervised learning setting, thereby improving the generalization capability of GG-CNN. Additionally, Yu et al. [13] utilized separable convolution for speed to replace traditional transpose convolution in order to reduce the model parameters and employed target segmentation technology to effectively distinguish the target from the background, resulting in higher accuracy. However, the checkerboard artifacts remain challenging, leading to feature map distortion in the decoder and impacting grasp prediction performance. In contrast to the above approaches, this study presents a deconvolution decoder based on bilinear difference to address the checkerboard artifacts issue.

2.3. Attention Mechanisms for Grasp Detection

The attention mechanism [14] has shown strong performance in natural language processing (NLP) tasks and, subsequently, researchers have applied this framework to grasping pose detection. Recent research on grasp detection [15] has demonstrated that incorporating an attention mechanism into the network can effectively enhance the network’s focus on grasp regions. In their work, Hu et al. [16] integrated a channel attention mechanism with a residual structure to establish inter-channel connections by encoding the weight of each channel. However, they did not consider encoding spatial feature information. Building on the SE attention, research [17] introduces the CBAM attention, which comprehensively considers information in both spatial and channel dimensions and utilizes the self-attention mechanism in the CoA module [18] to enhance the feature information recognition ability in the spatial dimension. The evaluation score of this region is obtained by summing the correlation between each region in the feature image and the adjacent region and employing the one-dimensional convolution operation to determine the weight of the space and the channel. AMSC-Net [19] effectively improves the network’s attention to the grasping area by fusing the attention mechanism of two dimensions: space and channel. Nonetheless, the attention mechanism of AMSC-Net is achieved through simple convolution, which does not fully exploit contextual information. DSC-Grasp-Net introduces a novel collaborative attention (CA) mechanism [20] which converts the feature tensor into a single feature vector through 2D global pooling to enhance the relevant features.

In contrast to the aforementioned, this article proposes an attention mechanism that employs a loss-guided approach. By utilizing a loss function to adjust attention, the recognition capability of the regions of interest is enhanced. It modifies feature weights in multiple dimensions to improve focus on crucial feature information.

3. Problem Statement

3.1. Grasp Pose Definition

During the process of grasp pose generation, precise grasping can be achieved by establishing a mathematical model that encompasses the robot arm, object, and camera. Typically, the grasp pose in the camera coordinate system is represented as a five-dimensional grasp representation [6], which can be visualized as depicted in Figure 1b.

The grasp pose of a pixel point in the camera coordinate system is initially defined as follows:

$$g=(p,\theta ,w,q)$$

(1)

The grasping point is denoted by coordinate $p=(u,v)$ in the plane, while angle $\theta$ represents the orientation of the width edge of the grasping box with respect to the horizontal X-axis. The range of angle $\theta$ is defined as $\left[-\frac{1}{2}\pi ,\frac{1}{2}\pi \right]$. $w$ denotes the gripper’s opening and closing width; $q$ quantifies the grasping quality of a point on a scale ranging from 0 to 1.

The grasping pose set of each pixel in the image is defined as follows:

$${\widehat{G}}_i=({\widehat{P}}_i,{\widehat{\theta }}_i,{\widehat{W}}_i,{\widehat{Q}}_i)$$

(2)

The grasping posture is defined within the world coordinate system as follows:

$${{\displaystyle G}}_i=({{\displaystyle P}}_i,{\theta }_i,{{\displaystyle W}}_i,{{\displaystyle Q}}_i)$$

(3)

${{\displaystyle P}}_i=({{\displaystyle X}}_i,{{\displaystyle Y}}_i,{{\displaystyle Z}}_i)$ represents the point in relation to the world coordinate system, where its elevation remains constant relative to the spatial relationship between the camera and the object, thereby ensuring a fixed value for ${{\displaystyle Z}}_i$ within the global coordinate system.

Finally, the transformation relationship $T_{camera}^{base}$ between the robotic arm base and the camera is determined by employing DH modeling [21] for coordinate system conversion.

The grasping pose in the camera coordinate system is transformed into the grasping pose in the world coordinate system, which can be achieved through Equation (4):

$${{\displaystyle G}}_i={{\displaystyle T}}_{camera}^{base}\ast ({{\displaystyle \widehat{G}}}_i)$$

(4)

The grasp detection network utilizes camera image $I\in {\mathbb{R}}^{n\times h\times w}$ as an input to compute the set of grasp poses ${{\displaystyle \widehat{G}}}_i$ for each pixel point in the camera coordinate system, followed by determining the optimal grasp pose using formula ${{\displaystyle \widehat{G}}}^{*}={{\displaystyle \max }}_{\widehat{Q}}\widehat{G}$ based on the highest grasp quality. Consequently, this paper focuses on investigating the set of grasp poses ${{\displaystyle \widehat{G}}}_i$ for every pixel point in the camera coordinate system.

3.2. The Chessboard Artifacts Problem Description

The output feature map of the transposed convolution process in a detection network with a decoding structure exhibits distinct grid-like patterns due to the “uneven overlap” resulting from the deconvolution process, as illustrated in Figure 2.

The chessboard artifacts can significantly impact the quality and accuracy of the generated images, leading to a reduction in image clarity and realism. This artifact can impede the visual perception of the image, potentially resulting in failure to recognize it. Therefore, effectively mitigating or eliminating the chessboard artifacts is crucial for ensuring the high quality of the generated images.

4. Method

This section presents a comprehensive description of LGAR-Net2, a proposed deep learning network for pixel-level grasp detection. The input to LGAR-Net2 is an RGB image, and the output consists of four pairs of grasp pose images with the same dimensions, where each pixel in the output corresponds to a pixel in the input image. Figure 3 illustrates the architecture of LGAR-Net2, which comprises three main modules: encoder, bottleneck, and decoder. The encoder module primarily incorporates multiple convolutional layers to decompose the multi-channel input image and extract relevant features. The bottleneck module leverages the fusion of the residual blocks and LGA2 blocks to capture multi-dimensional signals at different scales. Finally, the decoder module employs parallel standard convolutional processing to generate grasp predictions.

4.1. Encoder

In order to extract additional feature information, this paper uses the feature extraction module to extract the detailed features of input images through multi-layer convolution layers, as shown in Figure 4. Specifically, it includes one CBR Block*4 blocks, one CBR Block*2 blocks, and two average pooling layers (step length = 2). We set the convolution kernel size in the module to 3 × 3 so as to achieve stronger feature description capability. Through the feature extraction module, the input RGB data are converted into feature graph $F_e\in {{\displaystyle \mathbb{R}}}^{64\times h/2\times w/2}$.

The CBR Block*k can be characterized as follows:

$$T_{\mathrm{out}}=f_{avgpool}\left(f_{conv}^k(f_{conv}^{k-1}(\dots f_{conv}^1(T_{in})\dots ))\right)$$

(5)

The series structure of the 2D convolution layer, normalization layer, and RELU layer is denoted by $f_{conv}^k$, while $f_{avgpool}$ represents the sampling layer implemented through average pooling.

4.2. Bottleneck Part

At the bottleneck section, the network must extract a substantial amount of semantic information to provide the dependable feature images for the decoding phase. However, with an increasing number of network layers, the issue of gradient vanishing becomes more pronounced. Residual layers [16] offer an effective network design approach that circumvents the problem of gradient vanishing by backpropagating the output from the preceding layer while maintaining consistent input dimensions. Consequently, in this stage, we employ multiple residual modules to extract profound features from the image.

4.2.1. Residual Squeeze Excitation Block (RSE Block)

In this study, we propose a novel residual squeeze excitation network (RSE) with six locally connected skip connections to enhance the semantic features, as shown in Figure 5. The RSE architecture consists of two 2D convolutional layers followed by a squeeze block and an excitation block in series. The calculation formula for RSE is as follows:

$$\left\{\begin{array}{c}{{\displaystyle T}}_1={{\displaystyle f}}_{RSE}({{\displaystyle F}}_e)\\ {{\displaystyle T}}_2={{\displaystyle f}}_{RSE}({{\displaystyle T}}_1)\\ ⋮\\ {{\displaystyle T}}_n={{\displaystyle f}}_{RSE}({{\displaystyle T}}_{n-1})\end{array}\right.$$

(6)

The input feature image, ${{\displaystyle F}}_e$, is processed by n RSE blocks (T₁ − Tn) in this study. Specifically, six RSE blocks (n = 6) are employed to generate output feature images which are subsequently fused to yield multi-layer feature images with enhanced richness.

4.2.2. Loss-Guided Collaborative Attention Block (LGCA Block)

During the forward propagation of features, semantic features are enhanced while detailed information is gradually suppressed, thereby compromising the discriminative capability of the features. In this study, we propose a novel LGCA block, as shown in Figure 6, that combines the cross-entropy auxiliary loss function with the ground truth and deep features from the grasping region, self-attention mechanism, and MUS module. This module guides the network to focus on contextual information within graspable regions in order to enhance their discriminative ability. Compared to existing multi-branch and multi-scale context strategies based on hole convolution, which only considers spatial relationships between pixels, our LGCA block enhances each pixel’s contribution to grasp representation by considering associations between all the graspable object regions and individual pixel points. It also facilitates the selective learning of graspable regions while suppressing interference from non-graspable regions, thus improving the generalization capabilities of grasping detection algorithms.

The LGCA block consists of a loss-guided layer, a collaborative attention block based on local jump, and a multi-scale module. Each of these components will be individually introduced in the following sections.

1.

Loss guide block

During the process of network training, the loss-guided layer will be up-sampled into $T_6\in {{\displaystyle \mathbb{R}}}^{128\times h/4\times w/4}$ to $F_l\in {{\displaystyle \mathbb{R}}}^{1\times h\times w}$ with the same initial input size through the 1 × 1 convolution layer and the up-sampling layer based on the bilinear interpolation method. Then, the difference between $F_l$ and the standard grab pose pixels generated by the data label is taken as a loss. The loss is added to the total loss to guide the network to notice the best pose of the object.

2.

Collaborative attention block

It is crucial to accurately understand the object feature information in complex backgrounds, but traditional convolutional calculations with sliding operations can only establish connections between adjacent pixels, while they cannot establish effective connections between pixels in different channels or positions. Moreover, the semantic information in the deep layers of the network is expressed in different channels, but traditional 2D convolution methods only operate within a single channel, without establishing connections between channels. Therefore, in space, we establish spatial connections between pixels by means of cutting, transposition, and assembly, and in different channels, we establish connections between channels by performing average pooling down-sampling on each channel and then concatenating the results of the convolution. By adding channel-wise attention blocks, we can more accurately capture the semantic features of objects, as shown in the structure of the collaborative attention block in Figure 7.

The collaborative attention block comprises a spatial-correlation attention block and a channel-correlation attention block. The input of the channel-correlation attention module is T6, and its output has an identical shape. Initially, the vector Sc is obtained by average pooling down-sampling for each channel, and then the 1 × 3 convolution kernel (step size is 1) is employed for the one-dimensional convolution of Sc, and the feature information of the adjacent channels is convolved and fused. The magnitude of the vector Sc is then recovered using a deconvolution operation, and finally the weights are mapped between [0,1] using a Sigmoid function. The channel association attention block makes full use of the feature relationship between adjacent channels, allocates the weights between channels, and strengthens the connection between channels. Its structure can be expressed as follows:

$$\left\{\begin{array}{l}{{\displaystyle S}}_c=\frac{1}{H\times W}•{\displaystyle \sum _{h=1}^H{\displaystyle \sum _{w=1}^W{F}_{c,h,w}}}\\ S=\{{{\displaystyle S}}_1,{{\displaystyle S}}_2,\dots {{\displaystyle S}}_c\}\\ \widehat{S}=Sig(DConv(Conv(S)))\\ {{\displaystyle F}}_s=Fc,h,w•\widehat{S}c,1,1\end{array}\right.$$

(7)

where ${{\displaystyle S}}_c$ is the average pooling value of the C channel, and $F_{c,h,w}$ is a pixel in the image.

The spatial attention module is depicted in Figure 7. Similarly, by taking T6 as the input, the feature images of each channel are flattened into a vector and concatenated to obtain the vector E∈R^C×HW. Subsequently, E is multiplied with its transpose and passed through the softmax layer to derive the output vector S according to the following formula:

$$E=reshape(F)$$

(8)

$$S_{ij}=\frac{exp{(E^T\ast E)}_{ij}}{{\displaystyle {\sum }_{j=1}^{HW}exp{(E^T\ast E)}_{ij}}}$$

(9)

where $S\in R^{HW\times HW}$ and * are matrix multiplication.

3.

Multi-Scale Module

As a powerful technique, Atrous convolutions offer the ability to precisely manage the resolution of features calculated by deep convolutional neural networks and allow for fine-tuning of the filter’s viewpoint to capture information at multiple scales. In this study, we establish a connection between the deep layer of the network and the decoding module using the MUS module [22], where we input the feature map FA extracted by the attention module into it. Within the MuS module, we parallelize void convolutions with different expansion rates (6/12/18) to generate feature maps, which are subsequently restored to match the size of input feature maps through 2D convolution layers.

4.3. Feature Fusion Module

Using T₁~T₇ as the input, the feature image ${{\displaystyle T}}_{1~7}\in {{\displaystyle \mathbb{R}}}^{128\ast 7\times h/4\times w/4}$ is generated by channel splicing technology. Finally, 128 convolution kernels are used to restore the feature image to the same size as the input $F_s=convcat(T_1,\dots T_n)$.

The pre-stage input is squeezed by six residual excitation blocks to generate six sets of feature images (${{\displaystyle T}}_{1~6}\in {{\displaystyle \mathbb{R}}}^{128\times h/4\times w/4}$). Then, it is sent to the loss-guided collaborative attention block to adjust the attention to generate a set of feature images (${{\displaystyle T}}_7\in {{\displaystyle \mathbb{R}}}^{128\times h/4\times w/4}$). Then, seven groups of feature images are fused by feature fusion blocks to generate ${{\displaystyle T}}_{1~7}\in {{\displaystyle \mathbb{R}}}^{128\ast 7\times h/4\times w/4}$. Finally, 128 convolution kernels are used to restore the feature image to the same size as the input.

4.4. Decoder

Despite the successful recovery of feature images in the decoding stage by traditional deconvolution layers, they often lead to checkerboard artifacts between adjacent pixels. This issue significantly impacts performance, particularly for pixel-level grasp prediction. To address the problem of checkerboard artifacts, conventional methods typically modify the stride of the convolution kernel to ensure divisibility between kernel size and stride. However, since convolution weights change during learning, this approach fails to fundamentally resolve the checkerboard artifacts problem. Considering that bilinear interpolation can minimize differences in pixel values between interpolated and original input pixels, we employed a combination of bilinear interpolation layers for feature recovery. This approach effectively enhances pixel smoothness and successfully resolves the checkerboard artifacts problem.

The BIC-Block [23] (as illustrated in Figure 8) consists of a bilinear interpolation layer, 2D convolution layer, and RELU layer. Its operation can be described as follows:

$$F_b={{\displaystyle f}}_{BIC}({{\displaystyle f}}_{BIC}(F_s))$$

(10)

4.5. Loss Function

The loss function of LGAR-Net2 comprises the auxiliary LGCA loss and losses for the regression of the graspable region’s confidence, angle, and grasp width.

(1)

LGCA auxiliary losses: The function $loo{s}_{\mathrm{s}}=SmoothL1({\widehat{y}}_i^g,y_i^g)$ is defined, where ${\widehat{y}}_i^g$ represents the predicted value image of the network’s grabber region and $y_i^g$ denotes the label for the grabber region.

(2)

Regression loss estimation for the confidence, angle, and width of a crawlable region: The definitions of $loo{s}_{\cos }$, $loo{s}_{\sin }$, $loo{s}_{\omega }$, and $loo{s}_{uv}$ are equivalent to $SmoothL1\cos ,\sin ,uv({\widehat{y}}_i^g,y_i^g)$, where $y_i^g\in R^{H\times W}$ represents the ground truth image of the graspable region mapping for the i-th sample, while ${\widehat{y}}_i^g\in R^{H\times W}$ denotes the predicted value image of the graspable region for the i-th sample.

The definition of the total network loss is as follows:

$$loos={\lambda }_1(\frac{1}{n}{\displaystyle \sum _{i=1}^{n}loo{s}_{uv}})+{\lambda }_2(\frac{1}{n}{\displaystyle \sum _{i=1}^{n}loo{s}_{\cos }}+\frac{1}{n}{\displaystyle \sum _{i=1}^{n}loo{s}_{\sin }})+{\lambda }_3(\frac{1}{n}{\displaystyle \sum _{i=1}^{n}loo{s}_{\omega }})+{\lambda }_4(\frac{1}{n}{\displaystyle \sum _{i=1}^{n}loo{s}_{\mathrm{s}}})$$

(11)

where n represents the number of training samples, and, respectively, denotes the true value and estimated value for both the confidence in identifying a graspable region and the attitude information.

The definition of the Smooth L1 loss is as follows:

$${{\displaystyle Smooth}}_{L1}(x)=\left\{\begin{array}{c}0.5{{\displaystyle x}}^2,if\left|x\right|<1\\ \left|x\right|-0.5,otherwise\end{array}\right.$$

(12)

Given the substantial influence of grasp position and angle on grasping success, the relative importance of local feature range, as indicated by claw opening width, is considered to be less significant compared to these factors.

5. Results

In this work, we conducted a large number of comparative experiments to verify our LGAR-Net2 method. We perform trials on the publicly available crawl datasets from Cornell and Jacquard [24]. Furthermore, to mitigate the risk of overfitting caused by the limited size of the Cornell dataset, this study incorporates measures like data clipping, introducing random rotations, and applying random scaling to augment the data. The enhanced dataset is adjusted to 224 × 224 in order to conform with the LGAR-Net2 network’s input requirements. To test the model’s discrimination and generalization abilities, the Cornell dataset is divided into two sub-datasets based on image splitting and object splitting. The Jacquard crawl dataset, a large-scale crawl dataset created from CAD models, eliminates the need for manual data collection and label sampling. The training set and test set of Jacquard are divided using five-fold cross-validation.

5.1. Experimental Details

In this paper, the graphics card is configured as NVIDIA GeForce RTX 2080 Ti (11 GB), and the Ubuntu16.04 system combined with the Pytorch 2.1.1 deep learning framework is used as the software platform for experiments. The experiments and verification in this paper are completed in the above platform.

• Model training implementation: We employ the Xavier normal distribution for initializing network parameters and utilize the Adam optimization algorithm for network optimization. The number of epochs is 50, 1000 batches of data are processed in each epoch, and the number of samples per input is set to eight.
• Evaluation metrics: The angle difference and Jaccard index are two commonly employed criteria for assessing grasping performance. According to the angle difference criterion, a successful grasp is defined as having a deviation of less than 30° between the predicted grasping area and its corresponding label. Under the Jaccard index, a grasp is deemed successful if the overlap between the predicted and labeled grasping areas exceeds 25%. The mathematical expression of the Jaccard index is as follows:

  $$J({\mathrm{g}}_{\mathrm{p}},{\mathrm{g}}_{\mathrm{t}})=\frac{\left|g_p\cap g_t\right|}{\left|g_p\cup g_t\right|}>0.25$$

  (13)
• In the formula, $g_p$ and $g_t$ denote the predicted grasping area and true grasping area value, respectively. The intersection of predicted grasping area and grasping area label is represented by $g_p\cap g_t$, while the union of predicted grasping area and grasping area label is denoted by $g_p\cup g_t$.

5.2. Comparative Study

5.2.1. Attention Mechanism Experiment

In order to verify the attention of the attention mechanism in the deep semantic feature fusion strategy to the object area, an attention comparison experiment was conducted with the previous research, and the results are shown in Figure 9.

As depicted in the figure, the basic network without employing a feature fusion strategy exhibits issues such as semantic ambiguity and feature confusion in the object regions, which leads to inadequate attention towards the graspable regions of the objects. As shown in Figure 7, the cup in line 1 is expected to possess the optimal grasping area within its body. The basic network without the attention mechanism neglects to allocate sufficient focus on the cup’s body, while the LGAR-Net2 network proposed in this paper can effectively direct attention towards the central region of the cup’s body and suppresses excessive emphasis on grasping the handle, thereby enhancing overall grasping quality. The optimal grasping position for the scraper in the third row is at the handle. The basic network without an attention mechanism has an even distribution of attention to the whole scraper, failing to accurately identify the best grasping area at the handle. The LGAR-Net2 network proposed in this paper effectively focuses on the grasping area of the handle and suppresses the ungrasping blade area. Compared to other networks, LGAR-Net2 with an added attention mechanism demonstrates heightened focus on graspable object areas.

5.2.2. Checkerboard Artifact Experiment

To demonstrate the efficacy of the feature recovery module in LGAR-Net2, we visually depicted the feature images obtained from the BIC block output and provided a comparative analysis with the results achieved by GRCNN. As illustrated in Figure 10, it is evident that the basic network’s output exhibits noticeable checkerboard artifacts. However, through bilinear interpolation within the BIC block, as exemplified in columns 3 and 6 of the output images, adjacent pixel values are effectively smoothed out while eliminating these artifacts. This substantiates our proposed BIC block’s ability to successfully eradicate checkerboard artifacts and significantly enhance decoder performance.

5.2.3. Experimental Verification on Datasets

In this section, we conducted comparative experiments using LGAR-Net2 LGAR-Net [23] and GRCNN networks on the Cornell dataset and Jacquard dataset, as depicted in Figure 11. Herein, rectangular boxes are employed to represent the grasping pose of network output.

As depicted in Figure 11, the baseline network exhibits issues of semantic ambiguity and feature confusion when processing interpretable region features, leading to inadequate attention towards these regions. For instance, during the comparison of the razor (4), interference from the uncatchable area can impact the results. The absence of an attention mechanism in the baseline network restricts proper emphasis on the central position of the cylinder; however, LGAR-Net2 effectively addresses this problem by considering catchable areas on both sides of the razor handle while avoiding uncatchable regions at its top. Furthermore, when there is a color similarity between the cylinder (1) and its background, relying solely on LGAR-Net2 may allocate excessive attention to the background. Nevertheless, LGAR-Net2 not only suppresses background features but also significantly prioritizes catchable areas. In comparison to its counterpart network, LGAR-Net2 with an attention mechanism demonstrates an exceptional level of focus on pixels within catchable areas.

The findings from

Table 1. Cornell dataset experimental results.

Author	Method	Input	Accuracy (%)	Times (ms)
Jiang [2]	Fast Search	RGB-D	60.5	5000
Lenz [3]	SAE, struct. reg.	RGB-D	73.9	1350
Wang [18]	Two-stage closed-loop	RGB-D	85.3	140
Redmon [4]	AlexNet, MultiGrasp	RGB-D	88.0	76
Morrison [5]	GG-CNN	D	73.0	-
Karaoguz [25]	GRPN	RGB	88.7	200
Kumra [6]	GRCNN	RGB	96.6	19
Ours	LGAR-Net2	RGB	97.7	15

indicate that the grasping detection algorithm introduced in this paper achieves a better balance between accuracy and speed without relying on depth information, leading to improvements in real-time performance and accuracy.

The network proposed in this paper was utilized to conduct experiments on the Cornell dataset and compare with the existing research methods. The results are presented in

Table 2. Jacquard dataset experimental results.

Author	Method	Input	Accuracy (%)	Times (ms)
Depierre [24]	Jacquard	RGD	93.6	-
Morrison [5]	GG-CNN2	D	84	19
Kumra [6]	GRCNN	RGB	91.8	19
Ours	LGAR-Net2	RGB	94.4	17

, indicating that when using RGB data, the method proposed in this paper achieved a capture detection accuracy of 94.4% on the jacquard dataset.

5.3. Simulation Experiment Capture Detection

The UR5’s 3D model was imported into the ROS control system “Moveit” to establish a robotic arm control system, with configuration parameters automatically generated by algorithms. Subsequently, the Gazebo simulation environment incorporated 3D models of objects and cameras to create a grasping simulation test platform. Additionally, the grasp detection model was integrated into the ROS system for conducting simulation experiments. Figure 12 illustrates simulated capture and detection scenarios.

The method for robot grasping detection proposed in this paper is utilized for the purpose of grasping and detecting objects within a simulated environment. The simulation results for the detection of various objects are presented in

Table 3. Results of simulated grasping experiments.

Target Object	Grasp Pose Detection Accuracy (%)	Success/Attempt	Accuracy (%)
Coke Bottle	87.5	13/15	86.7
Wood Block	88.1	13/15	86.7
Screwdriver	93.5	14/15	93.3
Remote	90.1	13/15	86.7
Hexagon Nut	93.2	13/15	86.7
Weight	88.7	13/15	86.7

.

The analysis of the experimental results indicates that the pose detection accuracy of the network in this paper is up to 93.3%, which is slightly lower than the maximum accuracy of the model training, but it also demonstrates good detection performance. Utilizing the grasping pose detection algorithm designed in this paper, the robot can basically complete the grasping task in the simulated scene. Across various target objects and diverse object placements, the machine consistently achieves successful grasping actions, underscoring the algorithm’s robust adaptability to novel scenes.

6. Conclusions

We propose LGAR-Net2, an efficient grasp detection network for detecting the grasp pose of unknown objects. To address the checkerboard artifact problem commonly encountered in inverse convolution, we combine bilinear interpolation and introduce the LGCA block to enhance the feature extraction ability of graspable regions. As a result, our approach significantly improves both the accuracy and generalization of the network. Experimental verification on the Cornell and Jacquard datasets confirms its effectiveness. Future research will focus on further enhancing the grasp detection ability in complex backgrounds by fusing multi-modal input methods and deploying grasp detection algorithms to real robotic arms for achieving real-time grasping tasks.