ICNet: A Dual-Branch Instance Segmentation Network for High-Precision Pig Counting

Abstract

A clear understanding of the number of pigs plays a crucial role in breeding management. Computer vision technology possesses several advantages, as it is harmless and labour-saving compared to traditional counting methods. Nevertheless, the existing methods still face some challenges, such as: (1) the lack of a substantial high-precision pig-counting dataset; (2) creating a dataset for instance segmentation can be time-consuming and labor-intensive; (3) interactive occlusion and overlapping always lead to incorrect recognition of pigs; (4) existing methods for counting such as object detection have limited accuracy. To address the issues of dataset scarcity and labor-intensive manual labeling, we make a semi-auto instance labeling tool (SAI) to help us to produce a high-precision pig counting dataset named Count1200 including 1220 images and 25,762 instances. The speed at which we make labels far exceeds the speed of manual annotation. A concise and efficient instance segmentation model built upon several novel modules, referred to as the Instances Counting Network (ICNet), is proposed in this paper for pig counting. ICNet is a dual-branch model ingeniously formed of a combination of several layers, which is named the Parallel Deformable Convolutions Layer (PDCL), which is trained from scratch and primarily composed of a couple of parallel deformable convolution blocks (PDCBs). We effectively leverage the characteristic of modeling long-range sequences to build our basic block and compute layer. Along with the benefits of a large effective receptive field, PDCL achieves a better performance for multi-scale objects. In the trade-off between computational resources and performance, ICNet demonstrates excellent performance and surpasses other models in Count1200, $AP$ of 71.4% and $A{P}^{50}$ of 95.7% are obtained in our experiments. This work provides inspiration for the rapid creation of high-precision datasets and proposes an accurate approach to pig counting.

1. Introduction

Pig counting is an important part of livestock farming industry, as well as the vital process of recording and managing information such as pig growth, feeding, and reproduction costs in a pig farm. Accurate pig counting in pigsties can improve the efficiency of pig farm management. However, the traditional manual visual counting [1,2] and electronic ear tag counting [3] are inefficient, and difficult to reconcile at a later stage. With the arrival of the deep-learning era and the rapid development of computer graphics cards, the computing power of computers has been increasing, accelerating the development of computer vision and deep-learning networks. Computer vision research is expanding, and it is widely used in object counting. Object counting algorithms provide the possibility of accurate counting, which is widely used in the field of agronomy [4,5,6,7,8] and accelerates the development of animal husbandry informationization.

Pig counting is a type of object counting which implements several mainstream methods like object recognition, object classification and object detection. For example, object detection counting can quickly achieve accurate counting for sparser scenes. The R-CNN proposed by Girshick et al [9], introduced a two-stage detection method for the first time. Furthermore, Wang [10] proposed a detection method that recognizes ear tags to check the specific situation of pigs in a production environment. Object detection methods use deep convolutional networks to obtain excellent target detection accuracy, but the accuracy of target recognition decreases when faced with scenes with occlusion between pigs and the sticky edges of pigs, which means it is not conducive for us to deploy it in real-world scene applications. For denser scenes, the current mainstream method is the density map estimation [11,12,13] based on convolutional neural networks (CNN). Feng et al. [11] proposed an architecture composed of density map network and a K-means clustering algorithm, which can quickly estimate the number of high-density and high-overlap crowds in the scene, but this method has a lower counting accuracy, which does not meet our requirements for accurate counting. In addition, some researchers focus on keypoints detection to monitor the status of pigs. Chen et al. [14] created a model to detect keypoints of pigs’ bodies and identify individual pigs, which can associate pigs across video frames. Nonetheless, since only three points are used to mark a pig in frames, the method is prone to fault detection and identifying the background as a pig.

Instance segmentation is a technique used to perform pixel-level classification with high accuracy. There are a number of segmentation networks that have performed well so far. Kaiming He et al. proposed Mask R-CNN [15] by incorporating a Feature Pyramid Network (FPN) [16] for feature fusion and hierarchical detection, employing a more accurate bilinear interpolation approach RoIAlign instead of the traditional RoIPooling, which can generate richer feature mapping maps. Since then, Mask-RCNN has become the primary method of instance segmentation. Then, SOLOv1 [17] introduced the concept of “instance categories” to look at the instance segmentation task from a completely new perspective. The “Instance category” assigns a category to each pixel in an instance based on the location and size of the instance, effectively transforming instance mask segmentation into a categorizable problem to be solved. SOLOv1 is a simpler and more flexible instance segmentation framework that outperforms Mask R-CNN in terms of accuracy. However, SOLOv1 currently faces the problems of inadequate representation and learning of masks, insufficiently high resolution of masks obtained from prediction and is too time-consuming NMS for post-processing. Therefore, SOLOv2 [18] adopted the idea of dynamic convolution, which enabled the network to dynamically segment the instances based on the position information, and proposed Matrix NMS, which greatly improved the speed of post-processing. Maskformer [19] as well as Mask2former [20] were proposed to refine the task of instance segmentation even more, using a mask classification model that predicts class-specific masks for each instance in an image. Since Maskformer suffers from the problem of high memory occupation, leading to training difficulties, Cheng et al. further proposed Mask2former, which is keyed to the Masked-attention Mask Transformer structure, is based on the self-attention mechanism [21], and uses the mask technique to generate the segmentation masks. Compared to the traditional encoder–decoder-based neural network model, Mask2former achieves better results in instance segmentation tasks. Nonetheless, existing methods still face challenges such as interactive occlusion, overlapping and inaccurate counting, and the lack of a substantial high-precision pigcounting dataset, which is attributed to the inefficiency in human production of labels. In summary, to complete the task of precise counting, we choose instance segmentation to detect and count pigs instead of objection detection, density estimation, etc., owing to the advantages of recognizing the outline of an object independently and clearly separating it from the background.

In this paper, on the one hand, in order to acquire high-precision datasets, we design a semi-automatic labeling software based on Segment Anything Model (SAM) [22], which are faster and more accurate than the manual labeling methods in labeling segmentation labels. On the other hand, for accurate pig counting in realscenario, we adopt the framework of Mask-RCNN, combining the advantages of DCNv3 [23] and dual-branch structure to design a pig counting backbone named ICNet, which achieves a better performance in high-precision dataset and less computational resource. In general, the contribution of our paper are summarized in the following three aspects:

• We create a semi-auto instance labeling tool called SAI, which is based on the SAM, to produce faster and more accurate segmentation labels, and a high-precision pig counting dataset called Count1200.
• We reconstruct a more robust backbone called ICNet, with a Pipe Layer and four Parallel Deformable Convolutional Layers (PDCLs), which effectively exploits the long time-series modeling property and is able to expand the sensory field while saving computational resources.
• We design a module named Parallel Deformable Convolutional Block (PDCB) that consists of a double-branch structure and skip-connection, which is used to form the PDCL. PDCB is based on DCNv3, which is able to achieve more features with fewer parameters, and is more suitable for our detection work on multi-scale datasets.

2. Materials and Methods

2.1. Count1200 Dataset

Count1200 is an instance segmentation dataset primarily designed for the problem of pig counting. The dataset comprises 1220 images, encompassing situations from various scenarios, including different times of day and varying lighting conditions. We collect images from the Internet and take pictures at the livestock farm, representing 75% and 25% of the total number of images, respectively, consisting of situations of small-scale objects (Figure 1c,d), occlusion between pigs and other obstructions such as beams and feeding troughs (Figure 1a,f), overlapping or clustering among pigs (Figure 1e,f), and low-light challenges posed by nighttime perspectives (Figure 1e); this enables our dataset to possess diversity and robustness. Considering the need for multi-scale counting, the data we collected include 9–28 pigs per image.

It is a tedious and difficult task for researchers to make the ground truth of instance segmentation, which has persistently hindered the development of instance segmentation [24]. In traditional manual annotation methods, researchers often use the method of clicking around the pigs multiple times to draw polygons for mask labels. When there are too many objects in the image, this type of annotation process is very time-consuming and labor-intensive. With its strong ability to identify masks of natural objects, SAM can segment objects easily; that means this model can rapidly generate numerous points to outline the shape of anything, even in complicated environments. The details of SAI are shown in Figure 2.

Firstly, M-embedding files are generated by the encoder of ViT and correspond one-to-one with the images, which contain feature information and positional encoding. Prompt input is a series of mouse operation instructions used to select regions of concern. With the Graphical User Interface (GUI) of SAI, we can use left click to select pigs and right click to fine-tune the areas of interest in real time. When it comes to the details of the SAM model, the mask decoder plays a pivotal role and the peripheral operations are sent to the mask decoder after passing through the prompt decoder. In addition, the combination of embeddings and image masks serves as input for the mask decoder, too. The mask decoder utilizes the above resources to make mask predictions and provide real-time feedback in the image. The specific visualization effect is illustrated in the right size of Figure 2.

We successfully leverage the features above to develop a semi-auto instance labeling tool named SAI with a convenient GUI. With just left and right clicks, we can generate a mask in 5 s, which is seven times faster than manual annotation. The comparison of the two methods is illustrated in Figure 3. As opposed to the manual annotation with coarse-grained contour, the mask produced by SAI seamlessly fits around the objects, which helps models to separate pigs and background quickly and allows the feature of instances to be extracted efficiently without interference from irrelevant information, as shown in Figure 3a,b.

Along with the great performance of SAI, we create a COCO-type dataset [25] that consists of 25,762 instances and divide them into training and testing sets in an 8:2 ratio, containing 975 and 245 images, respectively. This dataset presents significant challenges for both instance segmentation and counting tasks. We aim to provide a foundational idea for subsequent research.

2.2. Methods

This research concentrates on a pig-counting task based on instance segmentation, with the aim of achieving clear separation among objects and between objects and the background that facilitates accurate counting. With its great performance in CNN models, the whole structure of our model follows the design of Mask-RCNN [15], as shown in Figure 4. The results of prediction consist of coordinates of bounding boxes, masks, and categories of instances.

2.2.1. Instances Counting Network (ICNet)

Deviating from the traditional pipeline of Mask-RCNN using ResNet [26] as a feature-extracting module, we suggest ICNet as a more robust and appropriate backbone for instance segmentation tasks (the area outlined by the blue dashed box in Figure 3). The details of ICNet are illustrated in Figure 5.

By combining it with a series of layers, we construct a network with the ability to achieve long-range dependence, adaptive spatial aggregation and efficient utilization of memory and computation. That means, compared to normal convolution, we have a larger receptive field and can better preserve information such as long-distance spatial information and dense pixel interaction information extracted from images at the same kernel size. One pipe layer and four PDCLs serve as the main building blocks of ICNet, with a downsampling layer positioned after each PDCL. The feature map is sent to FPN and RPN [27] for additional information compression and proposal generation after semantic and spatial information have been extracted from the input. After implementing ROI Align, we use a Fully Convolutional Networks Mask Head to generate a mask of regions proposals, and we follow the design of Mask R-CNN to use the fully convolutional layers to predict the coordinates and categories of instances.

2.2.2. Pipe Layer

The pipe layer is a combination of two cycles comprising $3\times 3$ convolution, Batch Normalization [28] and $ReLU$ [29], whose function is to preprocess the data and adjust the resolution of input, improving the resolution of basic semantic information. A single combination cycle is used to reduce the input resolution by two times; the configuration of the convolutions is $stride=2$ and $padding=1$. The specific pipeline of two cycles can be described using Formulas (1) and (2), where ${\mathit{X}}_{input}\in {\mathbb{R}}^{H\times W\times 3}$ and ${\mathit{X}}_{output}\in {\mathbb{R}}^{H\times W\times 64}$.

$$X_{temp}^{H\times W\times 32}=ReLU\left(BN\left(Conv\left(X_{input}\right)\right)\right)$$

(1)

$$X_{output}=BN\left(Conv\left(X_{temp}\right)\right)$$

(2)

2.2.3. Downsampling Layer

Instead of using a single normalization layer in other networks, we combine Batch Normalization (BN) and Layer Normalization (LN) [30] for the downsampling layer, which preserves the benefits of modeling long-range sequences and addresses internal covariate shift. The function of the downsampling layer is to reduce scales of input by half and double the number of channels, which is similar to the function of the pipe layer, as indicated by the blue box in the lower part of Figure 5.

The downsampling layer ensures that the model incorporates multi-scale information, making it more conducive to downstream tasks when data comprise objects of various sizes. The details of the downsampling are shown in Formula (3), where $Permute\left(\right)$ is a dimension transformation in the order of (0, 3, 1, 2).

$$X^{\prime H\times W\times C/2}=BN\left(Permute\left(LN\left(X^{H\times W\times C}\right)\right)\right)$$

(3)

2.2.4. Parallel Deformable Convolutions Layer (PDCL)

The PDCL is a sequence composed of computing basic blocks named parallel deformable convolution blocks (PDCBs). The implementation details of this computing layer are illustrated in Figure 6.

An aggregation of N basic blocks is represented by PDCL, where N determines the size of each layer and serves as a hyper-parameter for the number of PDCB. For instance, in this work, we stack four layers to build ICNet, where the array for N is $N_i\in [2,2,8,2]$ indicating that there are two blocks in $Laye{r}_1$, eight blocks in $Laye{r}_3$, and so on.

In order to achieve a balance between performance and resources consumption, a dual-branch structure is developed by us, whose core operation is DCNv3 that occurs after $LN$ and before a drop path layer [31]. In our experiment, layer normalization exhibits strong adaptability with DCNv3 and we make good use of this feature to build our basic block. Compared to the basic block in InternImage [23], our PDCB uses less parameters to achieve a better performance owing to the creative unities of double branches features and skip-connection [26]. The dual-branch blocks can be depicted as Formula (4):

$$X_i=Drop\_Path(\gamma \times DCNv3\left(LN\left(X\right)\right)),(i=1,2)$$

(4)

Integer i represents the ith branch in the current block, while $\gamma$ is a hyper-parameter used to scale the results and we set $\gamma =1$ in this research.

$Drop\_path\left(\right)$ is a layer of network regularization method, which is proposed for branch networks. Compared to the common method used to prevent overfitting—drop out [32], which applies to specific feature maps—$Drop\_path\left(\right)$ involves randomly dropping network branches and is particularly suitable for multi-branch structured networks.

In summation, the principal procedure of PDCL can be described as Formula (5):

$$Laye{r}_j=F_N\left(MLP\left(LN(X+\sum _{i=1}^2{X}_i)\right)\right),(j=1,2,3,4)$$

(5)

where j is the jth layer in ICNet, and $F_N$ means that we recursively call N PDCBs to process the input.

2.2.5. Deformable Convolutions v3 (DCNv3)

Deformable Convolutions was created in 2017 by Jifeng Dai [33] and further improved by Xizhou Zhu in 2019 [34]. It has been frequently used in instance segmentation and object detection-related tasks, since it has an irregular convolution shape, which allows a more flexible range of receptive fields when scanning images. Owing to the advantages listed above, deformable convolution is adapted to this work with the aim of achieving multi-scale detection of targets.

Regular shapes are always equipped in common convolution such as square ($1\times 1$, $3\times 3$ kernel), rectangle ($1\times 7$ kernel), and so on, which can traverse images patch by patch to a generate feature map. This can be summarized by the following formula, taking $3\times 3$ kernel as an example:

$$M_{out}\left(x_0\right)=\sum _{x_n\in \mathcal{C}}w\left(x_n\right)·M_{in}(x_0+x_n)$$

(6)

where $M_{in}$ and $M_{out}$ are the input feature map and output feature map, respectively, and $x_0$ is the current pixel point handled by the kernel. $\mathcal{C}$ represents a grid region of coordinates that determines the dilation and receptive field size of kernel, the visualization effect is as shown in Figure 7a and $x_n$ traverses the position in $\mathcal{C}$. $w\left(x_n\right)$ is a corresponding weight coefficient of enumeration in $\mathcal{C}$.

For the DCNv3, an offset matrix and ideas of groups dividing are incorporated into the convolution operation.

The offset matrix is a learnable convolutional layer that indicates the expansion scale and direction of the current convolution kernel. This particularity means common convolution had a deformable receptive field and the visible range of convolution is irregular. The values in the offset are highly flexible, granting characteristics of long-range dependencies and spatial aggregation which is similar to that of Transformer.

The concept of a multi-group [35] is initially applied to image classification tasks and significantly excels in MHSA. Partitioning operators into multiple groups is equivalent to dividing them into multiple subspaces, facilitating models to focus on different pieces of information from various zones. The calculation process of deformable convolution v3 is displayed as

$$M_{out}\left(x_0\right)=\sum _{g=1}^G\sum _{n=1}^N{w}_g{m}_{gn}·M_{in}(x_0+x_n+\Delta x_{gn})$$

(7)

G is the number of groups we set [2, 4, 8, 16] for each PDCL, meaning that the dimensions of the feature map in each group are $C_g=C_i/G_i$, where $C_i$ represents the total channels in $Laye{r}_i$. Positive integer N is the number of sampling pixels that equal $\left|\mathcal{C}\right|$ in Formula (6). The offset matrix is denoted by $\Delta x_{gn}\in {\mathbb{R}}^{H\times W\times 2S}$, where S is the kernel size of convolution and $2S$ rules the offset in the $\mathcal{X}$ and $\mathcal{Y}$ directions. $w_g\in {\mathbb{R}}^{C_i\times C_g}$ on behalf of the projection weights of the current group, which is similar to $w\left(x_n\right)$ in Formula (6). As for the $m_{gn}$, it represents a scale factor of $x_n$ in this group.

Throughout the stages of growth, pigs are housed in different pigsties, since pigs vary greatly in their body shape at different periods. Additionally, differences in the capture distance during imaging can result in variations in individual sizes within the same pen [36]. Moreover, the shooting edges of the camera can also cause image distortion. The truths mentioned above can easily cause inaccuracies in detection models with datasets collected from real farming scenarios. With the help of DCNv3, our model can adapt to the changing feature distributions, which can greatly alleviate the impact of the above issues.

2.3. Experimental Environment Setup

All experiments are conducted under the Pytorch & CUDA framework in a version of 1.12.1 and 11.3, respectively. We use a NVIDIA (Santa Clara, CA, USA) RTX A6000 to train these models from scratch with a batch size of 32 and 200 epochs.

2.3.1. Data Preprocessing

We set the size of input images to [512, 512], a common setting in many segmentation models, which creates a balance between segment accuracy and memory consumption. Apart from this, data augmentation methods are employed to enhance the generalization capability, including random flipping and random cropping, with a flipping ratio of 50% and crop size of [512, 512].

2.3.2. Optimization Strategy

AdamW [37] is used in our research with the strategy of layer decay [38]. We initialize the $lr=1\times {10}^{-4}$ with the weight decay = $5\times {10}^{-2}$. In addition, we perform a linear warm-up [39] in [180, 190] epochs with $\gamma =0.1$. The remaining parameter configurations remain consistent with Mask R-CNN [15] and InternImage [23].

2.3.3. Evaluation Metric

COCO evaluation metrics are used in this research to estimate our results. True Positive (TP), False Positive (FP), True Negative (TN), False Negative (FN), and Intersection over Union (IoU) are applied to compute the precision of segmentation. The $IoU$ describes the relationship between the predicted mask and the ground truth in terms of their overlapping areas, illustrated in Formula (8).

$$IoU=\frac{Area\left(O\right)}{Area\left(U\right)}$$

(8)

where $Area\left(O\right)$ represents the overlapping area between label and prediction and $Area\left(U\right)$ is the sum of these two areas. Meanwhile, the precision and recall at a specific threshold are represented by the following formula, where t is the threshold of $IoU$.

$$P=\frac{TP\left(t\right)}{TP\left(t\right)+FP\left(t\right)}$$

(9)

$$R=\frac{TP\left(t\right)}{TP\left(t\right)+FN\left(t\right)}$$

(10)

$AP$ is calculated using Formula (11) and $mAP$ is the average of $AP$ values across all categories.

$$AP={\int }_1^{0}P\left(r\right)dr$$

(11)

We will display the $AP$ and $AR$ at different IoU thresholds [0.5:0.05:0.95] of results in the next section.

Beyond that, we select the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) to assess the precision of counting [40]. These two metrics are calculated by the formula as follows:

$$MAE=\frac{1}{N}\sum _{n=1}^N|y_n-{\widehat{y}}_n|$$

(12)

$$RMSE=\sqrt{\frac{1}{N}\sum _{n=1}^N{(y_n-{\widehat{y}}_n)}^2}$$

(13)

where N is the total number of images and n is nth image in this set. The number of actual instances in the nth image is denoted by $y_n$, while the number of predicted masks is indicated by ${\widehat{y}}_n$. $MAE$ and $RMSE$ are inversely correlated with the model’s accuracy and robustness.

3. Results

3.1. Results on the Test Set

We train our model with an input size of $512\times 512$, and we conduct a comparative experiment between our model and mainstream instance segmentation models in terms of # params, # FLOPs, AP, and AR. The test results are summarized as follows in

Table 1. Comparison of the Count1200 test set. The volume of parameters and FLOPs are obtained across the entire model workflow. We show the key evaluation metrics from the COCO dataset.

Methods	Scale	Param.	# FLOPs	$\mathit{AP}$	${\mathit{AP}}^{50}$	${\mathit{AP}}^{75}$	${\mathit{AP}}^{\mathit{M}}$	${\mathit{AP}}^{\mathit{L}}$	$\mathit{AR}$
SOLOv1	${512}^2$	36M	106G	47.3	82.6	51.7	9.7	53.4	56.5
SOLOv2	${512}^2$	65M	133G	60.6	90.9	67.4	33.2	64.5	67.4
Mask R-CNN	${512}^2$	62M	134G	47.8	77.1	54.2	13.3	52.7	54.9
Mask2Former (R)	${512}^2$	63M	117G	53.5	85.1	59.9	15.5	58.9	60.0
Mask2Former (S)	${512}^2$	69M	125G	54.1	84.6	61.0	15.0	59.7	60.6
InternImage	${512}^2$	49M	117G	67.6	93.5	81.4	37.7	71.6	70.7
ICNet	${512}^2$	33M	106G	71.4	95.7	86.4	47.5	74.5	74.3

Mask2Former (R) and Mask2Former (S) means that use the ResNet and Swin Transformer as backbones, respectively. The bold numbers in the label represent the optimal value.

.

ICNet arrives at $AP$ of 71.4% and $A{P}^{50}$ of 95.7% with parameters of 33M, the highest score and the fewest parameters of all the models. In addition, we also outperform other networks in the rest of the metrics. Compared to SOLOv1 with similar parameter values, we achieve a 24.1% increase in $AP$ and a 13.1% increase in $A{P}^{50}$. In contrast, SOLOv2 obtains $AP$ of 60.6 % and $A{P}^{50}$ of 90.9%, which are close to our results, but it uses more parameters. Compared to the model based on Vision Transformer(ViT) [41] and Mask2Former, Mask2Former still cannot achieve matching results with different backbones, and due to the use of ViT’s architecture, its number of parameters is about twice that of ICNet. In comparison with InternImage, which also builds basic blocks based on DCNv3, ICNet surpasses InternImage in all metrics using only 33M parameters while InternImage has parameters of 49M. As for the $AP$ and $A{P}^{50}$, the results obtained using ICNet are 3.8% and 2.2% higher than those obtained using InternImage.

ICNet and InternImage have better segmentation accuracy compared to other models in various metrics, which verifies the superiority of DCNv3 in instance segmentation and pig counting. Furthermore, the novel design of the dual branch structure allows us to more effectively extract the feature information contained in the images, while the addition of a drop path prevents the model from overfitting. In addition, the use of skip connections helps us to mitigate feature loss after multiple layers of convolutional processing, which is crucial for information transmission. In addition, the combination of downsampling layers and the use of BN and LN between layers grants us abundant features compared to other models. Our ablation experiment also indirectly proves this point.

3.2. Ablation Study

We use one branch without drop path and a single LN as a normalization layer to build up the simple block, which is the foundation of the entire ablation study. We attempt to add BN, dual branch structure, and drop path for testing. The test results are shown in the

Table 2. The ablation experiment of ICNet. AP and AR are displayed to represent the improvement with different module combinations.

No.	Simple Block	BN	Dual	Drop Path	AP	AR
1	√				66.5	69.7
2	√	√			70.7	73.8
3	√	√	√		71.0	74.3
4	√	√		√	70.8	73.7
5	√	√	√	√	71.4	74.3

The √ symbol represents that we use this component in the current experiment. The bold numbers in the label represent the optimal value

below.

The experiment demonstrates that the model’s performance cannot be enhanced by a single LN layer. Combining BN and LN can better induce and compress data, achieving 70.7% and 74.3% on AP and AR, respectively. In addition, when we introduce the dual branch structure into the model, the model improves by 0.3% on AP compared to Experiment No. 2, but achieve the same 74.3% on AR as Experiment No. 5, which is the highest value in the entire experiment. This fully demonstrates the effectiveness of the dual branch structure. Drop path is introduced in experiment No. 4, but our AR decreases by 0.1% and our AP only improves by 0.1%, suggesting that the effect of drop path is not significant for a single branch. We conduct a No. 5 experiment by combining drop path with double branches, which shows significant improvements in both AP and AR compared to the initial simple block. We also choose this structure as the foundation for our PDCB.

3.3. Evaluation of Counting

In the evaluation of counting results, the test results of each model in MAE and RMSE are shown in the

Table 3. Comparison of counting result evaluations.

Metrics	MAE	RMSE
SOLOv1	3.71	4.69
SOLOv2	1.20	1.83
Mask R-CNN	2.86	3.55
Mask2Former (R)	1.73	2.37
Mask2Former (S)	2.39	3.13
InternImage	0.93	1.38
ICNet	0.68	1.07

The bold numbers in the label represent the optimal value.

.

It is evident from the table that ICNet performs best in terms of both MAE and RMSE (the smaller the MAE value, the higher the model evaluation accuracy, and the smaller the RMSE value, the stronger the model robustness). Combining it with Table 1 shows that models with better segmentation performance also have higher accuracy when it comes to counting. Therefore, the instance segmentation method for pig counting is very feasible and effective, and the model we build is accurate and robust enough.

3.4. Visualization Results

We test ICNet, InternImage and Mask R-CNN on the test set of Count1200 and the visualization results are displayed below.

We select scenes with multi scales and different lighting intensities. The visualization results of Mask R-CNN are shown in Figure 8, showcasing issues such as repeatedly segmenting the same pig, missing pigs and identifying the background as pigs, which is a undesirable situation for detection and counting purposes. Owing to its close accuracy compared with ours, as shown in Figure 9, InternImage exhibits similar detection performance on large-scale targets to ours, such as in the clear segmentation results in Figure 9a. In addition, when facing scenes with a large number of objects, InternImage may fail to count a significant number of pigs, as illustrated in Figure 9b,d. Compared to the rough detection results of Mask R-CNN and the coarse-grained segmentation results of InternImage, the masks we produce almost seamlessly fit the pigs and can still be effectively recognized even when the pigs overlap. We can even accurately segment pigs when they are obstructed by objects such as the iron pipes in Figure 10a,e,f. Besides, there are few cases of missed or incorrect detections, which are consistent with the results in the Table 3.

4. Discussion

To accomplish the goals of instance segmentation and pig counting, a dual-branch network and a well-prepared dataset are presented in this work. Count1200 is a dataset which is highly effective in counting and identification tasks owing to the advantages of instance segmentation. ICNet is a backbone specifically designed for segmentation and counting, consisting of a sequence of pipe layers and PDCLs which can enable the model to obtain multi-scale features. A dual branch structure of PDCB is used to design the PDCL using deformable convolution v3. This model benefits from the characteristics of long-range dependencies and spatial aggregation provided by DCN in conjunction with the idea of multi-group design, as shown in the experimental results table (Table 1 and Table 3), we achieve the best performance in both instance segmentation and counting tasks. Moreover, the visual results, which showcase the seamless masks and highly accurate counting, further highlight the performance of our dataset and model.

Although the test results of our model are good, far better than those of the rest of the models, it still encounters a few issues. The masks we predict are correct and accurate in number, but there are white areas in the middle, failing to completely cover the entire pig. In addition, our method still has a very small number of false positives and missed detections. The problems above will be solved in future model enhancements.

5. Conclusions

In this paper, with the help of SAI, a well-curated dataset named Count1200 is created. Beyond that, leveraging the variable receptive field range of DCNv3 and the robust extraction capabilities of the dual-branch structure, we develop a network with a small parameter count and outstanding performance named ICNet. We make good use of the strengths of our dataset and model to achieve better performance in tasks of instance segmentation and pig counting. Through a series of repeatable comparative experiments, we validate that our model exhibits better segmentation and counting capabilities compared to other models. We intend to further enhance the model by considering both lightweight and high-precision aspects.