DTNet: A Novel Infrared-Visible Fruit Object Detection Method Based on Dual-Modal Feature Interaction Fusion and Transformer Decoding

Abstract

Fruit detection in orchard environments is often challenged by illumination variation, leaf occlusion, cluttered backgrounds, and dense distributions of small targets, which limit the robustness of visible-light detectors in practical applications. To address these issues, this study proposes DTNet, an end-to-end RGB-IR dual-modal detection framework for close-range fruit instance recognition. Rather than relying on single-modal appearance cues, DTNet improves detection robustness by jointly exploiting cross-modal complementary perception, fine-grained local detail enhancement, and global relational modeling, thereby strengthening the representation of small, partially occluded, and illumination-sensitive fruit targets. In addition, a multimodal balanced detection loss is introduced to improve optimization stability under class imbalance and hard-sample interference. Experiments on a grape RGB-IR object detection dataset show that DTNet achieves a Precision of 0.9132, a Recall of 0.8931, an mAP@0.5 of 0.9552, and an mAP@0.5:0.95 of 0.8001, outperforming competing methods overall. On an additional tomato RGB-IR dataset, DTNet also maintains stable detection accuracy, indicating favorable adaptability to another fruit RGB-IR detection task. The results indicate that DTNet is a promising approach for robust RGB-IR fruit detection in complex orchard environments.

1. Introduction

1.1. Background and Significance

Horticultural crops such as grapes and tomatoes play an important role in the global food supply chain and high-value fresh produce market. Accurate and timely monitoring of fruit growth status, health condition, and maturity is a key prerequisite for yield estimation, quality grading, and precision management in modern orchards and protected agriculture systems [1,2]. With the rapid development of smart agriculture, vision-based fruit object detection has become one of the core supporting technologies for applications such as automatic spraying, robotic harvesting, and yield mapping [3,4,5]. However, constructing reliable detection systems in real production environments still faces many challenges, including complex and varying illumination conditions, occlusion caused by dense foliage, similar colors between fruits and leaves, and the coexistence of different cultivars and maturity stages [6,7].

1.2. Related Work

1.2.1. Fruit Object Detection Based on RGB Images

At present, most fruit detection methods still mainly rely on RGB images and commonly adopt one-stage detection frameworks based on single-modal convolutional neural networks, particularly the YOLO family of models [8,9,10]. These methods can usually achieve satisfactory detection accuracy and real-time performance under controlled conditions [11,12]. However, their performance often degrades significantly in low-light, backlit, or overexposed scenarios, because the color and texture information in visible images tends to deteriorate or even become distorted under such conditions [8,13]. In contrast, infrared (IR) imaging can provide complementary information beyond the visible spectrum and generally exhibits stronger robustness to illumination changes and background interference. In recent years, some studies have started to apply multispectral or RGB-IR data to crop monitoring and disease detection [13,14], while others have further demonstrated the application potential of multimodal information in complex agricultural scenarios [15,16,17]. Nevertheless, systematic and in-depth research on dual-modal feature extraction and fusion architectures for fruit detection remains limited.

Early fruit detection methods mainly relied on handcrafted features such as color, texture, and shape, combined with threshold segmentation, clustering, or traditional machine learning classifiers for recognition. Although such methods are simple to implement and computationally inexpensive, they are highly sensitive to illumination variation, background changes, and occlusion, making them unsuitable for complex orchard environments. With the development of deep learning, convolutional neural networks (CNNs) and one-stage detectors have gradually become the mainstream. Guo et al. proposed YOLOv4+ based on YOLOv4 [8], which improves grape detection robustness in complex backgrounds by introducing an attention module and multi-scale feature fusion, achieving better performance than the original YOLOv4 under occlusion and illumination variation. Liu et al. proposed TomatoDet [9], which adopts an improved anchor-free structure and attention mechanism, significantly enhancing detection accuracy and speed for greenhouse tomato detection under multi-scale, small-target, and severe-overlap conditions. Pinheiro et al. used multiple pre-trained YOLO models for grape bunch detection and health status classification, and established a public dataset, demonstrating the application potential of YOLO-based methods in grape bunch quality evaluation [10]. These studies indicate that RGB-based single-modal detectors are already capable of achieving high-accuracy fruit detection in specific orchard scenarios [13,18,19,20]. Meanwhile, to meet deployment requirements in agricultural scenarios, some studies have further balanced detection speed and model complexity through lightweight network design [21,22]. In addition, recent research on lightweight fruit recognition and localization methods has further shown that improving detection stability in complex environments while maintaining real-time performance remains an important research direction in this field [23,24].

However, most of the above studies rely heavily on visible-light images and remain vulnerable under extreme imaging conditions such as low illumination, strong shadows, backlighting, and intense reflection. When fruits have colors similar to leaves or are heavily occluded, RGB color and texture features alone are often insufficient to distinguish targets from the background. Moreover, to improve detection performance in complex environments, some studies introduce deeper or wider backbones, multi-scale feature pyramids, and complex attention modules, which inevitably increase the number of parameters and computational cost, making deployment on resource-constrained agricultural robots and edge devices more difficult. Therefore, how to introduce more stable and complementary perceptual information to enhance fruit detection robustness while maintaining model lightweightness remains an open problem [25].

1.2.2. Applications of Multispectral and Multimodal Imaging in Agricultural Perception

To overcome the limitations of RGB images in complex environments, an increasing number of studies have introduced near-infrared (NIR), multispectral, or hyperspectral imaging for crop monitoring and disease diagnosis. Multispectral sensors can capture spectral information beyond the RGB bands that is related to plant physiological status, and can therefore be used to identify features such as leaf nitrogen content and early disease symptoms that are difficult to perceive in standard RGB channels. In vineyard scenarios, Barros et al. utilized a UAV equipped with a high-resolution RGB camera and a multispectral sensor to construct a multimodal vineyard dataset containing RGB images, multispectral images, and digital surface models, and systematically compared the semantic segmentation performance of grapevines under different spectral combinations and resolutions [17]. Their results showed that near-infrared bands are particularly important for distinguishing grapevines from the background, whereas high-resolution RGB images can also achieve accuracy comparable to multispectral combinations.

In plant disease detection, the combination of multispectral/hyperspectral imaging and deep learning has also shown promising prospects. Several studies have used visible and near-infrared reflectance spectra, together with methods such as principal component analysis for band dimensionality reduction, followed by BP neural networks or 3D CNNs for disease recognition, enabling early diagnosis of fungal diseases. De Silva and Brown collected multispectral tomato leaf images under different environments and climatic conditions, compared the performance of multiple traditional machine learning methods and deep learning models for disease recognition, and further evaluated the influence of illumination changes and imaging angles on model stability. They also identified key spectral channels that contributed most to disease discrimination through band importance analysis [26]. Seiche et al. compared the performance of high-end and low-cost multispectral sensors for UAV-based weed detection using deep learning models, providing guidance for sensor selection in agricultural applications [27].

Overall, these studies demonstrate the advantages of spectral and multimodal imaging in crop monitoring and disease identification. However, most of them focus on canopy-level semantic segmentation or leaf-level disease classification, with target categories such as “healthy/diseased” or “crop/weed,” which are relatively coarse-grained. For RGB-IR dual-modal detection of fine-grained fruit instances in close-range orchard scenarios, specially designed feature extraction and fusion frameworks are still lacking. Existing multimodal methods often simply stack additional spectral channels at the input level or perform concatenation and weighting at high-level features, with insufficient emphasis on fine-grained cross-modal alignment and interaction modeling.

1.2.3. Multimodal Detection Models

From the perspective of network architecture, a variety of multimodal fusion strategies have been proposed, including early fusion at the input level, late fusion at the decision level, and intermediate fusion at the feature level [28,29]. Early fusion is simple to implement, but it is difficult to preserve the modality-specific characteristics sufficiently. Late fusion, in contrast, cannot effectively correct errors that have already been produced by single-modal branches. Intermediate fusion is more flexible; however, many existing studies still rely on simple feature concatenation or addition, or are mainly designed for large-scale remote sensing scenarios rather than fine-grained fruit instances in close-range agricultural environments. At the same time, Transformer-based detectors such as DETR and RT-DETR have demonstrated strong global modeling capability and competitive performance in general object detection [30,31]. Yet, their potential in multimodal fruit detection has not been fully explored, especially in combination with lightweight local enhancement and attention modules specifically designed for small and occluded targets.

In the broader field of multimodal learning, Baltrušaitis et al. and Zhao et al. systematically reviewed multimodal machine learning and deep multimodal fusion methods, pointing out that common fusion strategies can be categorized into early fusion, intermediate fusion, and late fusion, and comparing them from the perspectives of representation learning, cross-modal alignment, and collaborative learning [28,29]. In the object detection field, some studies mainly focus on effective interaction and fusion mechanisms among multimodal features [32,33,34], while others further emphasize lightweight design and deployment efficiency in order to balance detection accuracy and real-time performance [35,36,37].

In agricultural scenarios, multimodal detection and segmentation have also received considerable attention [33,38,39]. Existing studies have introduced RGB, RGB-D, near-infrared (NIR), multispectral images, and DSM data into tasks such as weed detection, crop row recognition, and vineyard canopy semantic segmentation. They have designed early fusion strategies based on vegetation indices, intermediate fusion strategies based on feature concatenation, and late fusion strategies based on case reasoning or voting mechanisms. The results show that multimodal input can usually improve the discrimination between crops and weeds, as well as between canopies and backgrounds, to some extent. However, the fusion strategies in these methods are mostly based on fixed rules or simple operators, and their modeling of fine-grained alignment and deep interaction across modalities remains insufficient. At the same time, multispectral and multi-filter studies for plant disease recognition focus more on “which spectral channels to select” and “what network structure to use”. In multimodal detection scenarios, additional modalities are often still simply stacked or concatenated at high-level features, with limited attention paid to the fine-grained perception of fruit targets at different scales and with varying occlusion levels.

Overall, existing studies either focus on improving detection accuracy under single-modal RGB conditions or utilize multispectral/hyperspectral imaging in more macroscopic crop monitoring or disease recognition tasks. Based on the above analysis, the current close-range orchard fruit detection task still faces two main challenges.

(1) The influence of complex illumination and background environments on visible-light detection accuracy. Orchard scenes commonly involve strong direct sunlight, backlighting, shadows, leaf reflections, and partial fruit occlusion, all of which can cause significant changes or even degradation in the color and texture features of RGB images. Single-modal detection models relying solely on visible-light information are prone to missed detections and false detections under such extreme or non-uniform illumination conditions, making it difficult to maintain stable performance across all-day and multi-scene environments.

(2) Limitations of existing dual-modal fusion detection methods. Although recent studies have attempted to combine RGB with near-infrared or other multi-source data for crop monitoring or disease recognition, most methods still remain at the level of simple input stacking or high-level feature concatenation, paying insufficient attention to cross-modal alignment and deep interaction. Meanwhile, dedicated dual-modal network architectures for close-range fruit instance detection that balance both accuracy and real-time performance are still relatively scarce, making it difficult to fully exploit the complementary advantages of RGB-IR information.

1.3. Main Contributions

To address these issues, this paper develops DTNet, an end-to-end network for RGB-IR dual-modal fruit detection. DTNet adopts a dual-branch feature extraction backbone to decouple the encoding of RGB and IR images. A Dual-modal Fusion Block (DFB) is introduced at the intermediate layers to achieve multi-level feature fusion through cross-modal alignment, fine-grained information interaction, and adaptive weight allocation, thereby effectively mining the complementarity between visible and infrared information. Based on the fused features, a lightweight Transformer-based adaptive demodulation detection head is designed to explicitly model long-range dependencies among targets as well as between targets and the scene, thereby enhancing the representation capability for complex spatial layouts and densely distributed targets. Meanwhile, a Local Enhancement Module (LEM) and an Adaptive Spatial Attention module (ASA) are embedded into multi-scale feature maps. On the one hand, they strengthen fine-grained texture descriptions of small-scale and partially occluded fruits; on the other hand, they suppress background redundancy and highlight key spatial regions. Finally, considering the class imbalance and ambiguous fine-grained class boundaries in orchard data, an improved detection loss function for multimodal input is constructed to coordinate the optimization of the classification and localization branches at the loss level, thereby improving overall accuracy while alleviating bias toward dominant classes.

The main contributions of this paper are summarized as follows:

(1) To address the insufficient robustness of single-modal detection in complex orchard scenarios, an end-to-end RGB-IR dual-modal fruit detection network, DTNet, is proposed. The network constructs a decoupled dual-branch feature extraction backbone to encode RGB and IR images separately at multiple levels, and designs a lightweight detection head for global relational modeling on top of the fused features. By organically combining the local detail representation capability of convolutional networks with the long-range dependency modeling capability of attention mechanisms, the proposed method significantly improves detection stability for small targets, occluded targets, and complex illumination scenarios on grape and tomato dual-modal object detection datasets.

(2) To overcome the limitation that traditional multimodal fusion methods often rely on simple channel concatenation and cannot fully exploit cross-modal complementary information, DFB is designed in DTNet. By aligning, interacting with, and adaptively aggregating RGB and IR features at multiple stages of image feature extraction, DFB enables more effective utilization of dual-modal complementary information while maintaining network lightweightness, thereby improving the discriminability and robustness of feature representations.

(3) To address the problem of fruit targets being small, densely distributed, and easily occluded by leaves, LEM and ASA are introduced into the feature pyramid. Specifically, LEM focuses on enhancing fine-grained local texture, boundary, and weak-structure representations for small-scale and partially occluded fruits, whereas ASA focuses on cross-scale feature interaction and spatial saliency reweighting to suppress background redundancy and highlight key fruit regions. Through this complementary design, the proposed framework achieves finer perception of small-scale, overlapping, and partially occluded fruit targets, thus improving detection accuracy with low additional computational cost.

(4) To address the insufficient optimization caused by class imbalance and subtle appearance differences among classes in orchard data, an improved detection loss function for multimodal input, termed Multimodal Balanced Detection Loss (MBDL), is proposed. By introducing targeted weights and constraint terms into the classification and localization branches, the method enhances attention to minority classes and hard samples, while coordinating classification confidence and bounding-box regression quality. As a result, a more balanced optimization process is achieved in multi-class fruit detection tasks, effectively alleviating model bias toward dominant classes.

The remainder of this paper is organized as follows. Section 2 presents the materials and methods, including the proposed DTNet framework, key module designs, datasets, evaluation metrics, and implementation details. Section 3 reports the experimental results, including baseline selection, comparative experiments, ablation studies, evaluation on an additional fruit dataset, and qualitative comparisons. Section 4 discusses the findings, limitations, and future research directions of the study. Section 5 concludes the paper. The overall organization follows the revised manuscript structure required by the journal.

2. Materials and Methods

2.1. Experimental Materials and Dataset Description

This study used two publicly available registered RGB-IR fruit datasets to evaluate the proposed DTNet, including a grape multimodal object detection dataset and a tomato multimodal object detection dataset with different maturity stages. Since both datasets were obtained from public repositories rather than collected by the authors, the dataset source and image acquisition information reported in this study follow the corresponding public dataset descriptions in Refs. [40,41]. The grape dataset was publicly released in 2023, whereas the tomato dataset was publicly released in 2025.

2.1.1. Grape RGB-IR Dataset

The main experiments were conducted on a public grape RGB-IR multimodal object detection dataset [40]. The dataset contains registered RGB-IR image pairs acquired in natural vineyard environments and annotated for object detection. According to the public dataset description, the multimodal images were captured using an Azure Kinect sensor (Microsoft Corporation, Redmond, WA, USA) at a resolution of 1280 × 720 in Hefei, Anhui Province, China. In this study, a total of 3954 image pairs were used, including 3163 pairs for training and 791 pairs for validation, corresponding to a fixed train/validation split ratio of 8:2 (80% for training and 20% for validation). The object categories were green_grape and purple_grape.

2.1.2. Tomato RGB-IR Dataset

To further evaluate the applicability of DTNet on an additional fruit RGB-IR detection task, a public tomato RGB-IR multimodal dataset with different maturity stages was used for additional-dataset evaluation [41]. Public dataset descriptions indicate that the source dataset includes multimodal tomato samples acquired under four lighting conditions, namely natural light, artificial light, low light, and sodium yellow light. In the present study, 1000 registered RGB-IR image pairs were selected from the public dataset, among which 800 pairs were used for training and 200 pairs were used for validation, corresponding to a fixed train/validation split ratio of 8:2 (80% for training and 20% for validation). The object categories were immature, half-ripe, and ripe.

2.1.3. Annotation, Data Selection, and Split Strategy

All images used in this study were registered RGB-IR image pairs with object detection annotations. For both datasets, a fixed train/validation split ratio of 8:2 was adopted, that is, 80% of the samples were used for training and 20% for validation. Specifically, for the grape dataset, all 3954 image pairs provided in the selected object detection subset were used and split into 3163 training pairs and 791 validation pairs. For the tomato dataset, 1000 image pairs were selected from the public source dataset and divided into 800 training pairs and 200 validation pairs. During training and inference, the paired RGB and IR images were synchronously loaded and resized to 512 × 512.

2.2. Overall Network Architecture

This study focuses on multi-class fruit object detection in close-range orchard scenarios. Given a registered RGB-IR image pair, the objective is to predict the bounding boxes and category labels of all fruit instances in the scene, including green grapes and purple grapes in the grape dataset, as well as immature, half-ripe, and ripe tomatoes in the tomato dataset. The network is required to maintain high detection accuracy under conditions of complex illumination, severe occlusion, and cluttered backgrounds, while also taking into account inference efficiency on agricultural robots and edge devices.

To this end, an end-to-end dual-modal detection network, DTNet, is constructed. DTNet consists of an RGB-IR dual-branch backbone, a Dual-modal Fusion Block (DFB), a Local Enhancement Module (LEM), an Adaptive Spatial Attention module (ASA), and a Transformer-based adaptive demodulation detection head. As shown in Figure 1, the overall architecture of DTNet is developed based on the published YOLOv13 framework [42] and its feature pyramid design, with several targeted modifications for RGB-IR fruit detection. In this study, YOLOv13 refers to the published version proposed by Lei et al. rather than a custom baseline defined by the authors:

(1) Lightweight feature extraction branches are separately constructed for RGB and IR in the backbone to preserve the modality-specific representation capability of each modality.

(2) At the medium- and high-level semantic feature scales, DFB is used to realize intermediate cross-modal alignment and fusion of RGB and IR features, yielding multi-scale joint representations.

(3) LEM and ASA are embedded into key nodes of the backbone and FPN/PAN, where LEM is used to enhance local detail and boundary representation, while ASA is used to strengthen cross-scale spatial interaction and salient-region emphasis.

(4) The fused multi-scale features are fed into a Transformer-based adaptive demodulation detection head to output the class probabilities and bounding-box parameters of fruit targets in a unified manner.

In the underlying published YOLOv13 architecture, HyperACE denotes the Hypergraph-based Adaptive Correlation Enhancement module, which is used to aggregate multi-scale features and capture high-order correlations across locations and scales. FullPAD denotes the Full-Pipeline Aggregation-and-Distribution paradigm, and the FullPAD Tunnel in Figure 1 refers to the feature propagation path that redistributes the correlation-enhanced features generated by HyperACE to different stages of the network pipeline. These components are inherited from the published YOLOv13 baseline and are retained in Figure 1 only to present the complete underlying architecture on which DTNet is built, rather than as newly proposed modules of this study.

During training, DTNet takes paired RGB-IR images and the corresponding annotated bounding boxes as input, and jointly optimizes the dual-branch backbone, DFB, LEM, ASA, and detection head in an end-to-end manner. During inference, the network requires only a single forward pass to complete detection and classification of all fruit objects in the image, which supports practical deployment-oriented inference in intelligent agricultural perception scenarios.

For convenience of description, the three detection feature maps in the feature pyramid, from high resolution to low resolution, are denoted as P3, P4, and P5, respectively.

2.3. RGB-IR Dual-Branch Backbone and Dual-Modal Fusion Block (DFB)

The detailed structure is shown in Figure 2.

2.3.1. Dual-Branch Feature Extraction Backbone

Considering the significant differences between RGB and IR in spectral response, texture information, and noise characteristics, DTNet adopts a decoupled dual-branch backbone to extract features from the two modalities separately. The two branches are structurally symmetric: shallow layers use standard convolution and depthwise separable convolution for spatial downsampling and channel expansion, while the middle and high layers employ improved residual structures and local enhancement units to progressively extract multi-scale features ranging from fine-grained textures to high-level semantics.

On the one hand, this design avoids the mutual interference caused by premature modality mixing in shallow layers, which helps preserve the advantages of RGB in color and texture as well as the stability of IR under illumination variation and occlusion. On the other hand, it provides structurally aligned and semantically corresponding feature inputs for the subsequent intermediate fusion performed by DFB, which is beneficial for cross-modal interaction at multiple scales.

2.3.2. Dual-Modal Fusion Block (DFB)

To fully exploit the complementary information between RGB and IR, DTNet introduces the DFB at multiple semantic scales. Let the feature maps from the RGB branch and the IR branch at a given scale be denoted as ${\mathrm{F}}^{rgb}\in R^{C\times H\times W}$ and ${\mathrm{F}}^{ir}\in R^{C\times H\times W}$, respectively. The objective of DFB is to achieve cross-modal alignment and adaptive fusion at the feature level, thereby generating the fused feature ${\mathrm{F}}^{dfb}$. The core process of DFB can be summarized into three steps:

(1)

Initial Fusion and Context Modeling

The features from the two modalities are first combined through element-wise addition to obtain an initial fused feature, as expressed in Equation (1).

$${\mathrm{F}}^{init}={\mathrm{F}}^{rgb}+{\mathrm{F}}^{ir}.$$

(1)

On this basis, an attention unit with deformable sampling and relative positional encoding is employed to perform contextual modeling on ${\mathrm{F}}^{init}$, thereby producing an enhanced feature ${\mathrm{F}}^{attn}$ that explicitly captures long-range dependencies and mines richer global information in both the spatial and semantic dimensions.

(2)

Pixel-Wise Adaptive Weight Estimation

Subsequently, ${\mathrm{F}}^{init}$ and ${\mathrm{F}}^{attn}$ are concatenated and fed into a pixel-level attention submodule. Through spatial-domain convolution and nonlinear mapping, a pixel-wise weight map $A\in {(0,1)}^{H\times W}$ is generated. The weight map A reflects which modality should be relied on more at each spatial location: at positions where texture is clear and color information is reliable, larger values of A tend to emphasize the RGB branch; at positions with complex illumination, severe shadows, or occlusion, the contribution of the IR branch is correspondingly strengthened.

(3)

Modality-Adaptive Fusion and Linear Remapping

Finally, the RGB and IR features are adaptively aggregated according to the weight map, combined with the initial fused feature, and then linearly remapped to obtain the final fused feature, as expressed in Equation (2).

$${\mathrm{F}}^{dfb}=Con{v}_{1\times 1}({\mathrm{F}}^{init}+A\odot {\mathrm{F}}^{rgb}+(1-A)\odot {\mathrm{F}}^{ir}),$$

(2)

where ⊙ denotes element-wise multiplication, and $Con{v}_{1\times 1}(·)$ denotes linear remapping in the channel dimension. In this way, DFB achieves adaptive balancing of the contributions of RGB and IR at each spatial position and each channel while maintaining an overall lightweight structure.

By stacking DFB at multiple scales such as P3, P4, and P5, DTNet can progressively perform cross-modal interaction from shallow to deep semantic levels, generating a set of multi-scale fused features that jointly encode color-texture information and infrared structural cues. This provides a more robust feature basis for subsequent small-object detection and occlusion-aware perception.

2.4. Local Enhancement Module (LEM)

In orchard scenarios, fruit targets often exhibit characteristics such as small size, dense clustering, and weak boundary contrast. In particular, under occlusion, abrupt illumination variation, or sparse infrared texture conditions, conventional convolutional operations on medium- and high-level features tend to suffer from detail attenuation and boundary blurring, leading to difficulty in separating overlapping fruits and insufficient responses at fruit stem junctions and fine lesion areas. To address this issue, DTNet introduces LEM at key stages of the backbone to strengthen fine-grained structural representation while maintaining the spatial resolution of the feature maps unchanged. LEM uses local windows as the basic modeling unit and explicitly captures correlations among neighboring features within each window through self-attention, while incorporating relative positional bias to enhance spatial relationship modeling. In this way, effective contextual information can be aggregated locally, thereby improving structural consistency and boundary discriminability. Meanwhile, LEM adopts lightweight designs such as depthwise separable convolution, which enhances local detail representation while effectively reducing parameter count and computational cost. Under a controllable computational budget, LEM further improves the distinguishability and localization stability of densely clustered targets and provides more reliable detail cues for subsequent feature fusion and detection head regression. The detailed structure of LEM is shown in Figure 3.

2.4.1. Overall Structure and Residual Learning

Given an input feature $x\in R^{B\times C\times H\times W}$, LEM adopts a two-stage residual structure corresponding to attention enhancement and nonlinear mapping (MLP) enhancement, as formulated in Equation (3).

$$X^{(1)}=X+A(X),Y=X^{(1)}+M(X^{(1)}),$$

(3)

where $A(·)$ denotes the attention enhancement branch, which is used to aggregate neighborhood context within local windows and strengthen responses to key details, and $M(·)$ denotes the MLP branch composed of pointwise convolutions, which is responsible for nonlinear transformation and recalibration in the channel dimension.

On the one hand, the two residual connections ensure that the semantic information of the backbone features can be stably propagated, thereby preventing excessive disturbance of the original representation caused by detail enhancement. On the other hand, they allow the enhanced information to be injected into the main branch in an incremental manner, thereby improving training stability and facilitating module convergence.

Furthermore, LEM maintains the spatial resolution of the features unchanged before and after the two residual enhancement stages. Therefore, it can strengthen the representation of dense-cluster boundaries, local textures, and structural continuity in occluded regions while controlling computational overhead, thereby providing more reliable fine-grained feature support for subsequent multi-scale fusion and detection head decoding.

2.4.2. Adaptive Normalization Module (LayNorm) and Recalibration

To alleviate scale drift among features from different levels, LEM introduces an adaptive normalization module, termed LayNorm, before the attention branch. Unlike standard LayerNorm, LayNorm not only outputs normalized features but also predicts a set of input-statistics-related rescale and rebias parameters, which are used for recalibration after the attention/MLP operations.

The procedure can be divided into the following steps.

(1) Global mean and variance computation.

The global mean and variance of the input feature are first computed, as expressed in Equation (4).

$$\mu =E[X],\sigma =\sqrt{E[{(X-\mu )}^2]+\epsilon }.$$

(4)

(2) Normalization with learnable affine transformation.

The input feature is then normalized and transformed by learnable affine parameters, as formulated in Equation (5).

$$\widehat{X}={\displaystyle \frac{X-\mu }{\sigma }},X_n=\widehat{X}\odot \gamma +\beta .$$

(5)

(3) Generation of recalibration parameters.

Based on the feature statistics, the recalibration parameters are generated as shown in Equation (6).

$$s={\psi }_1(\sigma ),b={\psi }_2(\mu ),$$

(6)

where ${\psi }_1$ and ${\psi }_2$ denote 1 × 1 convolution mappings (corresponding to meta1 and meta2 in the implementation).

Subsequently, recalibration is performed on the attention output, as expressed in Equation (7).

$$\mathrm{Re}cal(Z)=Z\odot s+b.$$

(7)

This enables the output of the enhancement branch to be adaptively restored to a numerical scale consistent with the original features, thereby improving cross-stage training stability.

2.4.3. Window Attention (WATT): Local Context Modeling and Relative Positional Bias

LEM models feature correlations at the local-window scale and aggregates neighborhood information through attention weighting to strengthen responses to key details. The feature map is first partitioned into non-overlapping windows, each containing $N=M^2$ tokens. For the features within each window, the query, key, and value are constructed as expressed in Equation (8).

$$Q=X{W}_Q,K=X{W}_K,V=X{W}_V.$$

(8)

For the h-th attention head, the attention weights and output features are computed as formulated in Equation (9).

$$A=soft\max ({\displaystyle \frac{Q{K}^{\top }}{\sqrt{d}}}+B),Z=AV,$$

(9)

where B denotes the relative positional bias. Specifically, the relative displacement $\Delta p$ within each window is first computed, and logarithmic compression is then applied as shown in Equation (10).

$$\Delta p_{\log }=sign(\Delta p)·\log (1+|\Delta p|).$$

(10)

The compressed relative displacement is then mapped to the bias term B of each attention head through a two-layer MLP. This design provides smoother modeling for displacements at different scales and enhances the representation of spatial consistency for boundaries and textures.

In addition, LEM supports shifted windows. When $shift\_size\succ 0$, the feature map is first circularly shifted, and reflect padding is applied at the window boundaries. After attention computation, inverse cropping is performed to restore the original size. In this way, information interaction across adjacent windows is introduced, thereby alleviating the boundary discontinuity caused by fixed window partitioning.

2.4.4. Attention Output Projection and Feature Fusion

After the window attention output $Z_{att}$ is obtained, LEM applies a 1 × 1 linear projection to perform channel mixing and alignment of the multi-head outputs, as expressed in Equation (11).

$$Z_p=\mathrm{Pr}oj(Z_{att}),$$

(11)

where $\mathrm{Pr}oj(·)$ denotes a 1 × 1 convolution.

Subsequently, the recalibration parameters s and b described in Section 2.4.2 are used to restore the scale and correct the bias of the enhanced feature, as formulated in Equation (12).

$$Z_r=Z_p\odot s+b.$$

(12)

Finally, the attention enhancement branch is injected into the backbone feature through a residual connection, as expressed in Equation (13).

$$X^{(1)}=X+Z_r.$$

(13)

In this way, the attention branch can enhance local detail representation while maintaining stable propagation of the original backbone semantics.

2.4.5. Channel-Wise Nonlinear Mapping and Second-Stage Residual Enhancement

To further improve the discriminability of fine-grained channel responses, LEM introduces an MLP branch based on pointwise convolution after attention enhancement. Let $X^{(1)}$ denote the output of the attention residual branch. The MLP output is then computed as expressed in Equation (14).

$$U=MLP(X^{(1)})=W_2\sigma (W_1{X}^{(1)}),$$

(14)

where $W_1$ and $W_2$ denote the two 1 × 1 convolutions, respectively, and $\sigma (·)$ denotes the nonlinear activation function.

Similarly, the MLP output can also be recalibrated, as formulated in Equation (15).

$$U_r=U\odot s^{\prime }+b^{\prime }.$$

(15)

Finally, the recalibrated MLP output is combined with the main branch through a second residual connection to obtain the final output, as expressed in Equation (16).

$$\mathrm{Y}=X^{(1)}+U_r.$$

(16)

This second-stage residual structure enables the original semantic information to be continuously propagated while injecting detail enhancement into the feature representation in an incremental manner, thereby improving training stability and avoiding excessive disturbance to the backbone feature distribution.

2.5. Adaptive Spatial Attention Module (ASA)

During the top-down construction of the feature pyramid, high-level features contain stronger semantic information but coarser spatial details, whereas low-level features possess higher resolution but weaker semantic abstraction. To facilitate more effective information interaction across scales, DTNet introduces an Adaptive Spatial Attention module (ASA) into the feature pyramid, enabling bidirectional interaction and spatial reweighting of cross-level features.

Given a high-level feature $F^H$ and a low-level feature $F^L$ at a certain scale, ASA first performs channel alignment through linear transformation and then computes the spatial attention maps for both branches, as expressed in Equation (17).

$$\begin{array}{cc}{G}^H=\sigma (F^H),& G^L=\sigma (F^L),\end{array}$$

(17)

where σ(⋅) denotes the element-wise Sigmoid function, and the values of $G^H$ and $G^L$ are constrained to the range (0, 1).

Subsequently, the features of the two branches are updated through self-enhancement and cross-level interaction, as formulated in Equations (18) and (19).

$$\tilde{F}=F^L+G^L\odot F^L+(1-G^L)\odot Up(F^H),$$

(18)

$$\tilde{F}=F^H+G^H\odot F^H+(1-G^H)\odot Down(F^L),$$

(19)

where ⊙ denotes element-wise multiplication, and Up(·) and Down(·) denote interpolation-based alignment operations that resize features to the spatial resolutions of the low-level and high-level branches, respectively. It can be seen that each branch consists of three components: the original feature, a self-enhancement term modulated by its own attention map, and a cross-level supplementary term injected from the other branch after attention-based selection.

Finally, the aligned features ${\tilde{F}}^H$ and ${\tilde{F}}^L$ are concatenated along the channel dimension and fused through convolution to obtain multi-scale features with rich local details and consistent high-level semantics. With the help of ASA, DTNet can explicitly highlight densely distributed fruit regions in the feature pyramid while suppressing interference from large leaf and branch areas, thereby improving the detection accuracy of small-scale, overlapping, and partially occluded fruit targets.

2.6. Transformer-Based Adaptive Demodulation Detection Head

After obtaining the fused multi-scale features, DTNet employs a Transformer-based Adaptive Demodulation Detection Head to decode features from scales such as P3, P4, and P5 in a unified manner. Centered on a set of learnable query vectors, this detection head formulates object detection as a set prediction task through cross-attention interaction with multi-scale feature maps. In this way, redundant candidate enumeration caused by dense anchor boxes can be avoided, and reliance on complex post-processing can be reduced.

Specifically, given the multi-scale features ${\left\{{\mathrm{F}}_l\right\}}_{l\in \{3,4,5\}}$ from the feature pyramid, the detection head first initializes a fixed number of query vectors $Q\in R^{N\times d}$. During decoding, cross-attention is used to aggregate contextual information relevant to each potential target from the multi-scale features, which can be formulated as Equation (20).

$$Q^{t+1}=Q^t+{\displaystyle \sum _{t}Attn}(Q^t,K(F_l),V(F_l)),$$

(20)

where $K(·)$ and $V(·)$ denote the linear mappings applied to the feature maps, and $Attn(·)$ denotes the multi-head attention operation. Through this cross-scale information aggregation, each query can adaptively select the most suitable scale and region, which is particularly beneficial for densely clustered fruits and occluded overlapping targets.

To further improve the stability and discriminability of candidate predictions, an adaptive demodulation mechanism is introduced at the output stage to dynamically recalibrate each query representation, which is expressed in Equation (21).

$${\tilde{q}}_i=q_i\odot s_i+b_i,[s_i,b_i]=g(q_i),$$

(21)

where $q_i$ denotes the i-th query vector, and $g(·)$ denotes a lightweight mapping network that generates the adaptive scaling factor $s_i$ and bias term $b_i$. After obtaining the demodulated query representation $\widetilde{q_i}$, the classification branch and regression branch are used to output the category distribution and bounding-box parameters, as formulated in Equation (22).

$$\widehat{p_i}=Soft\max (f_{cls}(\widetilde{q_i})),\widehat{b_i}=f_{box}(\widetilde{q_i}).$$

(22)

Compared with traditional one-stage detectors based on anchor boxes, the set prediction framework can significantly reduce redundant candidates and duplicate boxes, while maintaining more stable prediction behavior in densely clustered, heavily occluded, and overlapping target scenarios. At the same time, multi-scale cross-attention enables queries to adaptively select the most appropriate scale features for decoding, thereby improving localization quality for small targets and objects with ambiguous boundaries.

2.7. Multimodal Balanced Detection Loss (MBDL)

Close-range RGB-IR dual-modal fruit detection in orchards typically exhibits a long-tail class distribution and a high proportion of hard samples. The numbers of instances for different varieties or maturity levels differ significantly, while occlusion, strong illumination/shadow, and densely clustered distributions introduce considerable uncertainty in both localization and classification for some samples. When only standard detection losses such as BCE/FL and IoU-based regression are used, the optimization process is easily dominated by majority classes and easy samples, leading to insufficient gradient contribution from minority classes and hard samples.

To address this issue, this paper constructs a Multimodal Balanced Detection Loss (MBDL), which jointly optimizes classification reweighting, localization quality regression, and classification-localization consistency. The overall loss is defined as shown in Equation (23).

$${\mathcal{L}}_{MDBL}=\alpha {\mathcal{L}}_{cls}^{bal}+\beta {\mathcal{L}}_{box}^{PIoU-v2}+\gamma {\mathcal{L}}_{qc},$$

(23)

where α, β, and γ are branch weighting coefficients; ${\mathcal{L}}_{cls}^{bal}$ denotes the balanced classification loss for long-tailed data; ${\mathcal{L}}_{box}^{PIoU-v2}$ denotes the PioU v2 bounding-box regression loss adopted in this paper; and ${\mathcal{L}}_{qc}$ denotes the consistency constraint between classification confidence and localization quality.

2.7.1. Balanced Classification Loss

To alleviate class imbalance and insufficient focus on hard examples, both class-frequency weights and sample-difficulty weights are introduced for each positive sample. Let the category of the i-th positive sample be $c_i$, its predicted probability be $p_i$, and its label be $y_i\in \{0,1\}$. The class-frequency weight is defined as shown in Equation (24).

$$w^{cls}(c)={({\displaystyle \frac{N}{N_c+\epsilon }})}^{\eta },$$

(24)

where N denotes the total number of training instances, $N_c$ denotes the number of instances in category c, $\eta$ is the adjustment coefficient, and $\epsilon$ is a numerical stability term. Furthermore, to emphasize hard samples, such as those with severe occlusion, blurred boundaries, or obvious cross-modal discrepancies, a difficulty weight based on prediction confidence is introduced, as formulated in Equation (25).

$$w_i^{diff}={(1-p_i)}^{\delta },$$

(25)

where $\delta$ controls the emphasis placed on hard samples. The final balanced classification loss is therefore written as Equation (26).

$${\mathcal{L}}_{cls}^{bal}={\displaystyle \sum _{i\in {\mathsf{\Omega }}_{pos}}{w}^{cls}(c_i)}{\mathrm{w}}_i·{\mathcal{L}}_{BCE}(p_i,y_i),$$

(26)

where ${\mathsf{\Omega }}_{pos}$ denotes the set of positive samples, and ${\mathcal{L}}_{BCE}$ denotes the binary cross-entropy loss.

2.7.2. PioU v2 Localization Loss ${\mathcal{L}}_{box}^{PIoU-v2}$

In bounding-box regression, traditional IoU-series losses usually normalize penalty terms using the scale of the minimum enclosing box, which may induce anchor-box expansion and slow convergence. Powerful-IoU (PioU) constructs a scale-adaptive penalty factor based on the size of the ground-truth box and combines it with a gradient modulation function, guiding the predicted box to regress to the ground truth along a more direct path while avoiding unnecessary enlargement.

(1) Scale-adaptive penalty factor in PioU.

Let the predicted box be B = (x₁, y₁, x₂, y₂), and the ground-truth box be $B_{gt}=(x_1^{gt},y_1^{gt},x_2^{gt},y_2^{gt})$. The width and height of the ground-truth box are first defined as shown in Equation (27).

$$\begin{array}{cc}{w}_{gt}=x_2^{gt}-x_1^{gt},& h_{gt}=y_2^{gt}-y_1^{gt}.\end{array}$$

(27)

The distances between the four sides of the predicted box and the ground-truth box are then defined as in Equations (28) and (29).

$$d{w}_1=|x_1-x_1^{gt}|,d{w}_2=|x_2-x_2^{gt}|,$$

(28)

$$d{h}_1=|y_1-y_1^{gt}|,d{h}_2=|y_2-y_2^{gt}|.$$

(29)

Accordingly, the scale-adaptive penalty factor in PioU is defined as Equation (30).

$$P={\displaystyle \frac{1}{4}}({\displaystyle \frac{d{w}_1}{w_{gt}}},{\displaystyle \frac{d{w}_2}{w_{gt}}},{\displaystyle \frac{d{h}_1}{h_{gt}}},{\displaystyle \frac{d{h}_2}{h_{gt}}}).$$

(30)

This design makes the penalty term depend on the scale of the ground-truth box, thereby avoiding undesirable gradient directions caused by variations in the size of the enclosing box and fundamentally alleviating the anchor-box expansion problem.

(2) Gradient modulation function and PioU v1 loss.

To further modulate the optimization dynamics, a gradient modulation function is introduced as expressed in Equation (31).

$$f(P)=1-\exp (-P^2).$$

(31)

Based on the above penalty term, the PioU v1 loss is formulated as Equation (32).

$${\mathcal{L}}_{PIoU}={\mathcal{L}}_{IoU}+f(P)=(1-IoU)+(1-\exp (-P^2)),0\le {\mathcal{L}}_{PIoU}\le 2,$$

(32)

where IoU denotes the intersection-over-union between the predicted box and the ground-truth box.

(3) Non-monotonic attention layer and PioU v2 loss.

To enhance the emphasis on medium-quality samples, PioU v2 further introduces a non-monotonic attention layer controlled by the hyperparameter $\lambda$, as formulated in Equation (33).

$${\mathcal{L}}_{PIoU-v2}=u(\lambda q)·{\mathcal{L}}_{PIoU}=3(\lambda q)\exp (-{(\lambda q)}^2)·{\mathcal{L}}_{PIoU}.$$

(33)

Here, $\lambda$ controls the peak position and gradient distribution of the attention weighting. Accordingly, the localization branch in this work adopts the loss defined in Equation (34).

$${\mathcal{L}}_{cls}^{PIoU-v2}={\mathcal{L}}_{PLoU-v2}.$$

(34)

2.7.3. Classification-Localization Consistency Constraint

Under complex occlusion and background interference, detectors often produce predictions with high classification confidence but poor localization quality. To improve output reliability, MBDL introduces a consistency constraint between classification confidence and localization quality. For each positive sample, let $p_i$ denote the confidence score output by the classification branch, and let $Io{U}_i$ denote the localization quality. The consistency constraint is then defined as Equation (35).

$${\mathcal{L}}_{qc}={\displaystyle \sum _{i\in \mathsf{\Omega }pos}\left|\right|p_i-Io{U}_i|{|}_2^2}.$$

(35)

This term encourages the classification confidence and geometric matching quality to improve synchronously, thereby suppressing overconfident predictions at the global loss level and helping reduce false detections and duplicate boxes in densely clustered fruit scenarios.

3. Results

3.1. Experimental Setup and Evaluation Metrics

The experiments were conducted on the public RGB-IR fruit datasets described in Section 2.1. Baseline selection, comparative experiments, and ablation studies were performed on the grape dataset, whereas an additional-dataset evaluation was conducted on the tomato dataset to assess the applicability of DTNet to another fruit RGB-IR detection task. The following results are evaluated using Precision, Recall, F1-score, mAP@0.5, mAP@0.5:0.95, parameter count (Params), and floating-point operations (FLOPs).

3.1.1. Evaluation Metrics

To comprehensively evaluate the detection performance and deployment efficiency of the model, this paper adopts common metrics used in object detection tasks, including Precision, Recall, F1-score, mAP@0.5, mAP@0.5:0.95, and the model complexity indicators Params and FLOPs.

(1) Precision, Recall, and F1-score

Let true positives, false positives, and false negatives be denoted by TP, FP, and FN, respectively. Precision, Recall, and F1-score are then defined as formulated in Equations (36)–(38), respectively.

$$P={\displaystyle \frac{TP}{TP+FP}},$$

(36)

$$R={\displaystyle \frac{TP}{TP+FN}}.$$

(37)

$$F1={\displaystyle \frac{2PR}{P+R}}={\displaystyle \frac{2TP}{2TP+FP+FN}}.$$

(38)

Here, P measures the proportion of correctly predicted positive samples among all predicted positive samples, whereas R measures the proportion of true targets that are successfully detected by the model. The F1-score is the harmonic mean of Precision and Recall and can reflect the balance between false positives and false negatives more comprehensively.

(2) AP and mAP

At different confidence thresholds, the model yields a set of Precision-Recall curves. The Average Precision (AP) of a category is defined as the area under its Precision-Recall curve, as formulated in Equation (39).

$$AP={\displaystyle {\int }_0^{1}P(R)dR}.$$

(39)

By averaging the AP values over all categories, the mean Average Precision (mAP) is formulated as Equation (40).

$$mAP={\displaystyle \frac{1}{C}}{\displaystyle \sum _{i=1}^{C}A{P}_i}.$$

(40)

Here, C denotes the total number of categories. In this paper, two forms of mAP are mainly reported. mAP@0.5 is calculated with a fixed IoU threshold of 0.5, whereas mAP@0.5:0.95 is defined as the average mAP over IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05, as formulated in Equation (41).

$$mAP@0.5:0.95={\displaystyle \frac{1}{10}}{\displaystyle \sum _{k=0}^{9}mA{P}_{\Gamma =0.5+0.05k}}.$$

(41)

Compared with mAP@0.5, mAP@0.5:0.95 imposes stricter requirements on localization precision and can therefore reflect the overall detection quality more comprehensively.

(3) Model Complexity Metrics

In addition to accuracy-related metrics, model complexity is evaluated using the number of learnable parameters (Params) and floating-point operations (FLOPs). Params represents the total number of trainable parameters in the model, whereas FLOPs indicate the number of floating-point computations required during forward inference. These two metrics are used to measure the lightweightness and deployment potential of the model, thus providing a more comprehensive evaluation of the trade-off between accuracy and efficiency.

3.1.2. Implementation Details and Training Configuration

To ensure comparability among all experiments, all models are trained and evaluated under unified settings. The experiments are conducted on an NVIDIA GeForce RTX 4090 GPU (NVIDIA Corporation, Santa Clara, CA, USA) using PyTorch 2.3.0 (available online: https://pytorch.org/, accessed on 6 May 2026) with CUDA 12.1. The input image size is set to 512 × 512, the batch size is 8, and the number of training epochs is 200.

The optimizer is AdamW, with an initial learning rate of 0.001, an end-stage learning rate ratio of 0.01, and a weight decay coefficient of 0.01. To improve convergence stability in the early stage of training, a warmup strategy is adopted, with 3 warmup epochs, a warmup momentum of 0.8, and a warmup bias learning rate of 0.1. In addition, the momentum-related coefficient during training is set to 0.937. The random seed is set to 0, and the deterministic option is enabled to reduce the impact of randomness on experimental results.

During inference, the IoU threshold is set to 0.7, and the maximum number of detection boxes is set to 300. During training, Mosaic and MixUp are not enabled, while augmentation strategies such as random horizontal flipping, RandAugment, and random erasing are adopted to improve robustness against occlusion, local missing regions, and complex backgrounds. The early stopping patience is set to 100, and the best model weights are saved in real time based on validation set performance.

3.1.3. Edge-Side Deployment Evaluation on Raspberry Pi 5

To further address the reviewer’s concern regarding deployment efficiency on edge devices, an additional edge-side inference evaluation was carried out on a Raspberry Pi 5 platform (Raspberry Pi Ltd, Cambridge, UK; 8 GB RAM, official 64-bit operating system). The deployment test was implemented using a PyTorch-based inference script, and all models were executed in full precision (FP32) under the same software environment. The input image size was fixed at 512 × 512, consistent with the training and validation settings. The reported FPS and average latency per frame were measured under the same testing script after several warm-up iterations. The reported latency only reflects the runtime introduced by the object detection algorithm itself, i.e., the core model inference stage, and does not include image preprocessing, prediction postprocessing, one-time model loading, or program initialization.

On the Raspberry Pi 5 platform, YOLOv13 achieved 7.8 FPS, corresponding to an average latency of approximately 127 ms per frame. Under the same conditions, DTNet achieved 6.8 FPS, corresponding to an average latency of approximately 147 ms per frame. Compared with YOLOv13, DTNet introduces a moderate increase in inference time due to its dual-modal architecture, feature interaction mechanism, attention-based enhancement modules, and Transformer-based detection head. However, the observed decrease in speed is relatively limited, indicating that the proposed framework still maintains acceptable inference efficiency on a practical embedded platform.

It should be noted that the above latency values correspond to the runtime introduced by the object detection algorithm itself under the adopted inference pipeline, rather than end-to-end per-frame runtime. Therefore, the reported edge-side results are mainly intended to provide a controlled reference for the relative deployment efficiency of different models under a unified testing setting.

Considering that DTNet provides stronger detection accuracy and robustness in complex orchard environments, the above results demonstrate that the proposed model achieves a favorable balance between detection performance and deployment efficiency. Moreover, the model could run stably within the hardware resource constraints of the Raspberry Pi 5, suggesting that DTNet has practical potential for deployment in intelligent agricultural perception systems. Future work will further improve deployment efficiency through model compression, quantization, and hardware-oriented optimization.

3.2. Baseline Selection

To reasonably determine the backbone framework for the subsequent improved network, this study first compares several mainstream one-stage detectors as candidate baseline models. Considering that fruit detection in orchard scenarios requires not only high detection accuracy but also controllable model complexity and deployment feasibility, YOLOv8, YOLOv11, YOLOv12, and the published YOLOv13 model [42,43,44] were selected as candidate baseline models and compared under the same training settings. All models use identical data splits, input resolution, training epochs, and optimization strategies to ensure a fair comparison.

Table 1. Performance comparison of different single-modal baseline models on the grape dataset.

Model	Precision	Recall	mAP@0.5	mAP@0.5:0.95	Params	FLOPs
YOLOv8	0.8769	0.8024	0.8954	0.6598	2,690,598	6.9 G
YOLOv11	0.8963	0.8021	0.8810	0.6485	2,590,230	6.4 G
YOLOv12	0.8885	0.7837	0.8957	0.6540	2,538,486	6.0 G
YOLOv13	0.8760	0.7891	0.9031	0.7058	2,460,301	6.4 G

presents the quantitative comparison results of different single-modal baseline models on the grape dataset, Figure 4 shows the qualitative comparison of their prediction results, and Figure 5 illustrates the training convergence curves of the different baseline models. As shown in Table 1, the four models exhibit clear differences in Precision, Recall, mAP@0.5, mAP@0.5:0.95, and complexity-related metrics, indicating that different network architectures present different trade-offs between detection accuracy and efficiency in orchard scenarios.

From the perspective of detection accuracy, YOLOv11 achieves the highest Precision of 0.8963 among the four models, while YOLOv8 obtains a Recall of 0.8024, slightly higher than the 0.8021 of YOLOv11. YOLOv13 achieves the highest mAP@0.5 of 0.9031 among the four models and also reaches 0.7058 on mAP@0.5:0.95, which is significantly better than YOLOv8, YOLOv11, and YOLOv12. These results indicate that YOLOv13 not only achieves strong detection performance under a relatively loose IoU threshold, but also maintains more stable localization quality under stricter IoU thresholds, enabling it to provide higher-quality detection boxes in complex orchard scenarios and making it more suitable as the baseline framework for the subsequent dual-modal improvement network.

In terms of model complexity, YOLOv8 has 2,690,598 parameters and 6.9 G FLOPs; YOLOv11 has 2,590,230 parameters and 6.4 G FLOPs; YOLOv12 has 2,538,486 parameters and 6.0 G FLOPs; and YOLOv13 has 2,460,301 parameters and 6.4 G FLOPs. Overall, YOLOv13 has a relatively low parameter count among the four models while maintaining a moderate computational cost. Taken together with the accuracy metrics, these results suggest that YOLOv13 achieves a favorable balance between detection performance and model complexity.

From the qualitative visualization results in Figure 4, YOLOv8, YOLOv11, and YOLOv12 are prone to problems such as missed detections, insufficient separation of overlapping targets, and less precise bounding-box localization in some complex samples. By contrast, YOLOv13 shows more stable detection performance under conditions involving dense fruit clusters, small objects, and partial occlusion, with more accurate bounding-box localization, indicating stronger modeling capability for target boundaries and structural information. Meanwhile, the training convergence curves in Figure 5 show that YOLOv13 maintains good convergence stability during training and performs particularly well on the mAP@0.5:0.95 curve, further validating its advantage under strict localization metrics.

Considering the results in Table 1, Figure 4 and Figure 5, although YOLOv8 and YOLOv12 exhibit certain competitiveness on some metrics, YOLOv13 performs best on mAP@0.5:0.95, which better reflects the quality of detection boxes, while also maintaining lower model complexity and more stable training behavior. Therefore, YOLOv13 is ultimately selected as the baseline for the subsequent dual-modal improvement network, providing the baseline architecture for the subsequent introduction of modules such as DFB, LEM, ASA, the Transformer-based adaptive demodulation detection head, and MBDL.

3.3. Comparative Experiments and Result Analysis

To further validate the detection performance of DTNet in complex orchard scenarios, the proposed method is compared on the grape multimodal object detection dataset with several representative multimodal detection networks, including SDFMNet [45], ICANet [46], MSGANet [47], CGANet [48], MFMNet [49], and iRMBNet [50]. All comparison experiments adopt the same training settings and evaluation protocol as described previously, and uniformly report metrics such as Precision, Recall, mAP@0.5, mAP@0.5:0.95, Params, and FLOPs.

Quantitative Comparison with Mainstream Methods

Table 2. Performance comparison of different methods on the grape dataset.

Methods	Precision	Recall	mAP@0.5	mAP@0.5:0.95	Params	FLOPs
DTNet	0.9132	0.8931	0.9552	0.8001	4,141,053	11.9 G
SDFMNet [45]	0.9085	0.8425	0.9286	0.7408	4,038,653	10.7 G
ICANet [46]	0.9055	0.8372	0.9291	0.7310	6,908,449	11.4 G
MSGANet [47]	0.8789	0.8303	0.9180	0.7279	4,741,138	13.7 G
CGANet [48]	0.8953	0.8413	0.9276	0.7266	3,722,086	10.4 G
MFMNet [49]	0.9107	0.8375	0.9265	0.7223	2,486,525	8.5 G
iRMBNet [50]	0.8902	0.8444	0.9237	0.7328	4,096,509	11.8 G

presents the quantitative comparison results of different methods on the grape dataset. It can be seen that DTNet achieves 0.9132, 0.8931, 0.9552, and 0.8001 in Precision, Recall, mAP@0.5, and mAP@0.5:0.95, respectively, outperforming all comparison methods overall. In particular, DTNet achieves the highest mAP@0.5 among all methods, significantly surpassing SDFMNet, ICANet, MSGANet, CGANet, MFMNet, and iRMBNet. It also attains the highest Recall, indicating that DTNet can detect targets more completely in complex orchard scenarios and exhibits stronger robustness, especially for samples that are prone to missed detection.

Under the stricter metric of mAP@0.5:0.95, DTNet achieves a result of 0.8001, which is also higher than those of SDFMNet, ICANet, MSGANet, CGANet, MFMNet, and iRMBNet. This demonstrates the clear advantage of the proposed method in bounding-box localization accuracy. Although DTNet has 4,141,053 parameters and 11.9 G FLOPs, its overall complexity remains within an acceptable range and achieves a favorable balance between accuracy and complexity. In contrast, although MFMNet has lower parameter count and computational cost, its detection accuracy lags notably behind DTNet. MSGANet and iRMBNet have model complexity similar to or higher than DTNet, yet their Precision, Recall, and mAP metrics all fall short of the proposed method, indicating that DTNet achieves better overall performance while maintaining high detection accuracy.

The superior performance of DTNet can mainly be attributed to the following factors. First, DFB effectively promotes interaction and fusion between RGB and infrared features, allowing the model to obtain more stable discriminative information under complex illumination and background interference. Second, LEM strengthens local texture and boundary detail representation, helping improve the recognition of fruit contours and partially visible regions. Third, ASA enhances the aggregation of boundary and structural information during multi-scale fusion, thereby improving the distinguishability of targets at different scales. In addition, the Transformer-based adaptive demodulation detection head further improves the global relational modeling capability of candidate predictions, while MBDL enhances localization accuracy and training stability by improving the regression optimization process. Overall, DTNet demonstrates strong competitiveness in orchard multimodal object detection tasks. The qualitative prediction results and training convergence curves are shown in Figure 6 and Figure 7, respectively.

3.4. Ablation Study

To quantitatively analyze the contribution of each key module in DTNet to the overall performance improvement, ablation experiments are conducted on the grape multimodal object detection dataset. The ablation study uses the same training settings and evaluation protocol as the previous comparison experiments, namely an input resolution of 512 × 512, 200 training epochs, the AdamW optimizer, and unified evaluation metrics including Precision, Recall, mAP@0.5, mAP@0.5:0.95, Params, and FLOPs. Since the objective is to verify the performance gain brought by the stepwise introduction of each module, the ablation study takes single-modal YOLOv13 as the starting point and progressively analyzes the performance improvement from dual-modal fusion to the complete DTNet architecture. It should be noted that, because some later modules in DTNet also serve as lightweight structural substitutions rather than pure additive components, the model complexity in the progressive ablation is not necessarily monotonic with respect to module insertion.

Ablation Results

Table 3. Ablation study results of different module combinations and simple RGB–IR fusion baselines in DTNet on the grape multimodal object detection dataset.

No.	RGB Input	IR Input	+YOLOv13-MidFusion (Simple Addition-Based)	+DFB	+LEM	+ASA	+Transformer-Based Head	+MBDL	Precision	Recall	mAP@0.5	mAP@0.5:0.95	Params	FLOPs
1	✔								0.8760	0.7891	0.9030	0.7058	2,460,301	6.4 G
2		✔							0.8531	0.7642	0.8812	0.6898	2,460,301	6.4 G
3	✔	✔	✔						0.8833	0.7911	0.9117	0.7158	5,168,124	12.9 G
4	✔	✔		✔					0.9147	0.8567	0.9318	0.7412	11,139,217	22.9 G
5	✔	✔		✔	✔				0.9181	0.8715	0.9439	0.7759	3,782,645	10.7 G
6	✔	✔		✔	✔	✔			0.9169	0.8874	0.9520	0.7944	4,141,053	11.9 G
7	✔	✔		✔	✔	✔	✔		0.9140	0.8907	0.9537	0.7974	4,141,053	11.9 G
8	✔	✔		✔	✔	✔	✔	✔	0.9132	0.8931	0.9552	0.8001	4,141,053	11.9 G

Note: A check mark (✔) indicates that the corresponding module is included in the model configuration, while a blank cell indicates that the module is not used. The row “YOLOv13-MidFusion (simple addition-based)” denotes a simple non-adaptive RGB–IR mid-level fusion baseline without the proposed DFB.

reports the controlled settings and progressive module insertion results based on the YOLOv13 backbone. In addition to the progressive module insertion experiments, two supplementary control baselines, namely IR-only YOLOv13 and YOLOv13-MidFusion (simple addition-based), were included to better isolate the contribution of the dual-modal input itself and the effect of different fusion strategies. Here, YOLOv13-MidFusion denotes a simple non-adaptive RGB–IR mid-level fusion baseline, in which the paired modality features are fused by direct addition at the corresponding intermediate stage, without using the proposed DFB. Overall, the complete DTNet achieves the best overall performance, while the added control experiments further clarify that the performance gain comes not only from introducing an additional infrared modality, but also from the effectiveness of the proposed DFB and subsequent module designs.

As shown in the table, the RGB-only YOLOv13 baseline achieves Precision, Recall, mAP@0.5, and mAP@0.5:0.95 of 0.8760, 0.7891, 0.9030, and 0.7058, respectively. By contrast, the IR-only YOLOv13 baseline achieves 0.8531, 0.7642, 0.8812, and 0.6898 on the same metrics, indicating that infrared input alone can provide useful structural cues, but its overall discriminative ability remains weaker than that of RGB input alone for the current fruit detection task.

To further analyze the contribution of simple multimodal input, a YOLOv13-MidFusion (simple addition-based) baseline was constructed. This setting achieves Precision, Recall, mAP@0.5, and mAP@0.5:0.95 of 0.8833, 0.7911, 0.9117, and 0.7158, respectively, with 5,168,124 parameters and 12.9 G FLOPs. Compared with the RGB-only YOLOv13 baseline, this simple fusion setting shows consistent improvements on mAP@0.5 and mAP@0.5:0.95, suggesting that introducing an additional infrared modality is beneficial. However, the overall improvement remains limited, indicating that simple non-adaptive fusion cannot fully exploit the complementary information between RGB and IR modalities.

After introducing the proposed DFB, the model performance is further improved to 0.9147, 0.8567, 0.9318, and 0.7412 in Precision, Recall, mAP@0.5, and mAP@0.5:0.95, respectively. This demonstrates that, compared with the simple addition-based fusion baseline, DFB can more effectively enhance cross-modal interaction and representation learning. At the same time, the parameter count and computational cost increase substantially to 11,139,217 and 22.9 G, respectively, reflecting the additional overhead introduced by the dual-branch multimodal design and the more sophisticated fusion mechanism.

After further introducing LEM, the model’s Precision, Recall, mAP@0.5, and mAP@0.5:0.95 improve to 0.9181, 0.8715, 0.9439, and 0.7759, respectively. Meanwhile, the parameter count decreases to 3,782,645 and the FLOPs reduce to 10.7 G. This reduction does not contradict the cumulative ablation logic, because LEM is not merely an extra accuracy-oriented module; it also serves as a lightweight structural replacement at the corresponding feature-extraction stages. As described in Section 2.4, LEM adopts lightweight designs such as depthwise separable convolution, which enhance local detail representation while reducing parameter count and computational cost. Therefore, after LEM is introduced, the model complexity decreases even though the architecture becomes functionally richer. This indicates that LEM not only enhances local detail representation but also contributes to a more lightweight overall design.

On this basis, adding ASA further improves Precision, Recall, mAP@0.5, and mAP@0.5:0.95 to 0.9169, 0.8874, 0.9520, and 0.7944, respectively, demonstrating that ASA can effectively strengthen spatial information interaction among multi-scale features and improve the discriminability of dense targets and complex background regions. After replacing the original detection head with the Transformer-based adaptive demodulation detection head, Precision changes slightly, whereas Recall, mAP@0.5, and mAP@0.5:0.95 further improve to 0.8907, 0.9537, and 0.7974, respectively. This indicates that the proposed head enhances global modeling capability and localization accuracy without increasing overall model complexity.

Finally, after adding MBDL, the model achieves 0.9132, 0.8931, 0.9552, and 0.8001 in Precision, Recall, mAP@0.5, and mAP@0.5:0.95, respectively, yielding the best overall results in the ablation study. This demonstrates that MBDL can further coordinate the optimization process between the classification and localization branches, thereby improving the overall detection performance of the model in complex orchard scenarios. Overall, DFB, LEM, ASA, the Transformer-based adaptive demodulation detection head, and MBDL exhibit strong synergy and jointly promote the performance improvement of DTNet.

As shown in Table 3, the additional controlled experiments indicate that the performance gain of DTNet does not come solely from the introduction of an additional infrared modality, but also from the effectiveness of the proposed fusion strategy and subsequent architectural refinements. Compared with both the RGB-only baseline and the simple addition-based fusion baseline, the complete DTNet is significantly superior on both mAP@0.5 and mAP@0.5:0.95, demonstrating the effectiveness of the proposed dual-modal design in complex orchard scenarios.

3.5. Evaluation on an Additional Tomato RGB-IR Dataset

To further evaluate the applicability of DTNet on another fruit RGB-IR detection task, experiments were conducted on a public tomato RGB-IR dataset with different maturity stages under the same training and evaluation protocol used for the grape dataset. Although the tomato dataset was divided into training and validation subsets following an 8:2 split, it served as an additional dataset for assessing whether the proposed method could maintain stable performance on a fruit-detection task with different target morphology, maturity-related appearance variation, background texture, and occlusion patterns. The same evaluation metrics, including Precision, Recall, mAP@0.5, mAP@0.5:0.95, Params, and FLOPs, were adopted.

3.5.1. Results on the Tomato RGB-IR Dataset

Table 4. Performance of DTNet on different RGB-IR fruit datasets.

Dataset	Precision	Recall	mAP@0.5	mAP@0.5:0.95	Params	FLOPs
Grape dataset	0.9132	0.8931	0.9552	0.8001	4,141,053	11.9 G
Tomato dataset	0.9098	0.8843	0.9463	0.7863	4,141,053	11.9 G

presents the results of DTNet on the grape and tomato RGB-IR datasets. On the tomato dataset, DTNet achieved a Precision of 0.9098, a Recall of 0.8843, an mAP@0.5 of 0.9463, and an mAP@0.5:0.95 of 0.7863. These results indicate that the proposed model maintains stable detection performance on another fruit RGB-IR dataset with different category definitions and scene characteristics, demonstrating favorable adaptability to an additional fruit detection task.

Compared with the grape dataset, the mAP@0.5 of DTNet on the tomato dataset decreases from 0.9552 to 0.9463, a drop of 0.88 percentage points; mAP@0.5:0.95 decreases from 0.8001 to 0.7863, a drop of 1.38 percentage points; and Recall decreases from 0.8931 to 0.8843, only 0.88 percentage points. Overall, the performance degradation is relatively small, indicating that the proposed network architecture remains effective on another fruit RGB-IR dataset with different target categories and scene characteristics. The qualitative prediction results are shown in Figure 8.

3.5.2. Analysis of Results on the Additional Dataset

Compared with the grape dataset, the tomato dataset differs significantly in multiple aspects. First, the category definitions are different: the grape dataset mainly distinguishes categories by color, whereas the tomato dataset divides fruits into immature, half-ripe, and ripe according to maturity. Significant differences exist among these maturity levels in terms of color, texture, and surface reflectance characteristics. Second, the two crops also differ greatly in appearance, fruit size, clustered distribution pattern, and background structure, which may lead to feature distribution shifts when the model is applied across datasets. In addition, the branch and leaf structures, occlusion patterns, and local illumination conditions in tomato scenes are not fully consistent with those in grape scenes, which may also affect detection stability [51].

Despite these differences, DTNet is still able to achieve high Precision, Recall, and mAP values on the tomato dataset, indicating that the proposed multimodal fusion and feature enhancement mechanisms possess good adaptability to another fruit RGB-IR detection task.

Overall, the results on the tomato dataset indicate that DTNet is not only effective on the grape dataset but also maintains favorable performance on an additional fruit RGB-IR dataset. This suggests that the proposed framework has promising applicability in multimodal agricultural perception tasks involving different fruit categories and scene conditions.

3.6. Qualitative Visualization Comparison in Complex Scenarios

In orchard environments, fruit targets are often located in complex scenarios involving backlighting, occlusion, small-object scale, and strong background interference. These factors may weaken fruit surface texture information, distort brightness contrast, overexpose local regions, or cause blurred boundaries and missing features due to occlusion by leaves and branches, thereby significantly increasing the difficulty of object detection. To further verify the robustness of DTNet under such complex scenarios, three representative groups of samples are selected for qualitative visualization comparison, and DTNet is compared with the baseline single-modal YOLOv13.

As can be observed from the figure, single-modal YOLOv13 is easily affected by illumination variation, occlusion interference, and the scale limitation of small targets in complex scenarios. On the one hand, under backlighting or local strong illumination, fruit regions are prone to blurred boundaries and weakened textures, leading to shifted detection boxes or inaccurate localization. On the other hand, under leaf occlusion, densely clustered targets, and small fruit sizes, the response of the baseline method to some targets becomes weaker, making missed detections more likely. In particular, when the contrast between the fruit and background is low, or when the target is only partially visible, a single modality cannot provide sufficiently stable and complete discriminative information, resulting in less reliable detection results.

In contrast, DTNet exhibits more stable detection performance across all three groups of complex samples. For targets strongly affected by backlighting, fruits partially occluded by branches and leaves, and small-sized targets, DTNet can still localize fruit regions more accurately and effectively reduce missed detections and false detections. Its advantages are mainly reflected in the following aspects. First, the infrared modality can still preserve certain structural differences under complex illumination and occlusion, thus providing beneficial complementary information to the RGB modality. Second, DFB enhances the model’s discriminative capability in complex regions through cross-modal interaction and fusion. Third, LEM and ASA further improve the representation quality of boundaries and local details, enabling the model to maintain good target distinguishability even in scenarios involving backlighting, occlusion, and small objects. Therefore, the qualitative comparison under complex scenarios further verifies the robustness and detection stability of DTNet in real orchard environments. The qualitative comparison results are shown in Figure 9.

4. Discussion

The proposed DTNet demonstrates strong overall performance in dual-modal fruit object detection under complex orchard scenarios. Its core advantage lies in the coordinated optimization of dual-modal information complementarity, local detail enhancement, spatial saliency modeling, and a Transformer-based detection head, which collectively alleviate the insufficient robustness of traditional single-modal detection models under conditions of complex illumination, severe occlusion, and dense distributions of small targets. Compared with methods relying solely on RGB information, DTNet constructs a decoupled RGB-IR dual-branch backbone and introduces the DFB module at intermediate layers to achieve cross-modal feature interaction. Specifically, when visible-light images suffer from backlighting, local overexposure, deep shadows, or reduced chromatic contrast, the RGB branch may lose discriminative color and texture cues, causing fruit regions to become less salient against surrounding leaves and branches. In contrast, the infrared branch is less sensitive to these visible-light disturbances and can still preserve relatively stable intensity discontinuities and contour-level structural cues around the fruit body. For example, when a grape cluster appears visually merged with dark foliage or partially saturated background regions in the RGB image, the IR branch can still maintain clearer object-to-background separation and cluster-level contour information. By integrating these complementary cues, the DFB module enables the fused representation to retain fruit saliency even when the RGB branch becomes unreliable, thereby improving localization stability and reducing missed detections under complex illumination and occlusion conditions.

According to the ablation results, the introduction of DFB substantially improves detection performance over both the RGB-only baseline and the simple RGB-IR mid-level fusion baseline, confirming that the main gain does not come merely from adding an infrared modality, but from more effective cross-modal interaction. At the same time, DFB also introduces a clear efficiency cost: compared with the simple fusion setting, both Params and FLOPs increase substantially after DFB is introduced. This indicates that DFB is the main source of multimodal accuracy gain, but it is also the main source of additional computational overhead in the proposed framework. Therefore, the role of DFB should be understood as an explicit accuracy–efficiency trade-off: when stronger robustness and higher detection accuracy in complex orchard environments are prioritized, the use of DFB is justified; however, in extremely resource-constrained settings, simpler fusion schemes may still remain attractive if lower computational cost is the primary objective. After further introducing LEM, the model not only continues to improve in Precision, Recall, and mAP, but also shows a significant reduction in parameter count and FLOPs. This does not contradict the cumulative ablation logic, because LEM is not only an accuracy-oriented enhancement module but also a lightweight structural replacement that uses designs such as depthwise separable convolution. In this sense, the later modules partly compensate for the complexity increase introduced by DFB and help the final DTNet recover a more favorable balance between accuracy and efficiency.

In the quantitative comparison with mainstream methods, DTNet achieves high mAP@0.5 and Recall on the grape dataset, indicating strong overall competitiveness in complex orchard scenarios. In particular, the qualitative comparison in complex scenes shows that DTNet can still maintain stable detection performance under conditions such as backlighting, occlusion, and small-object targets, which indirectly confirms the importance of dual-modal fusion and local enhancement design for improving model robustness. In addition, the additional-dataset evaluation on the tomato dataset demonstrates that, although the target appearance, maturity distribution, and background characteristics differ significantly from those in the grape dataset, DTNet can still maintain high Precision, Recall, and mAP values. This suggests that the proposed method is applicable to another fruit RGB-IR detection task and has promising potential for multimodal agricultural perception under varying scene conditions.

At the same time, the likely sensitivity of DTNet to RGB-IR misalignment should be discussed more explicitly. The current framework, especially DFB, assumes that the paired RGB and IR inputs are reasonably well registered at the spatial level. In real agricultural environments, however, slight viewpoint deviation, camera vibration, sensor mounting error, rolling-shutter differences, or temporal desynchronization may introduce local cross-modal offsets. Under such conditions, pixel-wise adaptive weighting and cross-modal interaction may operate on partially mismatched regions, which can weaken object-to-background separation and reduce the reliability of fused contour cues, especially near fruit boundaries, occlusion regions, and densely clustered targets. Therefore, although the present results demonstrate the effectiveness of DTNet on registered RGB-IR datasets, the robustness of the current design to substantial modality misalignment should not be overstated. In practical deployment, DTNet is expected to perform best when the RGB and IR streams are carefully calibrated and synchronized, whereas noticeable registration errors may reduce part of the gain brought by DFB. Future work should therefore investigate more robust cross-modal alignment mechanisms, such as offset-aware fusion, feature-level registration compensation, or deformable cross-modal interaction strategies, to improve tolerance to registration errors in real orchard conditions.

Overall, DTNet achieves a relatively reasonable balance among detection accuracy, robustness, and deployment potential. The modules DFB, LEM, ASA, the Transformer-based adaptive demodulation detection head, and MBDL are highly complementary in function: DFB provides the basis for cross-modal information interaction, LEM balances detail enhancement and lightweight requirements, ASA strengthens multi-level feature fusion, the Transformer-based adaptive demodulation detection head enhances the global modeling capability of the detector, and MBDL further optimizes bounding-box regression and hard-sample learning. Nevertheless, several limitations remain. First, the current experimental validation is still based on relatively limited crop categories and dataset scales, which is insufficient to fully cover broader seasonal, varietal, and field-condition variability. Second, although edge-side tests on Raspberry Pi 5 indicate acceptable inference efficiency, the reported latency reflects the runtime introduced by the detection algorithm itself rather than full end-to-end pipeline cost, and further engineering optimization is still needed for practical field deployment. Third, although the final model complexity has been controlled to a reasonable level, the Transformer-based detection head may still introduce non-negligible overhead on more resource-constrained agricultural terminals. Future work may therefore focus on larger and more diverse RGB-IR agricultural datasets, more robust cross-modal alignment under real field perturbations, lighter global modeling units, model compression and quantized deployment, and tighter integration with downstream tasks such as maturity estimation, yield prediction, and robotic harvesting control.

5. Conclusions

To address common challenges in fruit detection in complex orchard environments, including severe occlusion, illumination variation, dense distributions of small targets, and the limited robustness of single-modal detectors, this paper proposes DTNet, an end-to-end RGB-IR dual-modal fruit detection network. Instead of depending on visible-light information alone, DTNet improves detection performance through a unified framework that combines cross-modal complementary perception, fine-grained feature enhancement, global relational modeling, and balanced optimization for hard samples. In this way, the proposed method strengthens target representation under complex illumination, occlusion, and dense-target conditions while maintaining a favorable balance between accuracy and model complexity.

Experimental results show that DTNet achieves a Precision of 0.9132, a Recall of 0.8931, an mAP@0.5 of 0.9552, and an mAP@0.5:0.95 of 0.8001 on the grape RGB-IR object detection dataset, outperforming multiple comparison methods overall. In the additional evaluation on the tomato RGB-IR dataset, the proposed model also maintains high detection accuracy, indicating favorable adaptability to another fruit RGB-IR detection task. The ablation study further confirms that the effectiveness of DTNet comes from the coordinated design of multimodal fusion, local detail enhancement, and robust optimization, while lightweight structural design helps control computational overhead.

Overall, DTNet provides an effective solution for RGB-IR dual-modal fruit object detection in orchard scenarios and shows promising potential for practical applications such as agricultural robot perception, automatic fruit harvesting, yield estimation, and intelligent orchard management. Future work will focus on improving robustness to cross-modal misalignment, enhancing lightweight deployment efficiency, extending validation to more crop categories and scenarios, and strengthening the integration between multimodal detection and downstream agronomic decision-making.