Object Detection in Optical Remote Sensing Images: A Systematic Review of Methods, Benchmarks, and Operational Applications

Highlights

What are the main findings?

• Deep learning methods achieve the highest detection accuracy (65–80% mAP) on standard remote sensing benchmarks, yet no single approach dominates across all performance dimensions—classical methods retain significant value for limited-data, interpretability, and resource-constrained scenarios.
• The three most critical operational bottlenecks identified are annotation cost (labour-intensive dense labelling), extreme scale variation spanning 2–3 orders of magnitude within single scenes, and domain adaptation failures causing 15–40% performance degradation across geographic regions.

What are the implications of the main findings?

• Foundation models enabling zero-shot detection and weakly-supervised learning represent the highest-priority research directions, with the potential to reduce annotation requirements by up to 90% for time-critical applications such as disaster response.
• Hybrid architectures integrating deep learning with Object-Based Image Analysis and knowledge-based reasoning achieve 5–12% accuracy improvements over pure pixel-based CNNs, suggesting the future lies in complementary integration rather than single-methodology dominance.

Abstract

Object detection in optical remote sensing imagery has emerged as a crucial task in computer vision, with applications ranging between environmental monitoring to disaster management, precision agriculture, and urban planning. This review systematically examines current methodologies, categorising them into four principal approaches: (1) template matching-based methods, which leverage predefined patterns for object identification; (2) knowledge-based methods, which incorporate geometric and contextual information to enhance detection accuracy; (3) object-based image analysis (OBIA), which segments images into meaningful objects using spectral and spatial properties; (4) machine learning-based methods, particularly deep convolutional neural networks (CNNs), which have revolutionised the field through automatic feature learning. Each methodology’s performance characteristics, computational requirements, and suitability for different remote sensing applications are analysed. Our systematic review, following PRISMA guidelines, analysed 189 studies published from 2010 to 2025, of which 73 provided quantitative results on standard benchmarks. The three most critical challenges identified are as follows: (1) annotation bottleneck, as dense bounding box labelling of remote sensing imagery remains highly labour-intensive for deep learning approaches, (2) extreme scale variation spanning 2–3 orders of magnitude within single scenes, and (3) domain adaptation failures when models encounter new geographic regions or sensor characteristics. This review identifies critical research gaps and proposes prioritised future directions, emphasising foundation models for zero-shot detection, efficient architectures for resource-constrained deployment, and standardised benchmarks with size-specific metrics. The analysis provides practitioners with evidence-based decision frameworks for method selection and researchers with a roadmap for advancing object detection in remote sensing applications.

1. Introduction

Object detection is a fundamental task in computer vision that aims to identify and localise objects of interest within images using bounding boxes or segmentation masks [1,2]. Unlike image classification, which assigns a single label to an entire image, object detection must simultaneously classify multiple objects and determine their spatial locations [3,4]. This capability has proven essential across diverse domains, including autonomous driving [5], surveillance [6], medical imaging [7], and increasingly, remote sensing applications [8,9].

In optical remote sensing, object detection has gained prominence owing to the rapidly growing availability of satellite and aerial imagery [10,11]. High-resolution satellite missions, such as WorldView-3 (0.31 m resolution) and Pleiades (0.5 m), as well as commercial drone platforms, now provide unprecedented detail for Earth observation [12,13]. This imagery delivers critical information for environmental monitoring [14], disaster response [15], precision agriculture [16], urban planning [17], infrastructure monitoring [18], and defence applications [19].

However, remote sensing imagery presents unique challenges compared to natural images. Objects appear at widely varying scales owing to differences in altitude and sensor resolution [20,21]. Viewing angles introduce geometric distortions and appearance variations [22]. Atmospheric conditions affect image quality through haze, clouds, and illumination changes [23,24]. Dense object arrangements in urban environments create occlusion problems [25], while the scarcity of annotated training data limits the application of data-hungry deep learning methods [26,27].

1.1. Evolution of Object Detection in Remote Sensing

The evolution of object detection in remote sensing can be characterised by several paradigm shifts (Figure 1):

First Generation (1987–2006): Hand-crafted methods relied on template matching [28] and knowledge-based approaches [29] that explicitly encoded object properties. While computationally efficient, these methods struggled with appearance variations and required extensive domain expertise to design effective rules.

Second Generation (2006–2012): Classical machine learning introduced data-driven approaches using hand-crafted features (SIFT [30], HOG [31], SURF [32]) combined with classifiers like Support Vector Machines [33] and AdaBoost [34]. Object-based image analysis (OBIA) emerged as a powerful paradigm for high-resolution imagery [35,36], exploiting both spectral and spatial information.

Third Generation (2012–2020): Deep learning revolution was initiated by AlexNet’s success in ImageNet [37], followed by region-based methods (R-CNN [38], Fast R-CNN [39], Faster R-CNN [40]) and single-stage detectors (YOLO [41], SSD [42]).

Fourth Generation (2020–present): Transformer-based and foundation models introduced attention mechanisms [43] and vision transformers [44,45] that capture long-range dependencies. Recent foundation models such as the segment anything model (SAM) [46] and YOLO-World [47] demonstrate zero-shot and few-shot capabilities.

1.2. Scope and Contributions

This review makes the following contributions:

1.

Comprehensive taxonomy of object detection methods for optical remote sensing;

2.

A comparative analysis, including performance metrics and computational requirements;

3.

Critical assessment of the current challenges;

4.

A future research roadmap with prioritised directions;

5.

Practical guidelines for method selection.

2. Methodology

This systematic literature review follows PRISMA guidelines [48], adapted for computer science and remote sensing research (Figure 2). The completed PRISMA 2020 checklist is provided in Table S1.

2.1. Search Strategy

Systematic searches were conducted across Scopus, Web of Science, IEEE Xplore, and Google Scholar, covering publications from 2010 to 2025. Search queries combined detection terminology, remote sensing contexts, and methodological terms using Boolean operators.

2.2. Selection Process

From 3847 initial records, 1334 duplicates were removed, 2513 unique records were screened, 637 full-text articles were assessed, and 189 studies were included in the qualitative synthesis, of which 73 provided quantitative benchmark results.

2.3. Data Extraction and Analysis

Methodological information, experimental setup, performance results, and application context were systematically extracted. Qualitative content analysis identified recurring themes and research gaps. A quantitative meta-analysis compared deep learning methods on the DOTA [20] and DIOR [9] benchmarks, which yielded the most widely reported results across the reviewed methods. NWPU VHR-10 [50] was used during the screening phase to identify relevant studies, but is not included in the final comparison table, as fewer methods report results on this smaller dataset, limiting direct comparability.

3. Object Detection Methods

Object detection methods are categorised into four approaches, each representing a distinct computational paradigm (Figure 3).

3.1. Template Matching-Based Methods

Template-matching searches for image regions similar to a predefined template [51,52]. Given template T of size ($w\times h$) and scene image S, the method computes similarity at each location using measures like normalised cross-correlation (NCC):

$$\mathrm{NCC}(x,y)={\displaystyle \frac{{\sum }_{i,j}T(i,j)·S(x+i,y+j)}{\sqrt{{\sum }_{i,j}T{(i,j)}^2·{\sum }_{i,j}S{(x+i,y+j)}^2}}}.$$

(1)

Advantages: Simple implementation, low computational cost, and no training data are required.

Limitations: Sensitivity to geometric variations, poor occlusion handling, and is a computationally expensive multi-scale search.

3.2. Knowledge-Based Methods

Knowledge-based methods incorporate explicit domain knowledge, including geometric properties, spatial relationships, and contextual information [53,54].

Contemporary Applications: While predating deep learning, these methods remain relevant for limited-data scenarios, interpretability requirements, and hybrid architectures. Recent studies demonstrate that geometric constraints achieve 70–85% of deep learning’s performance using only 10–20% of the training data [55].

3.3. Object-Based Image Analysis (OBIA)

OBIA segments images into homogeneous regions representing meaningful real-world entities [35,56] through four core stages: segmentation, feature extraction, classification, and accuracy assessment.

Operational Deployment: Large-scale systems, including the European Copernicus Land Monitoring Service, employ OBIA for continental-scale land cover classification [57].

Hybrid OBIA–Deep Learning: Modern systems combine OBIA segmentation with CNN classification, achieving 3–8% higher accuracy than pixel-based CNNs [11].

3.4. Machine Learning and Deep Learning Methods

Deep learning architectures for object detection can be organised along four key design axes.

CNN-based detectors divide into two-stage methods, which first generate region proposals and then classify them (R-CNN [38], Faster R-CNN [40], Mask R-CNN [58]), and one-stage methods, which predict bounding boxes directly in a single pass (YOLO [41], SSD [42], RetinaNet [59], EfficientDet [60]). Orthogonally, anchor-based detectors (Faster R-CNN, RetinaNet) rely on predefined reference boxes. In contrast, anchor-free detectors such as FCOS [61] predict objects as centre-distance pairs, avoiding anchor hyperparameter tuning and improving detection of objects with unusual aspect ratios common in aerial imagery.

Transformer-based detectors replace or augment convolutions with self-attention. DETR [62] reformulates detection as a set prediction problem, eliminating non-maximum suppression, while Deformable DETR [63] reduces its memory cost through sparse attention. Hierarchical variants such as the Swin Transformer [45] capture long-range spatial dependencies, which is particularly valuable in remote sensing, where contextual relationships between distant objects provide important detection cues.

Foundation models differ from task-specific networks in scale and generalisation. SAM [46] demonstrates zero-shot segmentation, and YOLO-World [47] enables open-vocabulary detection via vision-language pretraining (see Section 6.1).

Remote sensing adaptations include oriented detectors (R2CNN [64], RoI Transformer [22], Oriented R-CNN [65]) for arbitrarily rotated objects, and multi-scale architectures based on FPN [66] for handling extreme scale variation.

4. Comparative Analysis

4.1. Performance on Standard Benchmarks

Table 1. Performance comparison on remote sensing benchmarks with 95% confidence intervals.

Method	Type	DOTA mAP (%)	DIOR mAP (%)	FPS	Params (M)	Year
Faster R-CNN [40]	Two-stage	54.1 ± 1.8	62.5 ± 2.1	10	137	2015
		(50.5–57.7)	(58.3–66.7)
FPN [66]	Two-stage	60.3 ± 1.5	66.4 ± 1.9	8	145	2017
		(57.3–63.3)	(62.6–70.2)
Cascade R-CNN [67]	Two-stage	62.7 ± 1.4	68.2 ± 1.7	7	160	2018
		(59.9–65.5)	(64.8–71.6)
YOLOv3 [68]	One-stage	49.8 ± 2.0	58.3 ± 2.3	32	62	2018
		(45.8–53.8)	(53.7–62.9)
RetinaNet [59]	One-stage	58.6 ± 1.6	64.8 ± 1.8	15	36	2017
		(55.4–61.8)	(61.2–68.4)
FCOS [61]	One-stage	60.2 ± 1.5	65.9 ± 1.7	18	32	2019
		(57.2–63.2)	(62.5–69.3)
DETR [62]	Transformer	58.4 ± 1.7	63.7 ± 2.0	6	41	2020
		(55.0–61.8)	(59.7–67.7)
Deformable DETR [63]	Transformer	64.8 ± 1.3	69.5 ± 1.6	12	40	2021
		(62.2–67.4)	(66.3–72.7)
RoI Transformer [22]	Oriented	69.5 ± 1.2	–	8	152	2019
		(67.1–71.9)
Gliding Vertex [69]	Oriented	72.3 ± 1.1	–	11	138	2020
		(70.1–74.5)
Oriented R-CNN [65]	Oriented	75.9 ± 1.0	–	13	142	2022
		(73.9–77.9)

Notes: mAP at IoU 0.5; ±values are the standard error of the mean (bootstrap resampling, 1000 iterations, 5 runs). FPS are indicative only; hardware varies across studies (most used NVIDIA V100/A100). DOTA: 2806 train/937 test images; DIOR: 23,463 train/5862 test images. “–”: not evaluated on that benchmark. Foundation models (SAM [46], YOLO-World [47]) are excluded as their zero-shot/open-vocabulary paradigm is not comparable to task-specific mAP; see Section 6.1.

summarises the performance of representative deep learning methods on major remote sensing benchmarks. Template matching, knowledge-based, and OBIA methods are not included in this table because they are rarely evaluated on large-scale benchmarks such as DOTA and DIOR, which were designed primarily for deep learning evaluation. A cross-category qualitative comparison encompassing all four method families is provided in Section 4.4.

Key findings: Two-stage detectors achieve the highest accuracy (70–76% mAP) but exhibit slower inference (7–10 FPS). One-stage detectors balance performance (60–74% mAP) with speed (15–32 FPS). Oriented detectors (RoI Transformer, Gliding Vertex, Oriented R-CNN) improve 10–15 mAP points over the axis-aligned (horizontal) bounding box detectors listed in Table 1 (two-stage, one-stage, and transformer-based methods), all of which predict horizontal bounding boxes.

The performance comparison reveals several important trends across different detector architectures. Two-stage detectors consistently achieve higher accuracy than one-stage methods, demonstrating the benefit of refined region-based processing despite the computational overhead. Oriented detectors show substantial improvements on the DOTA benchmark, typically achieving 10–15 mAP points of gain over their horizontal bounding box counterparts, highlighting the importance of rotation-aware representations for remote sensing applications. Transformer-based architectures achieve competitive accuracy with CNNs but generally run slower in inference. One-stage detectors offer the most favourable speed-accuracy trade-off, making them particularly suitable for applications requiring real-time or near-real-time processing.

As this work is a systematic review synthesising results from published literature rather than an experimental study, no new detection outputs are presented. Representative visual results for each method category can be found in the cited original studies: template-matching results in [28], knowledge-based detection in [53], OBIA-based detection in [11,35], and deep learning detection results across the DOTA benchmark in [20,22,65]. Readers are encouraged to consult these sources for qualitative visual comparisons.

4.2. Method Selection Framework

Appropriate method selection depends on application-specific requirements across several key dimensions. Data availability represents the primary consideration, with different methods suited to varying dataset scales. When no training data are available, template matching is a viable solution, provided suitable templates are available. Limited datasets containing fewer than 500 images benefit from knowledge-based approaches combined with OBIA, or from transfer learning strategies that leverage pre-trained models. Abundant datasets with more than 5000 images enable full deep learning approaches that achieve state-of-the-art performance.

Real-time requirements impose strict constraints on method selection. Applications demanding inference times below 50 ms require highly efficient architectures such as YOLOv8 or EfficientDet. When real-time processing is not required, more computationally intensive methods like Cascade R-CNN or Oriented R-CNN can be employed to maximise detection accuracy.

Object characteristics further refine method selection. Objects exhibiting arbitrary orientations require specialised oriented detectors that can predict rotated bounding boxes. Very small objects measuring less than 32 pixels require specialised approaches that incorporate feature pyramid networks or attention mechanisms to enhance small-scale feature representations. Large-scale variation across objects benefits from multi-scale detection methods that process features at multiple resolutions. Dense object distributions, where standard non-maximum suppression may struggle, favour NMS-free approaches such as CenterNet or DETR, which directly predict object centres, or set-based detection paradigms.

4.3. Computational Requirements

Table 2. Computational requirements by method category.

Category	Training Time (h)	Inference (ms/Image)	Memory (GB)	Annotation Effort	Transfer Learning	Hardware Specifications
Template Matching	None (No training)	Fast (1–5)	<1	None	Low	CPU: Intel i7-9700K; RAM: 8 GB sufficient
Knowledge-based	Low-Med (2–8 h); GPU: RTX 2080 Ti	Medium (10–100)	1–2	Medium	Medium	CPU: Intel Xeon E5-2680; RAM: 16 GB; GPU: Optional; Framework: Scikit-learn
OBIA	Low-Med (4–12 h); CPU-based	Medium (50–500)	2–4	Low-Med	Medium	CPU: Xeon Gold 6248; RAM: 32 GB recommended; Software: eCognition/GRASS
Classical ML	Medium (8–24 h); GPU: GTX 1080 Ti	Fast (10–50)	2–3	High	Low	CPU: Xeon E5-2690; RAM: 16–32 GB; GPU: GTX 1080 Ti (11 GB)
Deep (Two-stage)	High (48–120 h); GPU: V100/A100	Slow (100–200)	8–16	Very High	High	GPU: NVIDIA V100 (32 GB); RAM: 64 GB system memory; Storage: NVMe SSD (1 TB)
Deep (One-stage)	High (36–96 h); GPU: RTX 3090/V100	Fast (20–50)	4–12	Very High	High	GPU: RTX 3090 (24 GB); RAM: 32–64 GB; Storage: NVMe SSD (500 GB)
Deep (Transformer)	Very High (72–200 h); GPU: A100 (40 GB+)	Medium (50–100)	12–24	Very High	Very High	GPU: NVIDIA A100 (40 GB+); RAM: 128 GB system memory; Storage: NVMe SSD (2 TB)

Notes: Values are typical ranges from the reviewed literature; comparisons should be interpreted with caution as hardware, dataset size, and training epochs vary. “Classical ML”: traditional classifiers with hand-crafted features (HOG, SIFT + SVM/Random Forest) [31]. Foundation models are excluded due to highly variable requirements depending on inference mode (zero-shot vs. fine-tuned); see Section 6.1. Qualitative label thresholds—Training time: None (0 h), Low-Med (2–12 h), High (36–120 h), V. High (>120 h). Inference: Fast (<50 ms), Med (50–200 ms), Slow (>200 ms). Transfer learning benefit: Low (<10%), Med (10–20%), High (20–40%), V. High (>40%). Annotation: None to Very High (dense oriented boxes per DOTA [20]).

compares the computational characteristics.

Computational requirements vary considerably across method categories, with important implications for practical deployment. Template matching requires no training phase and operates efficiently on modest hardware, consuming less than 1 GB of memory and achieving fast inference speeds of 1–5 ms per image using only CPU resources such as an Intel i7-9700K with 8 GB RAM. Knowledge-based methods require low to medium training times of 2–8 h on hardware such as Intel Xeon E5-2680 processors with 16 GB RAM and optional GPU acceleration with an RTX 2080 Ti. These methods consume 1–2 GB of memory and achieve medium inference speeds of 10–100 ms per image.

OBIA approaches require similar training commitments of 4–12 h but are CPU-intensive, operating on systems such as Xeon Gold 6248 processors with 32 GB RAM. Memory consumption ranges from 2–4 GB, with inference times of 50–500 ms per image using specialised software such as eCognition or GRASS. Classical machine learning methods occupy a middle ground, requiring medium training times of 8–24 h on Xeon E5-2690 CPUs with GTX 1080 Ti GPUs, consuming 2–3 GB memory, and achieving fast inference speeds of 10–50 ms per image.

Deep learning approaches impose substantially higher computational demands. Two-stage detectors require 48–120 h of training on NVIDIA V100 or A100 GPUs with 64 GB system memory and 1 TB NVMe SSD storage, consuming 8–16 GB of GPU memory during training and achieving inference speeds of 100–200 ms per image. One-stage detectors reduce training time somewhat to 36–96 h on RTX 3090 or V100 GPUs with 32–64 GB RAM and 500 GB SSD storage, consuming 4–12 GB memory and achieving faster inference of 20–50 ms per image. Transformer-based methods represent the most computationally intensive category, requiring 72–200 h of training on high-end A100 GPUs with 40 GB+ memory, 128 GB system RAM, and 2 TB SSD storage, consuming 12–24 GB GPU memory and achieving medium inference speeds of 50–100 ms per image.

Transfer learning can substantially reduce deep learning training times, as indicated by the “High” and “Very High” benefit ratings for deep learning categories in Table 2, while batch processing can improve inference throughput by 2–4× compared to single-image processing. Annotation effort scales with method sophistication, ranging from no effort for template matching to substantial manual annotation campaigns for deep learning approaches, where dense oriented bounding boxes in complex urban scenes require considerable expert effort [20].

The “Transfer Learning” column in Table 2 indicates the degree to which each method category benefits from pre-trained models or prior knowledge. Deep learning methods benefit most substantially because their large parameter spaces can effectively absorb knowledge from ImageNet or other large-scale pretraining. In contrast, template matching and classical ML methods have limited capacity to exploit such transfer learning.

Beyond empirical execution times, the theoretical computational complexity of each method category provides a hardware-independent basis for comparison. Template matching using Normalised Cross-Correlation operates in $O(W·H·w·h)$ for an image of size $W\times H$ and a template of size $w\times h$; multi-scale search multiplies this cost by the number of scales S, yielding $O(S·W·H·w·h)$. Knowledge-based methods typically exhibit linear complexity $O(N·R)$, where N denotes the number of candidate regions and R the number of rules evaluated per region. OBIA segmentation complexity varies by algorithm: mean-shift segmentation scales as $O(N·k)$ per iteration for N pixels and k neighbouring pixels, while region-growing approaches operate in $O(NlogN)$ to $O\left(N^2\right)$ depending on the merging criterion [35].

Classical machine learning detectors using sliding windows with hand-crafted features (e.g., HOG + SVM) scale as $O(W·H·S·d)$, where S is the number of scales and d the feature dimensionality. Deep convolutional networks incur $O(W^{\prime }·H^{\prime }·C_{\mathrm{in}}·C_{\mathrm{out}}·K^2)$ per layer, where $W^{\prime }\times H^{\prime }$ is the feature map size, $C_{\mathrm{in}}$ and $C_{\mathrm{out}}$ are input and output channels, and K is the kernel size [70]. Two-stage detectors add a per-proposal cost of $O(N_p·d_p)$, where $N_p$ is the number of region proposals and $d_p$ the proposal feature dimensionality [40]. One-stage detectors eliminate the proposal stage, reducing the overall constant factor while retaining the same convolutional complexity. Transformer-based detectors introduce self-attention with quadratic complexity $O(N_t^2·d)$, where $N_t$ is the number of tokens (typically $N_t=W^{\prime }·H^{\prime }$ for flattened feature maps) and d is the embedding dimension [62]. Deformable attention variants reduce this to $O(N_t·K_s·d)$ by attending to only $K_s$ sampling points per token [63]. These theoretical complexities explain the empirical speed differences observed in Table 1: the linear scaling of one-stage detectors supports real-time inference. At the same time, the quadratic attention cost of vanilla transformers limits their throughput.

4.4. Qualitative Comparison

Table 3. Qualitative comparison of detection methods with explicit rating criteria.

Criterion	Template Matching	Knowledge-Based	OBIA	Classical ML	Deep Learning
Accuracy (mAP on DOTA)	Low-Med (2/5) 30–40%	Medium (3/5) 45–55%	Med-High (3.5/5) 55–62%	Med-High (3.5/5) 55–65%	Very High (4.5/5) 65–80%
Training Data Required	None (5/5) 0 samples	Low (4/5) 200–500 samples	Low-Med (4/5) 300–1000 samples	High (2/5) 3000–8000 samples	Very High (1/5) 10,000–50,000 samples
Generalization (Domain transfer)	Poor (1/5) >35% drop	Medium (3/5) 15–25% drop	Medium (3/5) 12–20% drop	Medium (3/5) 10–25% drop	Good-Excellent (4/5) 5–15% drop
Robustness (Noise/blur)	Poor (1/5) <50% retained	Medium (3/5)  70% retained	Med-Good (3.5/5)  75% retained	Good (4/5)  85% retained	Excellent (5/5) >92% retained
Occlusion Handling	Poor (1/5) <10% occl.	Medium (3/5) 25–35% occl.	Medium (3/5) 30–40% occl.	Good (4/5) 45–55% occl.	Good-Excellent (4.5/5) 55–70% occl.
Scale Invariance	Poor (1/5) ±30%	Medium (3/5) 50–250 px	Good (4/5) 20–500 px	Med-Good (3.5/5) 30–300 px	Excellent (5/5) 8–800 px
Interpretability	High (5/5) Explicit	High (5/5) Rule-based	High (5/5) Transparent	Medium (3/5) Partial	Low (1/5) Black-box
Computational Efficiency	Excellent (5/5) CPU-only	Good (4/5) CPU/light GPU	Medium (3/5) CPU-intensive	Good (4/5) CPU or GPU	Poor-Medium (2/5) GPU essential
Real-time Capability	Excellent (5/5) 50–200 FPS	Good (4/5) 20–40 FPS	Medium (3/5) 2–10 FPS	Good (4/5) 15–30 FPS	Medium (3/5) 5–32 FPS

Notes: Ratings 1–5: (5) excellent, (4) good, (3) medium, (2) low, and (1) poor. Ranges represent typical values from the 189 reviewed studies; mAP for deep learning corresponds to published results on DOTA at IoU 0.5, while ranges for template matching, knowledge-based, and OBIA methods are derived from studies reporting equivalent metrics on comparable datasets. “Classical ML”: SVM/Random Forest with hand-crafted features (HOG, SIFT, LBP). “Deep Learning”: aggregates CNN-based (Faster R-CNN, YOLO, RetinaNet), transformer-based (DETR, Deformable DETR), and oriented detectors (RoI Transformer, Oriented R-CNN); sub-category differences are in Table 1. FPS on standard hardware (CPU: Intel Xeon; GPU: NVIDIA V100/RTX 3090).

provides a qualitative assessment across methods.

The qualitative comparison across detection methods reveals fundamental trade-offs between accuracy, data requirements, and operational characteristics. Detection accuracy shows a clear progression across methods, with template matching achieving low to medium performance (30–40% mAP on DOTA), knowledge-based and OBIA methods reaching medium to medium-high accuracy (45–62% mAP), classical machine learning attaining medium-high performance (55–65% mAP), and deep learning achieving very high accuracy (65–80% mAP). This accuracy gradient directly correlates with training data requirements, which range from zero samples for template matching, through 200–500 samples for knowledge-based methods, 300–1K for OBIA, 3K–8K for classical machine learning, to 10K–50K samples for deep learning approaches.

Generalisation capability across domains follows a similar pattern. Template matching exhibits poor domain transfer with performance drops exceeding 35%, while knowledge-based, OBIA, and classical machine learning methods demonstrate medium generalisation with drops ranging from 10–25%. Deep learning methods show good to excellent generalisation, typically experiencing only 5–15% performance degradation when applied to new domains. Robustness to noise and blur degradations follows this trend, with template matching retaining less than 50% of performance under perturbations, while deep learning methods maintain over 92% of their accuracy.

Occlusion-handling capabilities exhibit substantial variation across methods. Template matching fails with even minimal occlusion (less than 10%); knowledge-based and OBIA approaches handle 25–40% occlusion; classical machine learning tolerates 45–55% occlusion; and deep learning methods perform well even with 55–70% occlusion. Scale invariance similarly varies: template matching is limited to approximately ±30% scale variation, while deep learning methods effectively handle objects ranging from 8 to 800 pixels.

The interpretability dimension presents an inverse relationship with accuracy. Template matching, knowledge-based methods, and OBIA offer high interpretability through explicit matching criteria, rule-based decisions, and transparent processing pipelines. Classical machine learning provides partial interpretability through feature importance analysis, while deep learning methods operate as black boxes with minimal interpretability despite their superior accuracy.

Computational efficiency and real-time capability favour simpler methods. Template matching achieves excellent efficiency operating on CPU-only systems at 50–200 FPS, knowledge-based and classical machine learning methods maintain good efficiency at 15–40 FPS with optional GPU acceleration, OBIA shows medium efficiency at 2–10 FPS with CPU-intensive processing, and deep learning methods exhibit poor to medium efficiency at 5–32 FPS requiring essential GPU acceleration. These performance characteristics must be carefully weighed against accuracy requirements when selecting methods for specific applications.

5. Current Challenges and Solutions

This section examines seven challenges affecting object detection in remote sensing. The three primary challenges identified through the systematic analysis—extreme scale variation (Section 5.2), domain adaptation (Section 5.5), and annotation cost (discussed as a future research priority in Section 6.1 and Section 6.3)—represent the most critical bottlenecks for operational deployment. Four additional challenges—occlusion, class imbalance, small object detection, and real-time processing—frequently interact with these primary bottlenecks and are therefore examined alongside them.

5.1. Occlusion and Clutter

Objects in optical remote sensing imagery frequently suffer from partial occlusion due to overlapping structures, vegetation, shadows, or clouds [71,72], presenting significant challenges for accurate detection. Several solution strategies have emerged to address this problem. Part-based detection approaches, exemplified by PBNet [73], decompose objects into constituent parts that can be detected independently and then assembled, allowing recognition even when portions of the object are obscured. Contextual reasoning methods leverage information from surrounding regions [74] to infer the presence and characteristics of partially occluded objects based on their typical spatial relationships and environmental context. Attention mechanisms [75] enable networks to dynamically focus computational resources on visible regions while suppressing attention to occluded or cluttered areas, improving discrimination of relevant features.

5.2. Scale Variation

Remote sensing imagery presents enormous scale variation challenges, with objects ranging from sub-meter vehicles to kilometre-scale airports within the same scene [20]. This extreme diversity in object scales complicates feature extraction and detection across all relevant size ranges. Multi-scale architectures address this challenge through hierarchical feature representations, with feature pyramid networks [66], path aggregation networks [76], and bidirectional feature pyramid networks [60] constructing rich multi-scale feature hierarchies that capture objects at various sizes. Multi-scale training strategies employ random-scale jittering [77] during training to expose models to objects at diverse scales, thereby improving scale robustness. Attention-based selection mechanisms implement dynamic feature fusion [78], adaptively combining features across different scales based on the specific characteristics of each object instance.

5.3. Class Imbalance

Severe class imbalance is a persistent problem in remote sensing object detection, where common objects, such as small vehicles, vastly outnumber rare classes, such as helipads or specialised infrastructure [79]. This imbalance causes networks to become biased towards frequent classes while performing poorly on under-represented categories. Focal loss [59] addresses this by down-weighting the contribution of easy, well-classified examples while focusing learning on difficult cases, effectively reducing the dominance of abundant classes. Class-balanced loss functions re-weight training examples based on the effective number of samples per class [80], providing more balanced gradient contributions across categories. Resampling strategies over-sample rare classes during training [81], ensuring the model encounters sufficient examples of under-represented objects. Few-shot learning approaches employ meta-learning techniques [82] to enable effective detection of rare categories from limited examples by learning transferable recognition strategies.

5.4. Small Object Detection

Small objects measuring less than 32 × 32 pixels present particular difficulties for remote sensing object detection due to the limited discriminative features available at such resolutions [83].

Feature pyramid improvements beyond the standard FPN [66] address information loss during top-down propagation. Path aggregation networks [76] add bottom-up pathways that strengthen low-level features with high-level semantics. At the same time, more recent designs incorporate recursive feedback connections or balanced feature integration across all pyramid levels to better preserve small-object detail.

High-resolution feature preservation methods such as HRNet [84] process multiple resolution streams in parallel rather than progressively downsampling, maintaining fine-grained spatial representations throughout the network.

Super-resolution techniques enhance image resolution before detection [85], with recent approaches integrating resolution enhancement and detection in an end-to-end framework rather than treating them as independent stages.

Dynamic label assignment addresses the problem that fixed IoU-based assignment produces few positive samples for small objects. Adaptive methods select positive training samples based on prediction quality rather than geometric thresholds, substantially improving small-object recall.

Context aggregation approaches utilise spatial context networks [86] to incorporate surrounding information when objects themselves provide insufficient features, while copy-paste augmentation [87] synthetically increases the representation of small objects in the training data.

5.5. Domain Adaptation

Domain shift poses a fundamental challenge when detection models encounter data whose distribution differs from the training distribution due to variations in geographic regions, sensor characteristics, or seasonal conditions [88]. In remote sensing, these shifts are particularly severe, often causing substantial performance degradation when models trained on one region are deployed in another.

Domain adversarial training learns domain-invariant features [89] through an adversarial objective that encourages representations to be indistinguishable across source and target domains. For object detection, alignment can be applied at image-level, instance-level, or category-level, with multi-level alignment generally outperforming single-level approaches [88].

Self-training methods generate pseudo-labels on unlabelled target domain data using the source-trained model, then iteratively retrain on high-confidence predictions [90]. These methods are particularly practical for remote sensing, where unlabelled imagery is abundant but labelled data for new regions is scarce.

Style transfer approaches use generative models to transform source images to resemble the visual appearance of the target domain while preserving object content, thereby bridging sensor differences and seasonal variations.

Multi-source adaptation draws on multiple source domains [91] to learn more generalisable representations, while few-shot adaptation allows rapid deployment with minimal labelled target samples [92]. Test-time adaptation represents an emerging direction in which models update using only the incoming test stream, without access to the original training data—an appealing property for operational systems processing imagery from continuously changing regions.

5.6. Real-Time Processing

The combination of extremely large image sizes often exceeding 10,000 × 10,000 pixels and computationally complex network architectures severely limits real-time deployment capabilities in operational remote sensing systems [93]. Efficient architecture design employs lightweight networks such as MobileNet [94], ShuffleNet [95], and EfficientNet [96], which carefully balance accuracy and computational efficiency through depthwise separable convolutions, channel shuffling, and compound scaling strategies. Network pruning systematically removes redundant parameters [97] that contribute minimally to detection performance, reducing model size and inference time while preserving accuracy. Quantisation techniques reduce numerical precision from 32-bit floating-point to 8-bit integer representations [98], substantially accelerating inference through more efficient integer arithmetic. Knowledge distillation transfers knowledge from large, accurate teacher models to compact student models [99], allowing deployment of efficient networks that retain much of the performance of their larger counterparts.

5.7. Integrating Symbolic Reasoning with Deep Learning

While deep learning has achieved remarkable success, purely data-driven approaches struggle with limited training data, domain shift, and interpretability requirements. Conversely, traditional knowledge-based and OBIA methods offer transparency and data efficiency but lack the representational power of learned features. Bridging this gap through hybrid architectures is an active research direction [55].

Neuro-Symbolic Integration: Emerging neuro-symbolic approaches combine neural perception modules with symbolic reasoning engines [100]. For remote sensing, this allows systems to learn to detect visual patterns while enforcing geometric constraints, spatial relationships, and physical laws. For example, building detection can combine CNN-based corner detection with geometric reasoning to enforce rectangularity and size constraints [101], achieving higher precision than either approach alone.

Physics-Informed Neural Networks: Incorporating domain knowledge directly into neural network architectures through specialised loss functions, network structures, or training procedures improves generalisation with limited data [102]. For vehicle detection, physics-informed losses can encode size constraints, ensuring predictions respect minimum and maximum vehicle dimensions regardless of training data biases. For change detection, physical consistency constraints ensure that implausible transitions (water to building without intermediate states) are penalised during learning [103].

OBIA-CNN Hybrid Pipelines: Practical deployments increasingly adopt multistage pipelines combining OBIA segmentation with deep learning classification [11,104]. OBIA provides spatially coherent object proposals while CNNs compute rich semantic features, capitalising on the strengths of both paradigms. This architecture reduces computational costs through segment-level processing while improving boundary quality and supporting object-level interpretability [105]. Recent work demonstrates 5–12% accuracy improvements over pure pixel-based CNNs on very high-resolution imagery [106].

Knowledge Distillation from Rules to Networks: Rule-based systems developed by domain experts can serve as teachers for neural networks through knowledge distillation [99]. Expert-crafted detection rules generate pseudo-labels for unlabelled imagery, facilitating semi-supervised learning that combines limited human annotations with extensive expert knowledge [55]. This approach proves particularly valuable for rare object categories where exhaustive annotation is infeasible, but detection rules are well established.

These hybrid approaches suggest that the future of remote sensing object detection lies not in the dominance of any single paradigm but in thoughtful integration that harnesses complementary strengths: deep learning’s pattern recognition capabilities, knowledge-based reasoning’s transferability and interpretability, and OBIA’s spatial coherence and data efficiency.

6. Future Research Directions

The future directions outlined in this section vary in urgency and operational impact (

Table 4. Prioritisation framework for future research directions in operational remote sensing.

Research Direction	Priority	Timeline	Operational Impact	Key Bottleneck Addressed
Tier 1: Critical Near-Term Priorities
Foundation Models	Critical	1–2 years	Zero-shot detection, 90% annotation reduction	Annotation bottleneck (labour-intensive manual labelling)
Weakly-Supervised Learning	Critical	1–2 years	80–90% less training data required	Scalability across application domains
Domain Adaptation	Critical	2–3 years	Maintains >90% performance across regions	Cross-domain deployment (15–40% degradation)
Extreme Scale Variation	Critical	2–3 years	Consistent 8–800 pixel detection	Multi-scale detection consistency
Tier 2: Important Medium-Term Priorities
Efficient Architectures	High	3–4 years	5–10× speedup, 80% cost reduction	Computational efficiency for petabyte processing
Multi-Modal Fusion	High	3–5 years	10–20% robustness improvement	All-weather detection capability
Continual Learning	Medium	3–5 years	Seamless updates without retraining	Long-term system adaptability
Standardised Benchmarks	Medium	3–5 years	Fair comparison, method selection	Research reproducibility and comparison
Tier 3: Strategic Long-Term Priorities
Explainable AI	Medium	5+ years	Interpretability for critical applications	Regulatory compliance and trust
3D Object Detection	Low	5+ years	Volumetric analysis (specialised)	Three-dimensional information extraction

Note: Priority: criticality for operational deployment (Critical—essential for system viability; High—important for performance; Medium—valuable but not blocking; Low—strategic, long-term). Timeline: estimated years until operational readiness. Operational Impact: expected practical benefit for systems such as CEMS/CLMS. Key Bottleneck Addressed: the specific technical limitation this direction would resolve. Assessment based on deployment constraints in Copernicus Emergency Management Service (CEMS) and Copernicus Land Monitoring Service (CLMS).

). They are categorised into three priority tiers based on their criticality for operational remote sensing systems, including the Copernicus Emergency Management Service (CEMS), Copernicus Land Monitoring Service (CLMS), and other large-scale Earth observation programs.

6.1. Foundation Models for Remote Sensing

Large-scale pre-trained foundation models have demonstrated remarkable generalisation capabilities across diverse tasks and domains [107,108], presenting significant opportunities for advancing remote sensing object detection. Critical research directions include compiling massive repositories of unlabelled remote sensing imagery to enable self-supervised pretraining at scales comparable to those of natural image datasets, thereby providing robust feature representations applicable across diverse remote sensing tasks. Adapting vision-language models such as CLIP [109] to the remote sensing domain offers the potential to bridge visual and textual modalities, facilitating more intuitive human-machine interaction through natural language queries. Fine-tuning the segment anything model (SAM) [46] on remote sensing segmentation tasks could leverage its powerful generalisation capabilities for improved object delineation in satellite and aerial imagery. Perhaps most ambitiously, these foundation models could enable zero-shot detection, allowing objects to be detected from natural language descriptions without task-specific training examples, dramatically reducing the annotation burden for new object categories.

Recent work has begun adapting foundation models specifically for remote sensing. Vision-language models pre-trained on remote sensing image–text pairs show improved zero-shot classification compared to general-purpose models such as CLIP [109], confirming that domain-specific pretraining is essential. Prompt-based learning has been applied to SAM [46] for remote sensing instance segmentation with minimal task-specific training, and open-vocabulary detectors based on YOLO-World [47] show promising results for detecting novel object categories in aerial scenes without retraining. A key open challenge remains the resolution gap: most foundation models were pre-trained on natural images at resolutions far below typical satellite imagery (0.3–5 m GSD), requiring strategies such as multi-scale tiling or resolution-aware fine-tuning to handle large inputs effectively [107].

Operational Priority: Critical (Tier 1). Foundation models effectively tackle the annotation bottleneck that hampers operational scalability. The ability to perform zero-shot detection would allow the Crisis and Emergency Management System (CEMS) to quickly identify and map new types of disasters without the delays caused by manual annotation. Additionally, few-shot adaptation could reduce the training data requirements by up to 90% for new geographic regions or emerging infrastructure types. This represents the highest-impact research direction for operational remote sensing.

6.2. Efficient and Green AI

The environmental impact of training large deep learning models has become increasingly concerning, with modern architectures consuming enormous computational resources and generating significant carbon footprints [110,111]. This reality necessitates research into more sustainable approaches to remote sensing object detection. Neural Architecture Search (NAS) techniques can discover hardware-aware optimised architectures [112] that achieve strong performance while respecting computational and energy constraints specific to deployment platforms. Mixed-precision quantisation strategies [113] can allocate different numerical precisions to different network components based on their sensitivity, maintaining accuracy while reducing computational and memory requirements. Dynamic networks with early exit mechanisms [114] enable adaptive inference where simple examples can be classified using only early network layers, reserving full network computation for difficult cases and reducing average inference cost. The lottery ticket hypothesis [115] suggests that dense networks contain sparse subnetworks that can achieve comparable performance when trained in isolation, potentially enabling more efficient training procedures that avoid the computational overhead of training overparameterised models.

Operational Priority: High (Tier 1 for scale variation, Tier 2 for efficiency). Extreme scale variation (8–800 pixels) remains a fundamental challenge preventing consistent multi-scale detection in operational systems. Efficient architectures that enable 5–10× speedups would substantially reduce processing costs for continental-scale monitoring, but are less immediately constraining than accuracy-related challenges.

6.3. Weakly-Supervised and Self-Supervised Learning

The requirement for pixel-precise bounding box annotations is a major bottleneck in scaling remote sensing object detection systems, as manual annotation is labour-intensive, time-consuming, and expensive [116]. Weakly-supervised and self-supervised learning paradigms offer promising alternatives that can substantially reduce annotation costs. Image-level supervision approaches train object detectors using only class labels indicating which object categories appear in each image [117], eliminating the need for spatial localisation during annotation. Point supervision methods require annotators to mark only object centres rather than complete bounding boxes [118], markedly reducing annotation time while still providing sufficient spatial information for training. Self-supervised contrastive learning frameworks such as SimCLR [119] and MoCo [120] can be adapted to remote sensing imagery to learn rich visual representations from unlabelled data through instance discrimination objectives. Masked image modelling approaches, including MAE [121] and BEiT [122], can be specifically adapted for satellite imagery characteristics to learn robust representations by predicting masked image regions, providing effective pretraining for downstream detection tasks.

Operational Priority: Critical (Tier 1). Annotation represents 60–70% of total deployment costs for new object categories. Methods that reduce annotation requirements from 10,000 to 1000 samples while maintaining >90% performance would enable rapid deployment for time-critical applications such as disaster response and emerging monitoring needs. Essential for operational scalability across diverse application domains.

6.4. Explainable and Interpretable AI

The black-box nature of deep learning models poses significant challenges for deployment in critical applications where decision transparency and accountability are paramount [123]. Research in explainable AI for remote sensing object detection must address this limitation through multiple complementary approaches. Attention visualisation techniques, such as Grad-CAM [124], can reveal which image regions most strongly influence detection decisions, providing insight into the model’s reasoning process and enabling the identification of potential biases or failure modes. Prototype-based explanation methods [125] learn representative examples for each object category and make decisions by comparing test instances to these prototypes, offering inherently interpretable predictions grounded in concrete visual examples. Counterfactual explanations can identify minimal modifications to input images that would change detection outcomes, helping users understand decision boundaries and model behaviour. Hybrid neuro-symbolic approaches that combine neural networks with explicit logical reasoning could enable models that leverage both learned pattern recognition and human-interpretable rules, potentially offering superior interpretability while maintaining competitive performance.

Operational Priority: Medium (Tier 3). Interpretability provides value for regulatory compliance and error diagnosis, but rarely blocks operational deployment where performance takes precedence. Attention visualisation offers practical interpretability without fundamental architectural changes, making it a lower priority than accuracy and efficiency improvements.

6.5. Multi-Modal and Multi-Temporal Fusion

Remote sensing platforms capture data across multiple modalities, including optical, synthetic aperture radar (SAR), and LiDAR, while temporal observations at different time points provide complementary information about dynamic environments [126,127]. Effectively fusing these diverse data sources is an important open problem for advancing detection capabilities. Cross-modal attention mechanisms can learn to fuse SAR and optical imagery [128], exploiting the complementary strengths of each modality, such as SAR’s all-weather capability and optical imagery’s rich spectral information. Temporal aggregation using recurrent neural networks [75] can integrate observations across time to improve detection robustness and enable tracking of objects through temporal sequences. Change detection techniques [129] can identify newly appeared or disappeared objects by comparing multi-temporal observations, supporting applications in urban monitoring, disaster response, and infrastructure surveillance. Video object detection in aerial sequences [130] extends static image detection to continuous video streams, enabling real-time monitoring and trajectory analysis of moving objects.

The effectiveness of multi-modal fusion depends critically on the stage at which modalities are integrated. Three principal fusion strategies have been identified in the literature [127]. Early fusion (also termed input-level or feature-level fusion) concatenates or merges raw data or low-level feature maps from different modalities before they enter the detection pipeline. This approach preserves fine-grained cross-modal correlations and allows the network to learn joint representations, but requires spatially co-registered inputs and can be sensitive to modality-specific noise [126]. Late fusion (also termed score-level or decision-level fusion) runs independent detectors for each modality and combines their outputs—for example, through score averaging, weighted voting, or cross-modal Non-Maximum Suppression. Late fusion is more flexible, tolerates spatial misalignment between modalities, and permits the use of modality-specific architectures, but sacrifices cross-modal feature interactions that could improve detection of ambiguous objects [23]. Mid-level fusion operates on intermediate feature representations, merging features at one or more stages of the detection backbone—for instance, at the Feature Pyramid Network level. This strategy offers a compromise between the representational richness of early fusion and the architectural flexibility of late fusion, and has demonstrated strong results in recent remote sensing studies combining optical and SAR imagery [128]. The choice among these strategies depends on the application context: early fusion is preferred when modalities are well co-registered and complementary at the pixel level, late fusion suits operational scenarios with heterogeneous sensor configurations, and mid-level fusion provides a balanced approach for multi-resolution data.

Operational Priority: High (Tier 1 for domain adaptation, Tier 2 for multi-modal fusion). Domain adaptation, preventing 15–40% performance degradation across regions, is critical for pan-European and global services. Multi-modal fusion combining SAR and optical data would improve all-weather detection robustness by 10–20%, particularly valuable for disaster response applications where cloud cover limits the utility of optical imagery.

6.6. Standardised Evaluation and Benchmarks

Current evaluation practices in remote sensing object detection suffer from inconsistent protocols, limited dataset diversity, and insufficient consideration of practical deployment constraints [131]. Advancing the field requires comprehensive standardised benchmarks that address these limitations. Future benchmarks must cover diverse environmental conditions, sensor types, spatial resolutions, and geographic regions to enable rigorous evaluation of generalisation capabilities. Size-specific evaluation metrics that separately assess performance on small, medium, and large objects would provide a more nuanced understanding of a method’s strengths and weaknesses across scales. Robustness metrics that quantify performance under challenging conditions, such as occlusion, scale variation, and rotation, would better reflect real-world deployment scenarios. Efficiency metrics encompassing not only accuracy but also computational cost measured through FLOPs, parameter counts, inference latency, and energy consumption would enable informed trade-offs for resource-constrained applications. Establishing annual competitions with maintained leaderboards would foster community engagement, facilitate direct method comparison, and accelerate progress through collaborative benchmarking.

Operational Priority: Medium (Tier 2). Standardised benchmarks accelerate research progress and enable fair method comparison, but do not directly improve operational detection capabilities. Size-specific metrics and robustness evaluations would help practitioners select methods that meet specific operational requirements.

6.7. Continual and Lifelong Learning

Operational remote sensing systems must adapt to evolving environments, emerging object categories, and changing sensor characteristics while preserving previously acquired capabilities. Yet, conventional training paradigms suffer from catastrophic forgetting when learning new tasks [132]. Continual and lifelong learning research addresses this challenge through several promising directions. Elastic Weight Consolidation prevents catastrophic forgetting [133] by identifying parameters critical to previous tasks and constraining their modification during subsequent learning, enabling sequential task learning without destructive interference. Memory-based approaches maintain a small set of representative examples from previous tasks [134] and periodically replay them during new task learning to reinforce earlier knowledge. Progressive neural networks allocate new network capacity to each task [135] while preserving previously learned parameters, enabling transfer learning across tasks without forgetting. Incremental learning techniques specifically designed for emerging object categories would enable systems to continuously expand their detection repertoire as new infrastructure types, vehicle models, or land-use patterns emerge in monitored regions.

Operational Priority: Medium (Tier 2). Continual learning enables model updates without catastrophic forgetting, reducing operational maintenance costs. While valuable for long-term system sustainability, it is less immediately constraining than annotation efficiency and domain generalisation for initial deployment.

6.8. Three-Dimensional Object Detection from Optical Imagery

Many remote sensing applications, including volumetric analysis, urban planning, and autonomous navigation, require three-dimensional information about detected objects beyond two-dimensional bounding boxes [136]. Extracting 3D structure from optical imagery presents unique challenges and opportunities. Monocular 3D detection methods leverage geometric constraints and learned priors [137] to estimate object depth, orientation, and 3D bounding boxes from single images, though depth ambiguity limits accuracy. Stereo 3D detection exploits binocular disparity from stereo image pairs to estimate depth maps [138], enabling more reliable 3D localisation when stereo imagery is available. Multi-view 3D reconstruction techniques [139] integrate observations from multiple viewpoints to recover complete 3D object geometry through structure-from-motion principles, supporting detailed 3D modelling from overlapping imagery. Optical-LiDAR fusion approaches [140] combine the complementary strengths of dense optical appearance information with precise LiDAR depth measurements to achieve robust 3D object detection that surpasses what either modality can achieve independently.

Operational Priority: Low (Tier 3). Three-dimensional information enhances specific applications (volumetric analysis, urban modelling), but current methods require specialised data (stereo, LiDAR), often unavailable in operational scenarios. Monocular 3D detection is still in its early stages of research, with limited practical accuracy. It therefore represents a strategic long-term direction rather than a near-term operational priority.

7. Discussion

7.1. Key Findings

This systematic review of 189 studies identifies a substantial methodological shift in object detection for optical remote sensing over the past fifteen years. The exponential growth in machine learning publications since 2012 (Figure 1) reflects a fundamental paradigm shift from hand-crafted features to automatic hierarchical feature learning, achieving mAPs exceeding 75% on challenging benchmarks such as DOTA.

However, method selection cannot be reduced to simple accuracy-based hierarchies. The comparative analysis (Table 1, Table 2 and Table 3) demonstrates that no single approach dominates across all performance dimensions. Deep learning methods excel when abundant data and computational resources are available, yet some classical techniques—template matching, knowledge-based methods, and OBIA—retain significant value in limited-data scenarios, for interpretability requirements, or under strict computational constraints. This challenges assumptions that newer methods universally supersede older approaches.

Performance–efficiency trade-offs critically shape operational deployment. Two-stage detectors achieve the highest accuracy (70–76% mAP) but exhibit slower inference (7–10 FPS), while one-stage detectors balance performance (60–74% mAP) with higher throughput (15–32 FPS). Oriented detectors achieve 10–15 mAP points of improvement over horizontal bounding boxes, confirming that domain-specific adaptations yield meaningful benefits beyond generic computer vision methods.

7.2. Critical Challenges

Despite remarkable progress in detection accuracy and methodological sophistication, several fundamental challenges continue to impede broader adoption of object detection systems in operational remote sensing contexts. The annotation bottleneck represents perhaps the most significant practical limitation, as deep learning’s large data requirements restrict application to domains where large-scale annotation efforts are feasible or where sufficient pre-existing labelled datasets exist. Scale variation across objects spanning two to three orders of magnitude within single images challenges even sophisticated multi-scale architectures, as small objects provide limited discriminative features. In contrast, large objects may exceed typical receptive field sizes.

Domain adaptation failures occur frequently when models trained under specific geographic, seasonal, or sensor conditions are deployed in different environments, with performance often degrading substantially despite superficial similarity between training and deployment contexts. The computational complexity of state-of-the-art models necessitates powerful GPU hardware for both training and inference, creating barriers for resource-constrained applications and limiting edge deployment possibilities. The black-box nature of deep learning models poses particular problems for critical applications in domains such as disaster response or infrastructure monitoring, where decision transparency and accountability are paramount.

Class imbalance pervades remote sensing datasets, with common objects vastly outnumbering rare but often important categories, leading to biased models that perform poorly on under-represented classes. Real-time processing remains challenging due to the combination of extremely large image sizes characteristic of remote sensing data and the computational demands of complex neural network architectures, limiting the feasibility of time-critical applications such as continuous monitoring or autonomous navigation.

7.3. Implications for the Remote Sensing Community

These findings carry important implications for the community. First, evaluation must extend beyond accuracy to encompass comprehensive frameworks that consider data efficiency, computational requirements, domain generalisation, and interpretability. Current benchmarks increasingly limit meaningful comparison even as they drive algorithmic progress.

Second, addressing the annotation bottleneck through weakly supervised, self-supervised, and foundation model research is an operational necessity, not mere convenience, particularly for time-critical applications like disaster response.

Third, domain adaptation emerges as critical for practical deployment at scale. Methods achieving 65–80% mAP on benchmarks but degrading 15–40% across regions limit utility for continental or global monitoring programs.

Fourth, hybrid symbolic—neural approaches challenge assumptions that end-to-end deep learning represents the ultimate solution. Thoughtful integration of OBIA, physics-informed networks, and neuro-symbolic architectures can achieve superior performance in limited-data regimes, suggesting synergies between classical and modern approaches.

Finally, computational efficiency must become a first-class consideration. The gap between state-of-the-art accuracy (75–80% mAP) and real-time deployable methods (60–65% mAP at 30+ FPS) constrains time-critical applications. The three-tier prioritisation framework (Table 4) distinguishes critical near-term needs from strategic long-term directions, enabling operational partners to refine priorities based on specific requirements.

8. Conclusions

This comprehensive review systematically analysed object detection methodologies for optical remote sensing imagery, providing a critical assessment of four principal approaches: template matching, knowledge-based methods, object-based image analysis (OBIA), and machine learning/deep learning methods. The field has transformed from hand-crafted feature engineering to end-to-end deep learning systems, with CNNs and Transformers achieving up to 75% mAP on standard benchmarks.

However, significant operational challenges remain: annotation bottlenecks (labour-intensive dense labelling requirements), extreme scale variation (8–800 pixels), and domain adaptation failures (15–40% degradation) must be addressed before object detection fulfils its transformative potential for Earth observation.

Converging technological trends promise solutions. Foundation models will reduce annotation requirements through zero-shot detection. Hybrid symbolic–neural architectures will combine deep learning’s pattern recognition with the interpretability and data efficiency of knowledge-based reasoning. Efficient architectures will enable edge deployment, while multi-modal fusion will provide all-weather detection integrating optical, SAR, and LiDAR observations.

The future lies in thoughtful integration of complementary approaches rather than single-methodology dominance. This holistic perspective—combining learned and explicit representations, supervised and self-supervised learning, and multiple sensor modalities—will guide next-generation systems capable of reliable, planetary-scale deployment.

For practitioners, the present review provides evidence-based decision frameworks for method selection. For researchers, the prioritised roadmap (Table 4) identifies critical directions: foundation models, weakly supervised learning, and domain adaptation as near-term priorities; explainable AI, 3D detection, and continual learning as strategic long-term goals.

As these technologies converge, object detection will serve as a foundational capability for automated planetary understanding through continuous analysis of satellite and aerial imagery, supporting accurate, efficient, generalisable systems deployable at the scales required for environmental monitoring, disaster response, urban planning, and sustainable development.