Benchmarking Robust AI for Microrobot Detection with Ultrasound Imaging

Abstract

Microrobots are emerging as transformative tools in minimally invasive medicine, with applications in non-invasive therapy, real-time diagnosis, and targeted drug delivery. Effective use of these systems critically depends on accurate detection and tracking of microrobots within the body. Among commonly used imaging modalities, including MRI, CT, and optical imaging, ultrasound (US) offers an advantageous balance of portability, low cost, non-ionizing safety, and high temporal resolution, making it particularly suitable for real-time microrobot monitoring. This study reviews current detection strategies and presents a comparative evaluation of six advanced AI-based multi-object detectors, including ConvNeXt, Res2NeXt-101, ResNeSt-269, U-Net, and the latest YOLO variants (v11, v12), being applied to microrobot detection in US imaging. Performance is assessed using standard metrics (AP50–95, precision, recall, F1-score) and robustness to four visual perturbations: blur, brightness variation, occlusion, and speckle noise. Additionally, feature-level sensitivity analyses are conducted to identify the contributions of different visual cues. Computational efficiency is also measured to assess suitability for real-time deployment. Results show that ResNeSt-269 achieved the highest detection accuracy, followed by Res2NeXt-101 and ConvNeXt, while YOLO-based detectors provided superior computational efficiency. These findings offer actionable insights for developing robust and efficient microrobot tracking systems with strong potential in diagnostic and therapeutic healthcare applications.

1. Introduction

Microrobots have emerged as a transformative innovation at the intersection of bio-medical engineering and intelligent control, enabling new possibilities in minimally invasive medicine [1]. Their development has attracted increasing attention from both research and clinical communities, particularly for applications in non-invasive treatment, diagnosis, and targeted therapy. This progress marks an important step toward expanding the role of microrobotics across diverse areas of biomedicine [2]. These tiny robots can navigate through complex biological environments inside the human body. Size-wise, microrobots are broadly classified into three categories: nanoscale (100 nm–1 µm), microscale (1–100 µm), and millimeter scale (1–5 mm). For this reason, they can revolutionize the diagnosis and treatment landscape through targeted drug delivery, localized diagnostics, and microsurgical procedures [3]. Schmidt et al. [4] analyzed the effectiveness of a cell-based microrobot for tumor therapy. These robots can be guided to tumor sites and deliver drugs directly, thereby reducing adverse effects on surrounding healthy tissues, which is a limitation commonly encountered in conventional treatments such as radiation therapy and chemotherapy, as noted by Xie et al. [5]. Furthermore, as described by Lan et al. [6], these microrobots enable therapeutic interventions in challenging-to-reach regions of the body, including the gastrointestinal tract, cerebral vasculature, and deep lung airways, where traditional methods are ineffective or fail, providing a compelling argument for developing microrobot-based technologies for medical applications.

Yan et al. [7] studied microrobot tracking within the human body. The technical functional properties of microrobots help them overcome physical barriers and efficiently access confined anatomical areas. Research by Lee et al. [8] investigated the use of microrobots in diverse biological environments, including within the human body. They showed that the functional properties of microrobots, such as their small size and magnetic actuation, allow these devices to overcome physical barriers and access confined anatomical regions, thereby enabling more efficient targeted interventions. This perspective is echoed by Nauber et al. [9], who critically discuss the flexibility that microrobots bring to therapeutic interventions in reproductive medicine. It is evident that Microrobots are rapidly advancing across multiple medical applications, including reproductive medicine [9], ocular drug delivery [10], targeted therapeutic applications [11], immune evasion [12], toxin detection [13], and brain vasculature [14]. Ongoing research and development in micro-robotic technology are crucial for advancing less invasive treatments and diagnostics, which can lead to better outcomes by delivering the right treatments at the right place and promoting healing.

The above literature highlights how microrobots can benefit medical systems, with the potential to impact patient care and quality of life directly. A critical factor in advancing microrobot applications is the development of state-of-the-art methods for detecting, tracking, and supporting their navigation. Microrobots can be detected and tracked using various imaging modalities, including Magnetic Resonance Imaging (MRI) [14], Computed Tomography (CT) [15], optical imaging, and ultrasound (US) [16]. However, in clinical settings, they face significant challenges in detection and tracking, as noted by Pane et al. [17,18], including limited real-time tracking capabilities, radiation exposure, cost, and non-portability. Furthermore, as highlighted by Wrede et al. and Sawhney et al. [14,19], the microrobot’s small size, dynamic operating environments, and the presence of surrounding tissue artifacts further complicate the detection and tracking of microrobots. Figure 1 highlights therapeutic, diagnostic, monitoring, and clinical benefits of microrobots.

1.1. Aims and Contributions

This work aims to compare the standard state-of-the-art object detection models and establish a comprehensive and reproducible performance benchmark for ultrasound-guided microrobot detection. The novelty of this work lies in presenting an extensive benchmarking study of six object detectors: ConvNeXt, Res2NeXt-101, ResNeSt-269, U-Net, and two latest nano variants of YOLO (v11, and v12) for microrobot detection using US imaging. The focus of this work is not only on analyzing baseline performance metrics, including AP (75–90), precision, recall, and F1-score, but also on robustness under challenging operating conditions, efficiency, and feature-level contributions. This will also make the system more explainable and open to interpretation. The contributions of this work are as follows:

• Systematic performance benchmarking of six state-of-the-art multi-object detectors for microrobot detection using US imaging.
• Evaluation of the robustness of object detectors under realistic US imaging challenges, including blur, brightness variation, occlusion, and speckle noise.
• Feature-level sensitivity analysis that quantifies the contributions of visual cues (edges, sharpness, textures, and regions) to detector performance.
• Understand the detector’s efficiency and deployment feasibility by examining computational efficiency using metrics such as latency, frames per second (FPS), parameter count, and giga floating-point operations per second (GFLOPs).

1.2. Structure of the Paper

This paper consists of six sections that provide a clear and focused exploration of deep learning-based detectors for microrobot detection using the US. In Section 2, a literature review is conducted related to the role of US imaging in micro robotics and US-based object detection. Section 3 explains the methodology, including the setup of experiments and datasets, as well as a benchmarking pipeline for microrobot detection. Section 4 discusses the results. In Section 5, limitations of the study are discussed. Finally, the paper concludes in Section 6.

2. Literature Review

Real-time detection of microrobots remains challenging in various medical procedures, including targeted drug delivery, minimally invasive surgery, and localized diagnostics [20,21,22]. Additionally, microrobot tracking depends on the performance of microrobot detectors. US imaging is used for this purpose to provide visual feedback that guides medical procedures [23]. The major drawback of the US is its limited spatial resolution and signal-to-noise ratio, which are significant challenges in microrobotic scenarios [20,24]. On the other hand, today, AI is transforming virtually every domain, including healthcare [25], and medicine [26], physics [27], chemistry [28], education [29,30], robotics [31,32], astronomy [33], and beyond from different application perspectives. Similarly, AI can be leveraged to address the challenges posed by US imaging in detecting microrobots.

2.1. Why Ultrasound Imaging?

Various medical imaging modalities, such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT) scans, optical imaging, and US, can be beneficial for designing machine-learning- and deep-learning-powered microrobot detection and tracking methods [34,35,36]. MRI imaging can be used to visualize soft tissues and vascular structures. Tong et al. [37] discuss the importance of MRI in breast tumor identification. MRI uses a paramagnetic contrast agent that enhances tumor visualization, providing valuable characteristics that inform surgical decisions and treatment plans. Further, CT scans offer rapid imaging and high spatial resolution, making them a strong choice in trauma settings and for evaluating complex anatomical structures. The utility of CT in assessing scoliosis is excellent, as it provides a detailed view of vertebral rotation, which is not possible with other 2D imaging methods [38]. However, MRI and CT scans have limitations that make them unsuitable for detecting and tracking microrobots, as shown in Figure 2. From a technical perspective, MRI systems are costly to deploy and maintain, which limits their accessibility [39,40]. While CT scan systems can have adverse effects on human cells, as demonstrated in several research studies [41,42]. On the contrary, US imaging is a low-cost technology with minimal side effects, making it an optimal candidate for detecting and tracking microrobots.

Numerous studies [23,43,44,45] have demonstrated that US images are well suited for real-time monitoring of microrobots and are ideal for point-of-care use compared with MRI and CT. It is because the US is non-ionizing, cost-effective, portable, and compact, providing high temporal resolution. Figure 2 presents a detailed comparative analysis of MRI, CT, and US imaging. Further, building on this knowledge, Cheung et al. [38] stated that contrast-enhanced US imaging considerably increases diagnostic efficacy. Additionally, Figure 2 illustrates the reasons why the US is the optimal choice for microrobot detection and tracking. However, US imaging contains noise, has low contrast, and has limited resolution, which makes the detection and tracking of microrobots challenging. With advancements in AI and the emergence of deep learning as a powerful tool in computer vision tasks, some improvements have been achieved in the detection and tracking of microrobots using US.

2.2. Microrobot Detection

Microrobots in medical science are tiny robotic devices used inside the human body to aid in diagnosis, treatment, minimally invasive surgery, and tissue repair. Detection of these small robots is essential for real-time monitoring and precise movement control, enabling the microrobot to perform as planned on the intended target within the body. This also guarantees the safety, accuracy, and effectiveness of medical procedures [46]. As depicted in Figure 2, US images are preferred over MRI and CT scan images because they are more efficient in terms of size, cost, scan speed, and portability [47]. In the last decade, machine learning and, particularly, deep learning methods have made significant inroads in the development of microrobot detection methods based on US imaging [48].

Nowadays, Convolutional Neural Networks (CNNs)-based methods are widely used for microrobot detection in ultrasound imaging for medical applications [21,49]. Botros et al. [21] propose a deep learning-based method for detecting and tracking chain-like magnetic microrobots in real-time using ultrasound imaging, achieving up to 95% detection and 93% tracking accuracy in dynamic in vitro environments. Further, Sadak [49] introduces an explainable CNN model, the Attention-Fused Bottleneck Module (AFBM), to improve microrobot classification and localization using US imaging. It outperforms YOLO-based models, achieving a maximum mAP of 0.909 at an IoU of 0.95. Similarly, Li et al. [22] proposed a new variant of YOLOv5, which produces over 91% prediction accuracy for microrobots of sizes 1–3 mm. The proposed method is effective for both 2D and 3D US imaging in detection and tracking for in vivo applications.

Reinforcement learning (RL) [50], along with vision-based transformers (ViTs) [51,52], is also emerging as a strong contender for microrobot detection methods. However, most researchers currently focus on diagnosis-based detection using US imaging rather than microrobot detection. Medany et al. [53] introduce a model-based Rapproach for controlling and detecting microrobots using US imaging. The paper has a particular focus on utilizing image feedback to support real-time and complex environments. The advantage of the method is that it can operate with limited training data and achieves up to 90% accuracy across various scenarios. Liu et al. [51] presented a method that combines CNNs with a transformer for detecting robotic capsules in real time using 3D US. The advantage of working with transformers is that they can capture long-range context, thereby improving detection accuracy. The system achieved a detection accuracy of over 90% and a localization error of 1.5 mm. Microrobots face challenges in manual navigation due to complex dynamics and multiple motion parameters. Schrage et al. [54] use reinforcement learning to enable autonomous navigation by training microrobots on over 100,000 images for accurate detection and control. The method combines primary and secondary acoustic radiation forces to guide microswarms along desired paths with robustness and adaptability. The gastrointestinal tract is complex due to its intricate internal anatomy, making detection challenging, where US imaging plays a critical role. In conventional tracking of capsule location and operational state, a substantial challenge is added for clinical applications. To address this issue, Liu et al. [55] introduced more intelligent robotic behaviors by proposing a deep learning-based approach that combines hierarchical attention with ultrasound imaging for both state classification and 2D in-plane pose estimation of capsule robots. It achieved 97% accuracy in mechanism state detection and a mean centroid/orientation error of 0.24 mm.

3. Methodology

This work lies in establishing a robust and clinically relevant benchmarking framework for detecting microrobots of different shapes using US imaging. The focus is not just on conventional accuracy reporting, but also on moving beyond it and assessing detectors (models) under realistic biomedical conditions. As stated in Figure 3, we selected six of the most recent state-of-the-art object detector which includes ConvNeXt [56], Res2NeXt-101 [57], ResNeSt-269 [58], U-Net [59], and two latest variant of YOLO (v11 [60], v12 [61]), which have demonstrated superior performance across various domains and are chosen for their accuracy, speed, and efficiency. Thus, making them well-suited for real-time medical applications. Further, we selected the USMicroMagSet [20] dataset for its dedicated and diverse ultrasound microrobot images. To ensure model reliability, reproducibility, and statistically sound performance, all detectors are trained using the same hyperparameters and on 5-fold cross-validation. The analysis methodology covers accuracy, efficiency, and feature-level analysis. This multi-tiered design guarantees not only scientific robustness but also practicability for real-world biomedical applications.

3.1. Experimental Setup

All experiments are conducted using RunPod.io, a commercial GPU cloud service platform. We employed the NVIDIA B200 GPU, featuring the state-of-the-art Blackwell architecture, for the more computationally intensive multi-microrobot detection training and testing at an image size of 1920 × 1080, which requires a robust processing infrastructure. The B200 GPU node has 28 vCPUs from an AMD EPYC 9555 (64-core) processor and one B200 GPU with 180 GB vRAM. To store the dataset, 200 GB of network storage has been used. The B200 GPU delivers ~2250 FP16 TFLOPs (Dense) and up to ~4500 FP16 TFLOPs (Sparse), facilitating extreme throughput for next-generation AI workloads. The implementation uses the Ultralytics and Torchvision libraries in a Python environment built on PyTorch. We have standardized the core hyperparameters across all six models as stated in

Table 1. Uniform training settings.

Parameter	Value/Setting	Purpose
Epochs	30	Ensures equal training duration across architectures
Batch Size	16	Maintains identical gradient update frequency
Dataloader Workers	16	Ensures equal data throughput efficiency
Input Resolution	1080 × 1920 (rectangular)/YOLO imgsz = 1920	Same image scale for consistent feature learning
Optimizer	AdamW	Same optimization behavior across networks
Learning Rate	0.0001	Uniform learning dynamics
Mixed Precision (AMP)	Enabled	Same precision & GPU acceleration method
Model Compilation	torch.compile/compile = True	Equal computational optimization

. Furthermore, Figure 4 provides a detailed explanation of the B200 GPU computational infrastructure used in this work.

3.2. Dataset

In this work, we utilized USMicroMagSet [20], a comprehensive, publicly available B-mode US dataset for microrobots available on IEEE DataPort. It consists of over 40,000 annotated US frames of size 1920 × 1080 for eight different shapes of magnetic microrobots and two locomotion modes. However, the total number of usable images is 32,511. Table 1 contains all microrobot class distributions used during training and testing. It should be noted that we have not divided the dataset into training and testing sets; instead, this work used the same division provided in the USMicroMagSet dataset folder structure, where training and testing data are located in the (train + val) and test folders, respectively. It helps ensure the work is universally applicable and supports future benchmarking efforts that can be performed using the same training and testing data.

Furthermore, Figure 5 visualizes samples of unlabeled and labeled US images from USMicroMagSet. This dataset is commonly used for benchmarking detection and tracking algorithms [49], particularly in machine learning and deep learning. Other works have curated datasets from closed-loop control experiments, generally not openly available. Nauber et al. [62] used a custom US dataset to fine-tune deep learning models for microrobot detection and tracking. The lack of large, diverse, high-quality, and annotated datasets in the US has been a considerable barrier. Other than USMicroMagSet, few datasets address the gap between the need and availability of US datasets to fuel research in this field. Specifically, we observe a lack of deep learning-based methods for detection and tracking. The current issue is due to a lack of a dataset; training these powerful AI algorithms is not possible. Furthermore, ongoing works [49,61,62] focus on enhancing spatial resolution, minimizing artifacts, and improving US quality in complex, dynamic environments, such as the human body.

3.3. Benchmarking Pipeline

A standardized benchmarking pipeline, as depicted in Figure 6, has been developed for experimentation with six state-of-the-art detectors that have demonstrated effectiveness across multiple domains. These detectors can play a notable role in addressing the challenge of microrobot detection using ultrasound imaging. The most recent and widely used object detectors are selected, including ConvNeXt [56], Res2NeXt-101 [57], ResNeSt-269 [58], U-Net [59], and the twoYOLO variants (v11 [60] and 12 [61]). To have a robust, reliable, and reproducible final model, five-fold cross-validation (5-Fold CV) is used.

Table 2. Distribution of microrobot classes in training and testing sets.

Class Name	Training Count	Testing Count
cube	2267	1222
cylinder	1596	861
flagella	1956	1054
helical	3155	1699
rollingcube	1255	677
sheetrobot	3010	1622
sphere1	5900	3178
sphere3	1988	1071
Total	21,127	11,384

presents comparative details of the used detectors and their relative superiority from a specific perspective.

The training pipeline, as depicted in Figure 6, uses the 5-Fold CV; the training dataset is randomly partitioned for each fold for training (80%) and validation (20%) splits. For all models, we used 30 epochs. As the dataset is large and the images are high resolution (1920 × 1080), we need to design models that not only achieve high accuracy but are also computationally efficient. Therefore, we used mixed-precision (FP16) to leverage Tensor Core acceleration fully. The total number of workers used is 16, and a batch size of 16 is used to accommodate high-resolution input. A larger batch size would require more memory, potentially overwhelming even the most advanced GPUs. For reproducibility, a fixed random seed (42) is used. During 5-Fold CV training (patience = 5) and during final model refitting, disabled validation and early stopping (patience = 0) were used to consolidate the five best-performing models into a single final model trained on the entire dataset (training and validation). All trained weights and performance data are saved. The testing pipeline is designed to analyze the detectors from multiple perspectives: (1) accuracy, (2) visual perturbation, (3) feature contribution, and (4) computational efficiency.

4. Results and Discussion

Benchmarking has been conducted for multi-microrobot detection. The six state-of-the-art detectors are evaluated based on four criteria we set for this work.

First, the detector is evaluated using classical detection benchmark metrics, including Average Precision (AP 50–95), precision, recall, and F1 Scores. Secondly, each detector is evaluated under challenging conditions to test its robustness, including blurriness, brightness, occlusion, and noise. Thirdly, feature-level contributions are also computed for each detector. Lastly, computation resource requirements are also measured, which include latency (ms/image), frame per second (FPS), parameters in millions, and model file size in MB, revealing the classic speed/accuracy/size trade-off in deep learning.

4.1. Classical Detection Metrics

Overall and Per-class performance analysis, as presented in

Table 3. Comparison of YOLO11, YOLO12, ConvNeXt, Res2NeXt-101, ResNeSt-269, and U-Net.

Detectors Launch Year	Key Innovation	Strength for Ultrasound Microrobot Detection	Clinical Relevance
YOLO12 (2025)	Attention-centric modules (Area Attention, R-ELAN)	Improves contextual awareness and fine-grained discrimination, even if the background is noisy	supporting trustworthy clinical decision-making by maximizing precision in challenging imaging conditions
YOLO11 (2024)	Improved backbone–neck design, multi-task capability.	Greater adaptability for handling complex microrobot appearances and dynamics in ultrasound.	Supports advanced applications related to monitoring and controlling microrobot behavior.
ConvNeXt (2022)	The transformer-inspired CNN architecture uses large kernels, depthwise convolutions, and normalization improvements.	Robust hierarchical feature extraction, speckle noise handling, and low contrast in ultrasound.	Provides reliable baseline detection and supports fusion with more advanced detectors.
ResNeSt-269 (2020)	Split-attention blocks for channel-wise feature recalibration	Enhances robustness to background clutter in ultrasound by selectively emphasizing informative channels.	Supports precise localization of microrobots amidst complex tissue structures.
Res2NeXt-101 (2019)	Combines ResNeXt split-transform-merge strategy with Res2Net multi-scale feature representation.	Able to capture fine-to-coarse structural details, which results in better detection of varying sizes and shapes of microrobots.	Enables the accurate identification of diverse microrobot morphologies, which is crucial for heterogeneous robotic swarms.
U-Net (2015)	Symmetric encoder–decoder with skip connections for biomedical image segmentation.	During noisy ultrasound scans, it provides strong boundary delineation of microrobots.	Widely adopted in medical imaging, it facilitates accurate segmentation for monitoring the interactions of microrobots with tissue.

and

Table 4. Overall performance heatmap metric of six detectors.

Model	mAP (50–95)	mAP50	mAP75	Precision (Mean)	Recall (Mean)	95% CI Lower	95% CI Upper
YOLO12	0.617824433	0.962943476	0.670267625	0.93580446	0.840348152	0.5591	0.6126
YOLO11	0.613175223	0.949897181	0.690278055	0.962045926	0.769022882	0.3186	0.7536
ConvNeXt	0.866752148	0.98678273	0.916006863	0.985324123	0.995195712	0.7955	0.8484
ResNeSt269	0.878249645	0.988578439	0.930365086	0.986016259	0.995728513	0.8219	0.8515
Res2NeXt-101	0.866416454	0.988445342	0.919016898	0.985829465	0.995924184	0.8211	0.8563
U-Net	0.7309196	0.875163198	0.781938016	0.7309196	0.880745351	0.732	0.8141

, reveals an apparent dichotomy in the difficulties encountered during detection from the dataset. As stated in Table 3, ResNeSt-269 achieved the highest mAP across IoU thresholds (0.50–0.95) of 0.878. Whereas ConvNeXt and Res2NeXt-101 performed similarly to ResNeSt-269, with mAPs of 0.8667 and 0.8664, respectively. U-Net achieved moderate mAP (50–95) of 0.731. However, YOLO11 and YOLO12 trailed the others and achieved the lowest mAPs of 0.6178 and 0.6131, respectively. Furthermore, U-Net achieved the lowest precision of 0.7309196, whereas YOLO11 and YOLO12 achieved the lowest recall of 0.7690 and 0.8403, respectively. Utilizing the rigorous COCO standard, Mean Average Precision (mAP 50–95), and the F1-Score as primary metrics, our analysis demonstrates that an extensive evaluation of six detectors, as stated in Table 4, revealed that ResNeSt-269, Res2NeXt-101, and ConvNeXt show a high degree of proficiency in detecting eight different shapes of the microrobots using US imaging. As shown in Table 4, both YOLO11 and YOLO12 failed to detect rolling-cube-shaped microrobots, which would affect their overall performance in Table 3. For US microrobot detection, where accuracy is critical due to the nature of its applications, ResNeSt269 is the strongest detector, followed by Res2NeXt-101 and ConvNeXt. The YOLO11 and YOLO12 accuracies are the lowest among the six detectors.

Table 4 quantifies the robustness and stability of each detector. In Table 4, we report 95% confidence intervals (CIs) computed across the 5-fold CV results. Confidence intervals are used to show the variability of detection accuracy under different train and validation splits. However, it highlights the uncertainty associated with the CV process. Still, it does not prove the uncertainty of the final test evaluation, as the CV process splits the training data into 80% and 20% for training and validation, respectively. As a result, it is expected that the testing results may fall slightly outside the lower and upper confidence intervals.

Generally, for stronger architectures such as ResNeSt-269, Res2NeXt-101, and ConvNeXt, mAP may fall slightly outside the CV confidence interval, which is expected behavior, as the final detector is trained on 100% of the training data (as stated in Table 4). It tends to improve performance in stronger architectures. However, detectors that are architecturally weaker for the microrobot detection problem, such as YOLO11 and YOLO12, do not improve their performance even when trained on complete training data. Further, Section 4.4 can be referred to for understanding the weaker and stronger architectures of the detectors, particularly for the microrobot detection problem. Thus, the stated results in Table 4, related to confidence intervals, implicitly reflect the robustness of the detectors and complement mAP (50–95) as given in Table 4.

4.2. Perturbations

The goal of introducing perturbation is to conduct meaningful stress tests of ConvNeXt, Res2NeXt-101, ResNeSt-269, U-Net, YOLO11, and YOLO12 detectors using perturbation analysis. Four challenging conditions are introduced in the actual US images: blur, brightness, occlusion, and speckle noise.

A Gaussian blur with a kernel size of (21, 21) with kernel (k = 10) is introduced. Where OpenCV automatically computed σ ≈ 3.5, according to the single Gaussian convolution is applied. It is a noticeable and significant blur that can simulate real-world microrobot motion, physiological motion, and a poor acoustic window. The blurred image can be defined as

$$I_b\left(x,y,c\right)=(I\ast G_{\mathrm{\sigma }})(x,y,c)$$

(1)

where * is a 2d convolution, $G_{\mathrm{\sigma }}$ is a Gaussian kernel, x and y are pixel coordinates, and finally, color channels are denoted by $cϵ\{R,G,B\}$

Whereas brightness and contrast are adjusted using an alpha (α) value of 1.8 (contrast control) and a beta (β) value of −40 (brightness control), which simulate the effects of operator settings, compensation (TGC), and signal attenuation. We have applied an affine transformation to each given as

$$I_{bc}\left(x,y,c\right)=clip(\alpha I\left(x,y,c\right)+\mathrm{\beta })$$

(2)

For an occlusion, five black boxes are added to each image. Further, the dimensions of these boxes are relative to the image size, with each box’s height and width set to 15% of the original dimensions. This helps block parts of the ultrasound view and simulate anatomical blockages, including anatomical structures and imaging artifacts. The intensity of the pixel after occlusion (region O) can be given as

$$I_{oc}\left(x,y,c\right)=\left\{\begin{array}{c}0, \left(x,y\right) ϵ 0\\ \left(x,y,c\right), otherwise\end{array}\right.$$

(3)

Under blur perturbations, as depicted in Figure 7, ConvNeXt and Res2NeXt-101 exhibit strong resilience, with AP drops of only 12–14%. Whereas ResNeSt-269, YOLO12, and YOLO11 performed moderately with (18–30%) AP drop. However, the U-Net completely collapsed, resulting in a nearly 79% drop in AP. Figure 7 depicts the same trend seen in brightness perturbations, which appeared to be the most challenging and severely degraded AP. ConvNeXt and Res2NeXt-101 remain relatively stable with a (15–28%) AP drop. Whereas ResNeSt-269, YOLO12, and YOLO11 detectors suffer a severe AP drop of (32–71%), and the U-Net totally collapsed with an 84.85% AP drop. Figure 7 illustrates the overall moderate impact of occlusion, excluding the U-Net. In summary, for all three blur, brightness, and occlusion, ConvNeXt, Res2NeXt-101, and ResNeSt-269 are the most robust, while YOLO detectors performed moderately, and U-Net shows significant vulnerability.

Speckle noise perturbation is arguably the most crucial robustness test for microrobot applications using US imaging. Speckle noise generally results from the interference of backscattered echoes, which cannot be or are not artificially produced [63]. Speckle Noise with variances of 0.02 to 0.1 is added to simulate the US texture signature. This type of noise is not simple sensor noise and represents a highly intricate interference pattern of real-world sound waves scattered by microscopic structures within tissue. US environments with speckle noise add a substantial and realistic challenge for microrobot detectors. This is because it may arise from faulty equipment, an unskilled operator, or suboptimal acquisition settings. Therefore, this work systematically examines the performance robustness of ConvNeXt, Res2NeXt-101, ResNeSt-269, U-Net, YOLO11, and YOLO12 object detectors against speckle noise, a phenomenon commonly observed in US images.

We evaluated each detector on five levels of speckle noise perturbations with noise variances (${\mathrm{\sigma }}_s^2$ = 0.02, 0.04, 0.06, 0.08, 0.1). Each pixel is perturbed by an additive noise term $n\left(x,y,c\right)\sim N(0,{\mathrm{\sigma }}_s^2)$. Thus, the resulting intensity is given as

$$I_{sp}\left(x,y,c\right)=clip\left(I\left(x,y,c\right)\right[1+n\left(x,y,c\right)\left]\right)$$

(4)

Speckle noise analysis shows striking differences in model resilience as noise variance increases, as depicted in Figure 8. YOLO12 shows minor AP drops, with a range between (1.1 to 1.7), whereas YOLO11 AP drop range is (1.1 to 4.0), slightly higher than its successor. After YOLO’s based detector, ConvNeXt is also resilient to speckle noise with AP drop range (5.3 to 11.2). In contrast, ResNeSt-269 and U-Net failed under speckle noise, with APs dropping above 50% once the variance reached 0.08. highlighting their strong noise tolerance. In summary, YOLO-based detectors maintain reliable performance under challenging conditions with ultrasound noise. These findings provide substantial proof that noise brings challenging and sometimes catastrophic instability during microrobot detection.

4.3. Feature Contributions

To quantify feature-level contributions, we used a computationally lighter methodology. The idea here is to identify which visual factors each detector relies on during detection. A robustness-based feature contribution analysis on the mAP drops, which is observed under different perturbations. We mapped each perturbation to an abstract feature, such as Blur to Edges, Brightness to Illumination, Occlusion to Spatial, and Speckle Noise to Texture. The feature level contribution is given as

$$W_f=100·{\displaystyle \frac{\mathrm{m}\mathrm{a}\mathrm{x}(0,d_f)}{{\sum }_{f^{\prime }}\mathrm{m}\mathrm{a}\mathrm{x}(0,d_{f^{\prime }})}}$$

(5)

where the mAP drop is denoted by $d_f,$ and f is the element of edges, texture, illumination, and spatial. The most significant features reported for each detector are the features with the largest $W_f.$

We do not rely on pixel-level Grad-CAM as it provides local saliency and is less reliable. Further, they do not offer model-level feature dependence, and from an operational perspective, they are computationally too heavy for detector architectures. Instead, we used robustness perturbations to induce model-level feature dependence, often resulting in a more stable and more informative model.

Feature-level analysis reveals that different detector architectures focus on distinct features, a virtue of their architectures, as depicted in Figure 9. ConvNeXt exhibits a balanced profile across all four analyzed features, with spatial features (31%) slightly predominating. Whereas Res2NeXt-101 shows a different trend in feature contributions. It places a strong emphasis on illumination (35%) and texture (25%), making it more sensitive to light and effective under high-contrast conditions. However, this also means that it is potentially vulnerable to changes in illumination. ResNeSt-269 relies most heavily on the texture (43%) feature, suggesting a specialization in fine-grained surface patterns, which may be relevant to noise-rich US imaging. Furthermore, U-Net focuses on illumination (35%) and edges (32%), which is understandable given its segmentation-supportive architecture, which benefits from boundary detection and intensity gradients. Lastly, as depicted in Figure 9, YOLO11 and YOLO12 exhibit an overwhelming reliance on illumination, accounting for around 56%, and neglect texture, indicating that, despite architectural advances, they still rely heavily on illumination.

4.4. Explainability

Explainability fosters trust and transparency. It establishes accountability, debugging, and fairness. Particularly in high-stakes domains, it helps in understanding the safety profile. For example: healthcare and security. One major issue with AI today is that it is challenging to explain the decision-making process by which it arrives at its conclusions, as deep learning algorithms are inherently black boxes. Now, considerable research and development efforts have been going into development models whose decisions can be explained. In this work, we contribute to the effort to develop explainability for microrobot detection using ultrasound imaging. We applied our understanding of algorithmic design principles and the knowledge gained in Section 4.3, “Feature Contributions”. Based on integrated analysis of the AP (50–95) across all six detectors, the three worst-performing object classes are the rolling cube, the sheet robot, and sphere1, with AP (50–95) values of 0.598, 0.676, and 0.768, respectively. U-Net performed worst on Sphere1, YOLO12 performed worst on SheetRobot, and YOLO11 performed worst on RollingCube. An integrated analysis of model feature sensitivity and model architecture largely explains why Sphere1, Sheetrobot, and RollingCube emerged as the three worst-performing classes for specific detectors.

As stated above, the U-Net achieved the lowest performance on sphere1, due to weak boundaries, low contrast, and ultrasonic streaking artifacts. The U-Net’s failure on the sphere1 class stems from its architectural design. It is an encoder–decoder segmentation model with skip connections, where it depends significantly on edges (32%) and illumination (35%). However, its dependence on texture is only 13%. The Sphere1 microrobot class provides almost none of the boundary cues that the U-Net’s architectural design can exploit. This result is in poor detection performance. For YOLO11 and YOLO12, the worst results are observed for the rollingcube and sheetrobot microrobot classes. This includes a substantial performance deterioration for YOLO11 on rollingcube. Feature-level analysis provides strong evidence that YOLO models overwhelmingly prioritize illumination, with an approximate 56% focus. Whereas, it almost entirely ignores texture (<3%). Whereas, if we look at some of the properties of rollingcube and sheetrobot, they appear as dark, low-intensity, texture-driven microrobots in US imagining and possess significantly less brightness contrast. Therefore, YOLO becomes nearly feature-insensitive for rollingcube and sheetrobot classes. This also explains why ResNeSt-269 achieved top performance, as it uses split-attention blocks that yield the highest texture reliance of approximately 43%. This helps it capture fine-grained intensity fluctuations, which aid in identifying challenging microrobot shapes. This explains a texture-concentrated detector. ConvNeXt, with its transformer-inspired CNN architecture with hierarchical feature extraction, focuses on all features in a generalized manner. Res2NeXt-101 employs a split–transform–merge strategy, enabling it to focus on illumination and texture.

In summary, based on Table 2, Table 4, and Figure 9, the worst-performing classes are not random outliers but rather reflect a systematic mismatch between each model’s architectural design and the features of specific US microrobot classes.

4.5. Computational Efficiency

Accuracy is considered the most essential performance metric for evaluating detectors. However, computational efficiency plays an equally critical role in determining its real-world implementation. Particularly for microrobot detection applications, where detection is performed in real time. Latency, FPS, number of model parameters, and model size play a critical role in any detector’s success in real-time deployment beyond laboratory settings. As depicted in Figure 10a, YOLO11 is the fastest, with a latency of 6.1 and a FPS of 163. Res2NeXt-101 is the second-fastest, with a latency of 6.7 ms and a frame rate of 148 Hz, but it is a very large model, as shown in Figure 10b.

In contrast, YOLO12 ranked third in terms of latency and FPS, as shown in Figure 10a. ConvNeXt detectors are heavy, yet moderately efficient. Furthermore, U-Net and ResNeSt-269 are too heavy and slow, rendering them inefficient for deployment. In summary, YOLO11 is the best candidate for real-world deployment on limited hardware with the best compact efficiency. Whereas U-Net, and Res2NeXt-101 will have a high compute budget.

5. Limitations

The dataset scope and diversity are the limitations. The dataset is extensive, but its diversity may be questioned. Additionally, the dataset has class imbalance, as shown in Table 1. For example: sphere1 = 5900 images vs. rollingcube = 1255 images. The class-wise results in

Table 5. Class-wise performance metrics heatmap for six microrobot detectors.

Detectors	ClassName	mAP_50–95	Precision	Recall	F1-Score
YOLO12	cube	0.806887618	0.965999001	0.976479533	0.971210994
	cylinder	0.575627952	0.913242006	0.977932636	0.944480898
	flagella	0.687789202	0.923809947	0.9971537	0.959081665
	helical	0.7488874	0.921603437	0.987639788	0.953479589
	rollingcube	0.473179148	1	0	0
	sheetrobot	0.389423904	0.844249491	0.825443706	0.834740694
	sphere1	0.58550647	0.969081873	0.966539217	0.967808875
	sphere3	0.675293773	0.948449929	0.991596639	0.969543491
YOLO11	cube	0.754743564	0.991617812	0.968089958	0.97971265
	cylinder	0.624071655	0.937022061	0.975609756	0.955926652
	flagella	0.706563798	0.988828166	0.996204934	0.992502843
	helical	0.77122878	0.98956242	0.994114185	0.99183308
	rollingcube	0.436169457	1	0.002394288	0.004777139
	sheetrobot	0.419482015	0.859680018	0.827373613	0.843217488
	sphere1	0.577313013	0.980651387	0.971365639	0.975986427
	sphere3	0.615829503	0.949005542	0.417030685	0.579434752
ConvNeXt	cube	0.969044566	0.971360382	0.999181669	0.985074627
	cylinder	0.873152435	0.988399072	0.989547038	0.988972722
	flagella	0.906403005	0.994334278	0.999051233	0.996687175
	helical	0.947784066	0.996472663	0.997645674	0.997058824
	rollingcube	0.939757824	0.995575221	0.99704579	0.996309963
	sheetrobot	0.858920813	0.954896142	0.991985203	0.97308739
	sphere1	0.588896036	0.988976378	0.988042794	0.988509366
	sphere3	0.850058556	0.99257885	0.999066293	0.995812006
ResNeSt-269	cube	0.966920018	0.957647059	0.999181669	0.977973568
	cylinder	0.877728045	0.993039443	0.994192799	0.993615786
	flagella	0.919314921	0.998104265	0.999051233	0.998577525
	helical	0.953000605	0.997648442	0.998822837	0.998235294
	rollingcube	0.93028307	0.99704579	0.99704579	0.99704579
	sheetrobot	0.878918886	0.95317131	0.991368681	0.971894832
	sphere1	0.650590539	0.992407466	0.987098804	0.989746017
	sphere3	0.849241078	0.999066293	0.999066293	0.999066293
Res2NeXt-101	cube	0.964771032	0.971360382	0.999181669	0.985074627
	cylinder	0.868212938	0.987312572	0.994192799	0.990740741
	flagella	0.890216649	0.997159091	0.999051233	0.998104265
	helical	0.948049784	0.997647059	0.998234255	0.997940571
	rollingcube	0.941604495	0.99704579	0.99704579	0.99704579
	sheetrobot	0.861603558	0.952690716	0.993218249	0.972532448
	sphere1	0.609460592	0.990834387	0.986469478	0.988647114
	sphere3	0.847412467	0.992585728	1	0.99627907
U-Net	cube	0.580823183	0.580823183	0.580823183	0.580823183
	cylinder	0.820490539	0.820490539	0.820490539	0.820490539
	flagella	0.787256718	0.787256718	0.787256718	0.787256718
	helical	0.762020171	0.762020171	0.762020171	0.762020171
	rollingcube	0.887949109	0.887949109	0.887949109	0.887949109
	sheetrobot	0.647820532	0.647820532	0.647820532	0.647820532
	sphere1	0.573726833	0.573726833	0.573726833	0.573726833
	sphere3	0.787269592	0.787269592	0.787269592	0.787269592

show that detection accuracy does not scale proportionally with the number of training instances, whether small or large. An important point to note is that even minority classes, such as rollingcube, still contain sufficient training images to train advanced object detectors to produce accurate detections. Table 5 provides evidence that heavier models, such as ConvNeXt and ResNeSt-269, achieve excellent accuracy on these smaller classes. However, for the majority classes, such as sphere1, they do not achieve higher accuracy. These performance traits highlight that the model architecture plays a more decisive role than raw sample count, particularly for microrobot detection in US imaging. The class imbalance in the dataset does not appear to affect model behavior. Despite this, it remains a limitation because YOLO based lighter models may be affected.

Patient anatomy varies from case to case, depending on probe settings or operator preference. However, we assume the findings are generalizable. Similarly, perturbation conditions on which detectors are tested are limited. In the real world, perturbation values for each type may vary across cases and operating environments. Furthermore, we used a powerful B200 GPU node; however, the availability of such computational resources may not be feasible in clinical practice, which could affect results on resource-constrained or embedded devices. Despite these limitations, this study can be used to develop technologies that improve diagnosis and treatment effectiveness.

6. Conclusions

Microrobot technology and its applications are in their infancy. However, they can transform the healthcare sector, leading to improved diagnosis, treatments, and overall quality of life. This study systematically benchmarked six state-of-the-art object detectors, including ConvNeXt, Res2NeXt-101, ResNeSt-269, U-Net, YOLO11, and YOLO12, for microrobot detection using US imaging. The detectors are evaluated not only using accuracy metrics but also through four different visual perturbations, feature-level contributions, and computational efficiency. ResNeSt-269 achieves the highest accuracy. However, YOLO-based detectors offer superior computational efficiency, proving the trade-off between precision and real-time feasibility. The findings emphasize the challenges in selecting a detector that balances detection quality with real-time and real-world implementation constraints, particularly in clinical applications.

Future research will concentrate on the development of devices that can seamlessly embed advanced detectors.