Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling

Zhang, Tiyao; Yuan, Xue; Xu, Hongze

doi:10.3390/jimaging11100364

This is an early access version, the complete PDF, HTML, and XML versions will be available soon.

Open AccessCommunication

Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling

by

Tiyao Zhang

,

Xue Yuan

^* and

Hongze Xu

School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China

^*

Author to whom correspondence should be addressed.

J. Imaging 2025, 11(10), 364; https://doi.org/10.3390/jimaging11100364

Submission received: 9 September 2025 / Revised: 11 October 2025 / Accepted: 13 October 2025 / Published: 15 October 2025

(This article belongs to the Section Image and Video Processing)

Download Versions Notes

Abstract

Accurate segmentation of surgical instruments in endoscopic videos is crucial for robot-assisted surgery and intraoperative analysis. This paper presents a Segment-then-Classify framework that decouples mask generation from semantic classification to enhance spatial completeness and temporal stability. First, a Mask2Former-based segmentation backbone generates class-agnostic instance masks and region features. Then, a bounding box-guided instance-level spatiotemporal modeling module fuses geometric priors and temporal consistency through a lightweight transformer encoder. This design improves interpretability and robustness under occlusion and motion blur. Experiments on the EndoVis 2017 and 2018 datasets demonstrate that our framework achieves mIoU improvements of 3.06%, 2.99%, and 1.67% and mcIoU gains of 2.36%, 2.85%, and 6.06%, respectively, over previously state-of-the-art methods, while maintaining computational efficiency.

Keywords: surgical instrument segmentation; segment-then-classify framework; instance-level spatiotemporal consistency modeling

Share and Cite

MDPI and ACS Style

Zhang, T.; Yuan, X.; Xu, H. Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling. J. Imaging 2025, 11, 364. https://doi.org/10.3390/jimaging11100364

AMA Style

Zhang T, Yuan X, Xu H. Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling. Journal of Imaging. 2025; 11(10):364. https://doi.org/10.3390/jimaging11100364

Chicago/Turabian Style

Zhang, Tiyao, Xue Yuan, and Hongze Xu. 2025. "Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling" Journal of Imaging 11, no. 10: 364. https://doi.org/10.3390/jimaging11100364

APA Style

Zhang, T., Yuan, X., & Xu, H. (2025). Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling. Journal of Imaging, 11(10), 364. https://doi.org/10.3390/jimaging11100364

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Surgical Instrument Segmentation via Segment-then-Classify Framework with Instance-Level Spatiotemporal Consistency Modeling

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI