SegMed: An Open-Source Desktop Tool for Deploying Pretrained Deep Learning Models in 3D Medical Image Segmentation
Featured Application
Abstract
1. Introduction
2. Methods
2.1. Software Architecture
- Images Module: This module manages medical image input and associated metadata. It stores image array, segmentation arrays (both predicted by the model and ground truth that are loaded independently), spatial information (such as voxel spacing and orientation) and maintains the association between images and segmentation masks.
- DicomSeriesSelectionDialog Module: This module provides a window for the user to choose image series from the available series in case of loading from DICOMDIR files.
- PreProcessingWindow Module: This module shows preprocessing options to the user (Figure 1A) to define which operations should be applied before inference.
- PreprocessingExecutor Module: This module executes the selected preprocessing operations and enforces compatibility constraints between preprocessing steps.
- Networks Module: This module provides a unified interface for loading, configuring and executing deep learning models implemented in either PyTorch (version 2.X) or Keras (version 2.X), and created in MONAI (version 1.X) and nnU-Net (both versions V1 and V2) frameworks.
- NetworkDimensionInputWindow Module: This module provides a panel (Figure 1B) to manually input the model configuration such as dimensions, processing type (3D or 2D) and resizing options.
- nnunetHandler and nnunetHandlerV1 Modules: These modules are designed to specifically handle the nnU-Net pipeline and are used by the Networks module only.
- Mesh Module: This module converts segmentation masks (produced by the module or loaded by the user) into surface meshes and stores them as matrices of vertices and triangles for each class.
- ColorSelectionWindow Module: This module controls visualization parameters, including class-specific colors for both 2D and 3D views, and allows users to enable or disable individual segmentation classes interactively, as well as assigning a name to each class.
- MainWindow Module: This module orchestrates the overall application workflow and user interaction (Figure 2 [23,24,25,26]). It connects all other modules and manages both 2D visualization (via matplotlib version 3.10.7) and 3D visualization (via OpenGL (version 3.1.10, Khronos Group, Beaverton, OR, USA)).
2.2. Software Functionality
2.2.1. Data Loading and Visualization
2.2.2. Image Preprocessing
2.2.3. Model Loading
- Keras/TensorFlow Models: These models can be loaded as a JSON architecture file (.json) or YAML architecture files (.yaml, .yml) with separate weights files (.h5) or a complete saved models (.h5, .keras) containing both architecture and weights in case the system employs custom object handling to manage Keras models with custom loss functions, metrics, or layers. When custom objects are detected in the model configuration, dummy functions are automatically created to enable successful loading without requiring the original custom implementations.
- PyTorch Models: These models can be TorchScript (.ts) with optional external weights files, or Checkpoint files (pt, pth,pth.tar) containing complete model state or model dictionary.
- MONAI Bundles: These bundles can be loaded after confirming the presence of required directory structure, including configs directory containing “inference.json” or “inference.yaml”; and models directory with trained model weights. The bundle loader implements a sophisticated multi-step initialization process, that uses the official Application Programming Interface (API) of MONAI.
- nnU-Net (V1 and V2) Models: Detection of these models employs automated version-specific validation, depending on the detected files. V2 detection requires: “plans.json” or “nnUNetPlans.json” and at least one “fold_X” directory containing “checkpoint_final.pth” or “checkpoint_best.pth”. Instead, V1 detection requires at least one “.pkl” file (plans, postprocessing configuration), and at least one “.model” or “model_*.pth” checkpoint file.
2.2.4. Model Inference
- Standard Keras/TensorFlow Model: Since these models store input shape properties, SegMed first applies the selected preprocessing operations and then determines whether the input image must be resized or padded with zeros to match the required model dimensions. SegMed explicitly informs the user of any structural modifications applied to the input image to match the model expectations. The preprocessed image is then passed through the network as a volume if the model expects a 3D input, or processed slice-by-slice otherwise. Finally, postprocessing involves applying rounding (in the case of binary segmentation) or the argmax operation (in the case of multiclass segmentation) to convert probability maps into label-wise segmentation masks.
- Standard PyTorch 3D Model: Image preprocessing and postprocessing match the case of the Keras/TensorFlow Model Inference. However, PyTorch models do not have the tag that describes the dimensions of the input. Thus, SegMed tries to determine the required input by passing a dummy image through the network, if it fails SegMed tries to pad the image, where each dimension is equal to the nearest greater power of 2. In cases when input dimensions cannot be reliably inferred through dynamic probing, such as in cases involving custom forward implementations or non-standard architectural constraints, the automatic configuration procedure may fail. In such instances, SegMed prompts the user to manually define the required input dimensions and processing mode.
- MONAI Bundle: For MONAI bundles, SegMed executes a full end-to-end clinical inference workflow derived from the bundle configuration. Input images are first converted to temporary NIfTI files with correctly constructed affine matrices to preserve spacing, orientation, and origin. Preprocessing follows the extracted MONAI transform; thus all pre- and post- processing are determined by the bundle itself, and user choice is ignored. Inference is performed using MONAI’s sliding window strategy using the window size described in the configuration.
- nnU-Net (v1 and v2): nnU-Net models are handled through dedicated pipelines that replicate the official inference behavior. For nnU-Net v2, SegMed uses the official nnU-Net predictor, automatically selecting the most appropriate configuration (3D full-resolution, 3D low-resolution, or 2D) based on the detected plans. Preprocessing, resampling, normalization, sliding-window inference, and postprocessing are executed exactly as defined during training. Ensemble prediction across multiple folds is supported by averaging probability maps prior to label fusion. For legacy nnU-Net v1 models, SegMed reconstructs the inference pipeline from stored pickle files, manually handling preprocessing, patch-based inference, aggregation of overlapping predictions, inverse resampling, and postprocessing, while maintaining compatibility with modern PyTorch versions and operating systems.
2.2.5. Result Visualization
2.2.6. Hardware Acceleration and Export
3. Illustrative Examples
4. Discussion
Author Contributions
Funding
Data Availability Statement
- Pretrained Models:
- MONAI Spleen CT Segmentation (v0.6.0), Pancreas CT DiNTS Segmentation (v0.5.1) and Swin UNETR BTCV Multi-organ Segmentation (v0.5.8) available via the MONAI Hugging Face Hub at https://huggingface.co/MONAI/ (accessed on 9 February 2026).
- CEREBRUM-7T brain segmentation model, available at https://github.com/rockNroll87q/cerebrum7t (accessed on 9 February 2026).
- Pretrained nnU-Net models for Hippocampus and Dental segmentation (DentalSegmentator), available on Zenodo at https://doi.org/10.5281/zenodo.4003545 and https://doi.org/10.5281/zenodo.10829675 (accessed on 9 February 2026).
- Datasets:
- Medical Segmentation Decathlon (MSD) datasets, including Spleen, Pancreas, and Hippocampus tasks, available at http://medicaldecathlon.com/ (accessed on 9 February 2026).
- EBRAINS dataset (ID: 2b24466d-f1cd-4b66-afa8-d70a6755ebea), available via EBRAINS Search Service (accessed 9 February 2026).
- ToothFairy2 dataset, available through Grand Challenge at https://toothfairy2.grand-challenge.org/ (accessed on 9 February 2026).
- Multi-Atlas Labeling Beyond the Cranial Vault (Abdomen) dataset, available via Synapse (ID: syn3193805) at https://www.synapse.org/ (accessed on 9 February 2026).
Conflicts of Interest
Abbreviations
| CPU | Central Processing Unit |
| MRI | Magnetic Resonance Imaging |
| CT | Computed Tomography |
| DICOM | Digital Imaging and Communications in Medicine |
| GUI | Graphical User Interface |
| GPU | Graphics Processing Unit |
| HU | Hounsfield Unit |
| MONAI | Medical Open Network for AI |
| NIfTI | Neuroimaging Informatics Technology Initiative |
| NRRD | Nearly Raw Raster Data |
References
- Diaz, O.; Kushibar, K.; Osuala, R.; Linardos, A.; Garrucho, L.; Igual, L.; Radeva, P.; Prior, F.; Gkontra, P.; Lekadir, K. Data preparation for artificial intelligence in medical imaging: A comprehensive guide to open-access platforms and tools. Phys. Medica 2021, 83, 25–37. [Google Scholar] [CrossRef] [PubMed]
- Hesamian, M.H.; Jia, W.; He, X.; Kennedy, P. Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. J. Digit. Imaging 2019, 32, 582–596. [Google Scholar] [CrossRef]
- Amiriebrahimabadi, M.; Rouhi, Z.; Mansouri, N. A Comprehensive Survey of Multi-Level Thresholding Segmentation Methods for Image Processing. Arch. Comput. Methods Eng. 2024, 31, 3647–3697. [Google Scholar] [CrossRef]
- Xia, Q.; Zheng, H.; Zou, H.; Luo, D.; Tang, H.; Li, L.; Jiang, B. A comprehensive review of deep learning for medical image segmentation. Neurocomputing 2025, 613, 128740. [Google Scholar] [CrossRef]
- Gao, Y.; Jiang, Y.; Peng, Y.; Yuan, F.; Zhang, X.; Wang, J. Medical Image Segmentation: A Comprehensive Review of Deep Learning-Based Methods. Tomography 2025, 11, 52. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv 2015. [Google Scholar] [CrossRef]
- Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S.S.; Brox, T.; Ronneberger, O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. arXiv 2016. [Google Scholar] [CrossRef]
- Isensee, F.; Jaeger, P.F.; Kohl, S.A.A.; Petersen, J.; Maier-Hein, K.H. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 2021, 18, 203–211. [Google Scholar] [CrossRef] [PubMed]
- Hatamizadeh, A.; Nath, V.; Tang, Y.; Yang, D.; Roth, H.R.; Xu, D. Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. arXiv 2022. [Google Scholar] [CrossRef]
- Antonelli, M.; Reinke, A.; Bakas, S.; Farahani, K.; Kopp-Schneider, A.; Landman, B.A.; Litjens, G.; Menze, B.; Ronneberger, O.; Summers, R.M.; et al. The Medical Segmentation Decathlon. Nat. Commun. 2022, 13, 4128. [Google Scholar] [CrossRef]
- Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J.S.; Freymann, J.B.; Farahani, K.; Davatzikos, C. Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 2017, 4, 170117. [Google Scholar] [CrossRef] [PubMed]
- Menze, B.H.; Jakab, A.; Bauer, S.; Kalpathy-Cramer, J.; Farahani, K.; Kirby, J.; Burren, Y.; Porz, N.; Slotboom, J.; Wiest, R.; et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imaging 2015, 34, 1993–2024. [Google Scholar] [CrossRef] [PubMed]
- Svanera, M.; Benini, S.; Bontempi, D.; Muckli, L. CEREBRUM-7T: Fast and fully volumetric brain segmentation of 7 Tesla MR volumes. Hum. Brain Mapp. 2021, 42, 5563–5580. [Google Scholar] [CrossRef] [PubMed]
- Lumetti, L.; Pipoli, V.; Bolelli, F.; Ficarra, E.; Grana, C. Enhancing Patch-Based Learning for the Segmentation of the Mandibular Canal. IEEE Access 2024, 12, 79014–79024. [Google Scholar] [CrossRef]
- Bolelli, F.; Marchesini, K.; Van Nistelrooij, N.; Lumetti, L.; Pipoli, V.; Ficarra, E.; Vinayahalingam, S.; Grana, C. Segmenting Maxillofacial Structures in CBCT Volume. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: New York, NY, USA, 2025; pp. 5238–5248. [Google Scholar] [CrossRef]
- Bolelli, F.; Lumetti, L.; Vinayahalingam, S.; Di Bartolomeo, M.; Pellacani, A.; Marchesini, K.; van Nistelrooij, N.; van Lierop, P.; Xi, T.; Liu, Y.; et al. Segmenting the Inferior Alveolar Canal in CBCT Volumes: The ToothFairy Challenge. IEEE Trans. Med. Imaging 2025, 44, 1890–1906. [Google Scholar] [CrossRef]
- Bhati, D.; Neha, F.; Amiruzzaman, M. A Survey on Explainable Artificial Intelligence (XAI) Techniques for Visualizing Deep Learning Models in Medical Imaging. J. Imaging 2024, 10, 239. [Google Scholar] [CrossRef]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv 2019. [Google Scholar] [CrossRef]
- TensorFlow Developers. TensorFlow (v2.20.0); Zenodo: Geneva, Switzerland, 2025. [Google Scholar] [CrossRef]
- Cardoso, M.J.; Li, W.; Brown, R.; Ma, N.; Kerfoot, E.; Wang, Y.; Murrey, B.; Myronenko, A.; Zhao, C.; Yang, D.; et al. MONAI: An open-source framework for deep learning in healthcare. arXiv 2022. [Google Scholar] [CrossRef]
- Kikinis, R.; Pieper, S.D.; Vosburgh, K.G. 3D Slicer: A platform for subject-specific image analysis, visualization, and clinical support. In Intraoperative Imaging and Image-Guided Therapy; Springer: Berlin/Heidelberg, Germany, 2013; pp. 277–289. [Google Scholar] [CrossRef]
- Wasserthal, J.; Breit, H.-C.; Meyer, M.T.; Pradella, M.; Hinck, D.; Sauter, A.W.; Heye, T.; Boll, D.T.; Cyriac, J.; Yang, S.; et al. TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiol. Artif. Intell. 2023, 5, e230024. [Google Scholar] [CrossRef]
- Gao, S.; Zhou, H.; Gao, Y.; Zhuang, X. BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizability. Med. Image Anal. 2023, 89, 102889. [Google Scholar] [CrossRef]
- Wu, F.; Zhuang, X. Minimizing Estimated Risks on Unlabeled Data: A New Formulation for Semi-Supervised Medical Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 6021–6036. [Google Scholar] [CrossRef] [PubMed]
- Zhuang, X.; Shen, J. Multi-scale Patch and Multi-modality Atlases for Whole Heart Segmentation of MRI. Med. Image Anal. 2016, 31, 77–87. [Google Scholar] [CrossRef] [PubMed]
- Zhuang, X. Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 41, 2933–2946. [Google Scholar] [CrossRef]
- Lorensen, W.E.; Cline, H.E. Marching cubes: A high resolution 3D surface construction algorithm. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’87); Association for Computing Machinery: New York, NY, USA, 1987; pp. 163–169. [Google Scholar] [CrossRef]
- He, Y.; Yang, D.; Roth, H.; Zhao, C.; Xu, D. DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: New York, NY, USA, 2021; pp. 5837–5846. [Google Scholar] [CrossRef]
- Xia, Y.; Liu, F.; Yang, D.; Cai, J.; Yu, L.; Zhu, Z.; Xu, D.; Yuille, A.; Roth, H. 3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training. arXiv 2020. [Google Scholar] [CrossRef]
- Tang, Y.; Yang, D.; Li, W.; Roth, H.R.; Landman, B.; Xu, D.; Nath, V.; Hatamizadeh, A. Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis. arXiv 2022. [Google Scholar] [CrossRef]
- Harrigan, R. Segmentation Outside the Cranial Vault Challenge. Synapse. 2015. Available online: https://repo-prod.prod.sagebase.org/repo/v1/doi/locate?id=syn3193805&type=ENTITY (accessed on 10 February 2026).
- Dot, G.; Chaurasia, A.; Dubois, G.; Savoldelli, C.; Haghighat, S.; Azimian, S.; Taramsari, A.R.; Sivaramakrishnan, G.; Issa, J.; Dubey, A.; et al. DentalSegmentator: Robust open source deep learning-based CT and CBCT image segmentation. J. Dent. 2024, 147, 105130. [Google Scholar] [CrossRef] [PubMed]




| Structure | Window Level (HU) | Window Width (HU) |
|---|---|---|
| Brain | 40 | 80 |
| Subdural | 50 | 130 |
| Stroke | 35 | 25 |
| Lung | −600 | 1500 |
| Mediastinum | 40 | 400 |
| Liver | 50 | 150 |
| Abdomen | 60 | 400 |
| Bone | 400 | 1800 |
| Angiography | 300 | 600 |
| Heart | 40 | 350 |
| Source | Framework | Segmentation Task | Type of Input | Number of Segmented Classes | Source of the Example Image |
|---|---|---|---|---|---|
| [28] | PyTorch using MONAI | Spleen | CT | 1 | [10] |
| [29] | PyTorch using MONAI | Pancreas and Tumor | CT | 2 | [10] |
| [13] | Keras | Brain | T1-weighted 7-Tesla MRI | 6 | [13] |
| [30] | PyTorch using MONAI | Abdomen | CT | 13 | [31] |
| [8] | PyTorch using nnU-Net v1 | Hippocampus | T1-weighted MRI | 2 | [10] |
| [32] | PyTorch using nnU-Net v2 | Dental | CT | 5 | [14,15,16] |
| Source | Training/Evaluation Protocol | Reported Test D Across Structures (%) |
|---|---|---|
| [28] | 282 training/139 testing images | 62.0 |
| [29] | 32 training/9 validation images | 96.0 |
| [13] | 120 training/4 validation/21 testing images | NR |
| [30] | 24 training/6 testing images | 82.8 |
| [8] | 5-fold cross-validation on the 195-image training set + held-out test submission | 89.7 |
| [32] | 470 training/validation images; 133 internal test images; 123 external test images | 92.2/94.2 |
| Segmentation Task | Class | D (%) | I (%) |
|---|---|---|---|
| Brain | Gray matter | 89.7 | 81.4 |
| Basal ganglia | 89.0 | 80.2 | |
| White matter | 93.1 | 87.1 | |
| Ventricles | 88.2 | 79.0 | |
| Cerebellum | 91.5 | 84.4 | |
| Brainstem | 93.0 | 87.0 | |
| Pancreas and Tumor | Pancreas | 74.7 | 59.7 |
| Tumor | 64.5 | 47.6 | |
| Spleen | Spleen | 94.4 | 89.3 |
| Abdomen | Spleen | 91.8 | 84.9 |
| Right kidney | 88.9 | 80.0 | |
| Left kidney | 90.1 | 82.0 | |
| Gallbladder | 41.1 | 25.8 | |
| Esophagus | 69.0 | 52.7 | |
| Liver | 96.8 | 93.8 | |
| Stomach | 85.3 | 74.4 | |
| Aorta | 77.6 | 63.4 | |
| Inferior vena cava | 78.4 | 64.5 | |
| Portal vein and splenic vein | 78.8 | 65.0 | |
| Pancreas | 72.7 | 57.1 | |
| Right adrenal gland | 68.4 | 51.9 | |
| Left adrenal gland | 77.8 | 63.7 | |
| Hippocampus | Anterior hippocampus | 80.4 | 67.3 |
| Posterior hippocampus | 84.0 | 72.3 | |
| Dental | Upper Skull | 61.0 | 43.9 |
| Mandible | 85.8 | 75.1 | |
| Upper Teeth | 96.1 | 92.5 | |
| Lower Teeth | 87.4 | 77.7 | |
| Mandibular Canal | 83.5 | 71.7 |
| Feature | SegMed | 3D Slicer | MONAI Label | nnU-Net (Official Implementation) |
|---|---|---|---|---|
| Standalone desktop GUI | Yes | Yes | No 1 | No 2 |
| Arbitrary pretrained model loading | Yes | Limited 3 | No | No |
| Native nnU-Net support | Yes | No | No | Yes 4 |
| Framework-agnostic (PyTorch, Keras, MONAI) | Yes | No | PyTorch only | PyTorch only |
| Multi-format input | Yes | Yes | Via host platform 5 | NIfTI only |
| Integrated quantitative evaluation | Yes | Limited | No | No |
| 3D mesh export | Yes | Yes | Via host platform | No |
| Preserves official inference pipelines | Yes | No | Partial | Yes |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Mortada, M.J.; Sbrollini, A.; Proniewska-van Dam, K.; Dam, P.M.V.; Burattini, L. SegMed: An Open-Source Desktop Tool for Deploying Pretrained Deep Learning Models in 3D Medical Image Segmentation. Appl. Sci. 2026, 16, 3490. https://doi.org/10.3390/app16073490
Mortada MJ, Sbrollini A, Proniewska-van Dam K, Dam PMV, Burattini L. SegMed: An Open-Source Desktop Tool for Deploying Pretrained Deep Learning Models in 3D Medical Image Segmentation. Applied Sciences. 2026; 16(7):3490. https://doi.org/10.3390/app16073490
Chicago/Turabian StyleMortada, Mhd Jafar, Agnese Sbrollini, Klaudia Proniewska-van Dam, Peter M. Van Dam, and Laura Burattini. 2026. "SegMed: An Open-Source Desktop Tool for Deploying Pretrained Deep Learning Models in 3D Medical Image Segmentation" Applied Sciences 16, no. 7: 3490. https://doi.org/10.3390/app16073490
APA StyleMortada, M. J., Sbrollini, A., Proniewska-van Dam, K., Dam, P. M. V., & Burattini, L. (2026). SegMed: An Open-Source Desktop Tool for Deploying Pretrained Deep Learning Models in 3D Medical Image Segmentation. Applied Sciences, 16(7), 3490. https://doi.org/10.3390/app16073490

