Search Results (142)

Imaging-based spatial transcriptomics has advanced from low-plex single-molecule fluorescence in situ hybridization to a diverse set of highly multiplexed platforms, with recent multimodal and pathology-compatible capabilities. Despite major differences in chemistry, coding, and imaging strategies across different platforms, their biological interpretation often converges on a few notable computational biology problems. This review examines imaging-based spatial transcriptomics through the lens of data interpretation and applications, focusing on the analytical framework that converts raw fluorescence signals or accompanying in situ sequencing data into molecule-, cell-, and tissue-level representations. We discuss the key challenges in preprocessing, registration, restoration, feature detection, barcode decoding, molecule calling, cell segmentation, transcript assignment, probabilistic cell typing, spatial-domain inference, and atlas integration. We also highlight how optical crowding, tissue thickness, panel bias, and multimodal complexity increase computational difficulty. Finally, we summarize applications of imaging-based spatial transcriptomics techniques, ranging from subcellular RNA localization to atlas-scale and pathology-aware spatial analysis. Full article

Introduction: Neurosurgery has evolved from an anatomy-driven analog discipline into a digitally augmented field supported by multimodal imaging, neuronavigation, intraoperative imaging, neurophysiological monitoring, robotics, augmented reality, and artificial intelligence. Objective: To examine how this transition has altered professional responsibility, informed consent, training, and medico-legal accountability in neurosurgical practice. Methods: We performed a structured narrative review of the literature on digital neurosurgery and its ethical and professional implications, focusing on publications from 1990 onward and supplemented by landmark historical papers. Sources were selected for relevance to cranial, spinal, skull base, stereotactic, and neuro-oncological neurosurgery, and then synthesized into thematic domains including brain shift, eloquent cortex preservation, stereotactic accuracy, intraoperative neurophysiology, workflow integration, equity, and liability. Results: Digital systems improve lesion localization, function-preserving surgery, stereotactic precision, documentation, and training, but they also introduce new vulnerabilities related to registration error, brain shift, platform dependence, data overload, cost, cybersecurity, deskilling, and diffuse accountability. Conclusions: Digital augmentation expands rather than diminishes the neurosurgeon’s responsibility. The neurosurgeon remains accountable for surgical indication, interpretation of technology-generated information, intraoperative override, and communication of technology-specific risks. The central ethical challenge is to integrate digital tools without weakening patient-centered judgment. Full article

Infrared thermography provides key information for a wide range of diagnostic applications within built and natural environments. As thermal states are changing with ambient conditions, it is important to deploy thermal imaging systems and operators opportunistically. It is therefore an attractive proposition to make these systems more affordable and accessible. Low-cost thermal sensors generally produce low-resolution outputs. To increase data density across large subjects, diagnosticians may create image mosaics from multiple overlapping thermographs. The registration of individual inputs into large mosaics is aided by the acquisition of additional sensor data (photographs and depthmaps), which can provide critical spatial references. In many cases, the materials inherent to the modern built environment present challenges to traditional data registration workflows between multiple sensor streams. Mobile devices offer an opportunity to innovate in the creation of these mosaics, integrating rapid geospatial mapping functionality with radiometric thermography within a 3D context. In this paper the authors evaluate the FLIR One Pro thermal camera module along with iOS/iPhone specific rapid mapping capabilities, and present a methodology: (1) introducing a workflow for the integration of short-range (within 0.3–5 m capture distance) iPhone mobile sensor data into modeling pipelines; (2) introducing a calibration model enabling effective registration and fusion of multi-modal inputs from the iPhone mobile sensor stack and FLIR One thermographic module; and (3) detailing an alternative open-source methodology for the evaluation and translation of thermographic imagery for multi-sensor fusion. The end product of this pipeline is a 3D radiometric thermographic mosaic: a spatially continuous, textured surface model in which hundreds of individual low-resolution thermographs are fused into a single queryable output retaining full 16-bit temperature values at every point. All datasets have been made openly available and the two case studies used in this paper have been made accessible at full resolution for interactive 3D online viewing. Full article

To address the issues of difficult heterogeneous image registration and low segmentation accuracy caused by the severe lack of illumination and significant modal differences in concrete cracks in extremely dark environments, this paper proposes a two-stage processing framework of registration–fusion first, and decoupling–segmentation later. In the registration and fusion stage, a registration algorithm based on morphological priors and multi-level quadtree spatial constraints is designed. This approach transforms the problem from pixel grayscale matching to spatial topological matching, achieving a feature fusion of high infrared saliency and high visible light sharpness. In the segmentation stage, a Latent Frequency-Decoupled Topological Network (LFDT-Net) is proposed. It utilizes Discrete Wavelet Transform (DWT) to achieve high-fidelity frequency decoupling of the low-frequency infrared backbone and the high-frequency visible light edges. Furthermore, a Cross-Frequency Guidance Module is utilized to eliminate double-edged artifacts, and a skeleton-aware topological loss function is introduced to constrain the topological integrity of the cracks. Experimental results on a self-built heterogeneous multi-modal crack dataset demonstrate that the proposed method significantly outperforms existing mainstream methods in registration accuracy, fusion quality, and segmentation accuracy. Achieving a mean Intersection over Union (mIoU) of 81.7%, the method effectively suppresses background noise in dark environments and precisely restores the microscopic edges and continuous topological structures of faint cracks. Full article

Synthetic Aperture Radar (SAR) and optical image fusion technology plays a crucial role in remote sensing applications. It effectively combines the high spatial resolution and rich spectral information of optical images with the all-weather and penetrating observation advantages of SAR images, thereby significantly enhancing image interpretation accuracy and task execution capabilities. This paper systematically reviews deep learning-based fusion methods for SAR and optical images, with a particular focus on recent advances in deep learning models. Furthermore, it summarizes commonly used evaluation metrics for assessing fusion image quality, providing a basis for comparing and analyzing the performance of different methods. In addition, commonly used SAR-optical fusion datasets are briefly reviewed to highlight their roles in algorithm development and performance evaluation. Unlike conventional review articles, this paper further analyzes the guidance and supporting role of fusion algorithms from the perspective of typical and specific applications. Finally, it identifies key challenges and issues faced by current fusion methods, including data registration, model lightweight design, and multimodal feature alignment, and offers perspectives on future research directions. This review aims to provide routes and references for the development of SAR and optical image fusion technology. Full article

Background/Objective: Cardiac innervation plays a critical role in regulating myocardial function and enabling the heart to adapt to physiological and pathological conditions. Although the general features of sympathetic and parasympathetic innervation of the myocardium are well described, the spatial organisation of nerve fibres within the cardiac muscle remains incompletely characterised. This study aimed to develop and validate the SKUF (Slice–Keep–Unwrap–Fuse) protocol, a multimodal framework for mapping myocardial innervation through the integration of histological data and magnetic resonance imaging (MRI). Methods: The study was performed on the heart of a 7-year-old patient who died from rupture of a cerebral vascular malformation without evidence of cardiovascular disease. Prior to histological processing, post-mortem MRI was performed to provide a precise anatomical reference. The heart was sectioned into sequential transverse rings of 4 mm thickness, yielding 71 paraffin blocks. Histological sections (3 μm) were immunostained with antibodies against UCHL-1 to visualise nerve fibres and scanned using an Aperio AT2 system (20× magnification). Automated image analysis was conducted using the SVSSlide Processor module, which included tissue segmentation, colour-based nerve fibre detection, and sliding-window density mapping. Heatmaps were assembled into ring-based myocardial reconstructions and co-registered with MRI slices using combined rigid and deformable registration, followed by three-dimensional reconstruction of innervation patterns. Results: A higher density of nerve fibres was observed in the right ventricular myocardium compared with the left ventricle, whereas larger nerve trunks were identified in the epicardium of the left ventricle. Quantitative analysis revealed a pronounced longitudinal gradient of innervation, with minimal density in the apical region and progressive increases towards the mid-ventricular segments, where maximal density and spatial organisation of neural structures were observed. The atrioventricular groove exhibited the greatest heterogeneity of innervation due to the presence of large nerve trunks and ganglionated plexuses. Integration of histological maps with MRI enabled three-dimensional visualisation of spatial clusters of nerve fibres. Conclusions: The SKUF protocol provides a robust framework for integrating histological and MRI data to generate three-dimensional maps of myocardial innervation. This approach may facilitate the development of high-resolution anatomical atlases of cardiac innervation and support future studies of neurocardiac mechanisms of arrhythmogenesis and targeted neuromodulation. Full article

Infrared thermography is a non-contact tool for monitoring inflammatory processes in the diabetic foot, but quantitative bedside use remains challenging with low-cost thermal infrared cameras due to radiometric drift, non-uniformity (vignetting), geometric distortions, and visible–thermal parallax. This paper presents an end-to-end clinical and instrumental framework built around a cheap thermal camera to ensure reproducible acquisition and physically consistent temperature estimation. The approach combines a standardized mobile acquisition setup and measurement protocol, extraction of embedded radiometric data from raw images, radiometric inversion with atmospheric correction, vignette correction performed in the radiometric domain, and geometric calibration of both visible and infrared sensors using dedicated (thermal) calibration targets. Accurate visible–infrared registration is obtained from hybrid heated markers, enabling reliable overlay and downstream analysis. The full processing chain yields quantitative thermograms with radiometric errors below 0.15 °C and sub-pixel multimodal alignment, supporting the detection of clinically relevant plantar temperature asymmetries and paving the way for routine calibrated low-cost thermography in diabetic foot care. Full article

This paper introduces a hybrid two-stage registration framework for reconstructing three-dimensional (3D) kidney anatomy from macroscopic slices, using CT-derived models as the geometric reference standard. The approach addresses the data-scarcity and high-distortion challenges typical of macroscopic imaging, where fully learning-based registration (e.g., VoxelMorph) often fails to generalize due to limited training diversity and large nonrigid deformations that exceed the capture range of unconstrained convolutional filters. In the proposed pipeline, the Optimal Cross-section Matching (OCM) algorithm first performs constrained global alignment—translation, rotation, and uniform scaling—to establish anatomically consistent slice initialization. Next, a lightweight deep-learning refinement network, inspired by VoxelMorph, predicts residual local deformations between consecutive slices. The core novelty of this architecture lies in its hierarchical decomposition of the registration manifold: the OCM acts as a deterministic geometric anchor that neutralizes high-amplitude variance, thereby constraining the learning task to a low-dimensional residual manifold. This hybrid OCM + DL design integrates explicit geometric priors with the flexible learning capacity of neural networks, ensuring stable optimization and plausible deformation fields even with few training examples. Experiments on an original dataset of 40 kidneys demonstrated that the OCM + DL method achieved the highest registration accuracy across all evaluated metrics: NCC = 0.91, SSIM = 0.81, Dice = 0.90, IoU = 0.81, HD95 = 1.9 mm, and volumetric agreement ${\mathrm{DC}}_{\mathrm{Vol}}$ = 0.89. Compared to single-stage baselines, this represents an average improvement of approximately 17% over DL-only and 14% over OCM-only, validating the synergistic contribution of the proposed hybrid strategy over standalone iterative or data-driven methods. The pipeline maintains physical calibration via Hough-based grid detection and employs Bézier-based contour smoothing for robust meshing and volume estimation. Although validated on kidney data, the proposed framework generalizes to other soft-tissue organs reconstructed from optical or photographic cross-sections. By decoupling interpretable global optimization from data-efficient deep refinement, the method advances the precision, reproducibility, and anatomical realism of multimodal 3D reconstructions for surgical planning, morphological assessment, and medical education. Full article

Digital elevation models (DEMs) play a critical role in geospatial analysis and surface modeling. However, due to differences in data collection payload, data processing methodology, and data reference baseline, DEMs acquired from various sources often exhibit systematic spatial offsets. This limitation substantially constrains their accuracy and reliability in multi-source joint analysis and fusion applications. Traditional registration methods such as the Least-Z Difference (LZD) method are sensitive to gross errors, while multimodal registration approaches overlook the importance of elevation information. To address these challenges, this paper proposes a DEM registration method based on physically consistent rendering and multimodal image matching. The approach converts DEMs into image data through irradiance-based models and parallax geometric models. Feature point pairs are extracted using template-based matching techniques and further refined through elevation consistency analysis. Reliable correspondences are selected by jointly considering elevation error distributions and geometric consistency constraints, enabling robust affine transformation estimation and elevation bias correction. The experimental results demonstrate that in typical terrains such as urban areas, glaciers, and plains, the proposed method outperforms classical DEM registration algorithms and state-of-the-art remote sensing image registration algorithms. The results indicate clear advantages in registration accuracy, robustness, and adaptability to diverse terrain conditions, highlighting the potential of the proposed framework as a universal DEM collaborative registration solution. Full article

Accurate registration of optical and synthetic aperture radar (SAR) images is a fundamental prerequisite for multi-source remote sensing data fusion and analysis. However, due to the substantial differences in imaging mechanisms, optical–SAR image pairs often exhibit significant radiometric discrepancies and spatially varying geometric inconsistencies, which severely limit the robustness of traditional feature or region-based registration methods in cross-modal scenarios. To address these challenges, this paper proposes an end-to-end Optical–SAR Registration Network (OSR-Net) based on multi-constraint joint optimization. The proposed framework explicitly decouples cross-modal feature alignment and geometric correction, enabling robust registration under large appearance variation. Specifically, a multi-modal feature extraction module constructs a shared high-level representation, while a multi-scale channel attention mechanism adaptively enhances cross-modal feature consistency. A multi-scale affine transformation prediction module provides a coarse-to-fine geometric initialization, which stabilizes parameter estimation under complex imaging conditions. Furthermore, an improved spatial transformer network is introduced to perform structure-preserving geometric refinement, mitigating spatial distortion induced by modality discrepancies. In addition, a multi-constraint loss formulation is designed to jointly enforce geometric accuracy, structural consistency, and physical plausibility. By employing a dynamic weighting strategy, the optimization process progressively shifts from global alignment to local structural refinement, effectively preventing degenerate solutions and improving robustness. Extensive experiments on public optical–SAR datasets demonstrate that the proposed method achieves accurate and stable registration across diverse scenes, providing a reliable geometric foundation for subsequent multi-source remote sensing data fusion. Full article

Accurate and interpretable staging of Alzheimer’s disease (AD) remains challenging due to the heterogeneous progression of neurodegeneration and the complementary nature of imaging and clinical biomarkers. This study implements and evaluates an optimized Hierarchical Multi-Modal Fusion Framework (HMFF) that systematically integrates 3D structural MRI with clinical assessment scales for robust three-class classification of cognitively normal (CN), mild cognitive impairment (MCI), and AD subjects. A standardized preprocessing pipeline, including N4 bias field correction, nonlinear registration to MNI space, ANTsNet-based skull stripping, voxel normalization, and spatial resampling, was employed to ensure anatomically consistent and high-quality MRI inputs. Within the proposed framework, volumetric imaging features were extracted using a 3D DenseNet-121 architecture, while structured clinical information was modeled via an XGBoost classifier to capture nonlinear clinical priors. These heterogeneous representations were hierarchically fused through a lightweight multilayer perceptron, enabling effective cross-modal interaction. To further enhance discriminative capability and model efficiency, a hierarchical feature selection strategy was incorporated to progressively refine high-dimensional imaging features. Experimental results demonstrated that performance consistently improved with feature refinement and reached an optimal balance at approximately 90 selected features. Under this configuration, the proposed HMFF achieved an accuracy of 0.94 (95% Confidence Interval: [0.918, 0.951]), a recall of 0.91, a precision of 0.94, and an F1-score of 0.92, outperforming unimodal and conventional multimodal baselines under comparable settings. Moreover, Grad-CAM visualization confirmed that the model focused on clinically relevant neuroanatomical regions, including the hippocampus and medial temporal lobe, enhancing interpretability and clinical plausibility. These findings indicate that hierarchical multimodal fusion with interpretable feature refinement offers a promising and extensible solution for reliable and explainable automated AD staging. Full article

Magnetic resonance imaging (MRI) and hyperspectral imaging (HSI) provide complementary information for image-guided neurosurgery, combining high-resolution anatomical detail with tissue-specific optical characterization. This work presents a novel multimodal phantom dataset specifically designed for MRI–HSI integration. The phantoms reproduce a three-layer tissue structure comprising white matter, gray matter, tumor, and superficial blood vessels, using agar-based compositions that mimic MRI contrasts of the rat brain while providing consistent hyperspectral signatures. The dataset includes two designs of phantoms with MRI, HSI, RGB-D, and tracking acquisitions, along with pixel-wise labels and corresponding 3D models, comprising 13 phantoms in total. The dataset facilitates the evaluation of registration, segmentation, and classification algorithms, as well as depth estimation, multimodal fusion, and tracking-to-camera calibration procedures. By providing reproducible, labeled multimodal data, these phantoms reduce the need for animal experiments in preclinical imaging research and serve as a versatile benchmark for MRI–HSI integration and other multimodal imaging studies. Full article

Establishing digital twin scenes facilitates the understanding of geospatial phenomena, representing a significant research focus for GIS scientists and engineers. However, current research on digital twin scenes modeling relies on manual intervention or the overlay of static models, resulting in low modeling efficiency and poor standardization. To address these challenges, this paper proposes a knowledge graph-guided and multimodal data fusion-driven rapid modeling method for digital twin scenes, using bridge tower construction as an illustrative example. We first constructed a knowledge graph linking the three domains of “event-object-data” in bridge tower construction. Guided by this graph, we designed a knowledge graph-guided multimodal data association and fusion algorithm. Then a rapid modeling method for bridge tower construction scenes based on dynamic data was established. Finally, a prototype system was developed, and a case study area was selected for analysis. Experimental results show that the knowledge graph we built clearly captures all elements and their relationships in bridge tower construction scenes. Our method enables precise fusion of 5 types of multimodal data: BIM, DEM, images, videos, and point clouds. It improves spatial registration accuracy by 21.83%, increases temporal fusion efficiency by 65.6%, and reduces feature fusion error rates by 70.9%. Local updates of the 3D geographic scene take less than 30 ms, supporting millisecond-level digital twin modeling. This provides a practical reference for building geographic digital twin scenes. Full article

With the rapid development of medical imaging technology, the early diagnosis and treatment of heart disease have been significantly improved. Cardiac CT (Computed Tomography) and ultrasound images are often used in combination to provide more comprehensive information on cardiac structure and function due to their respective advantages and limitations. However, due to the significant differences in imaging principles, resolutions, and viewing angles between these two imaging modalities, how to effectively register cardiac CT and ultrasound images has become an important research topic in imaging and clinical applications. This article summarizes the research progress of cardiac CT and ultrasound image registration, and analyzes the existing registration methods and their advantages and disadvantages. Firstly, this article summarizes traditional registration methods based on image intensity, feature points, and regions, and explores the application of rigid and non-rigid registration algorithms. Secondly, in view of common challenges in cardiac CT and ultrasound image registration, such as image noise, deformation, and differences in imaging time, this article discusses the recent advances in multimodal registration technology in cardiac imaging and forecasts the potential of deep learning methods in registration. In addition, this article also evaluates the application effects and limitations of these methods in clinical practice, and finally looks forward to the future development direction of cardiac image registration technology, especially its potential applications in personalized medicine and real-time monitoring. Through a comprehensive review of the current research status of cardiac CT and ultrasound image registration, this article provides a systematic theoretical framework for researchers in related fields and provides a reference for future technological breakthroughs and clinical translation. Full article

Emerging artificial intelligence (AI) and robotic surgical technologies have the potential to influence digital implant dentistry substantially. As a narrative review, and building on the foundations outlined in Part I, which described current digital tools and workflows alongside their persistent interface-related limitations, this second part examines how AI and robotics may overcome these barriers. This synthesis is based on peer-reviewed literature published between 2020 and 2025, identified through searches in PubMed, Scopus, and Web of Science. Current evidence suggests that AI-based approaches, including rule-based systems, traditional machine learning, and deep learning, may achieve expert-level performance in diagnostic imaging, multimodal data registration, virtual patient model generation, implant planning, prosthetic design, and digital smile design. These methods offer substantial improvements in efficiency, reproducibility, and accuracy while reducing reliance on manual data handling across software, datasets, and workflow interfaces. In parallel, robotic-assisted implant surgery has advanced from surgeon-guided systems to semi-autonomous and fully autonomous platforms, with the potential to provide enhanced surgical precision and reduce operator dependency compared with conventional static or dynamic navigation. Several of these technologies have already reached early stages of clinical deployment, although important challenges remain regarding interoperability, standardization, validation, and the continuing need for human oversight. Together, these innovations may enable the gradual convergence of digital technologies, real-time-assisted, unified, end-to-end implant prosthodontic workflows, and gradual automation, while acknowledging that full automation remains a longer-term prospect. By synthesizing current evidence and proof-of-concept applications, this review aims to provide clinicians with a comprehensive overview of the AI and robotics toolkit relevant to implant dentistry and to outline both the opportunities and remaining limitations of these disruptive technologies as the field progresses towards seamless, fully integrated treatment pathways. Full article