Journal of Imaging

At the plasma membrane, transmembrane receptors are at the interface between cells and their environment. They allow sensing and transduction of chemical and mechanical extracellular signals. The spatial distribution of receptors and the specific recruitment of receptor subunits to the cell membrane is crucial for the regulation of signaling and cell behavior. However, it is challenging to define what regulates such spatial patterns for receptor localization, as cell shapes are extremely diverse when cells are maintained in standard culture conditions. Bone morphogenetic protein receptors (BMPRs) are serine-threonine kinases, which build heteromeric complexes of BMPRI and II. These are especially interesting targets for receptor distribution studies, since the signaling pathways triggered by BMPR-complexes depends on their dimerization mode. They might exist as preformed complexes, or assemble upon binding of BMP, triggering cell signaling which leads to differentiation or migration. In this work we analyzed BMPR receptor distributions in single cells grown on micropatterns, which allow not only to control cell shape, but also the distribution of intracellular organelles and protein assemblies. We developed a script called ComRed (Center Of Mass Receptor Distribution), which uses center of mass calculations to analyze the shift and spread of receptor distributions according to the different cell shapes. ComRed was tested by simulating changes in experimental data showing that shift and spread of distributions can be reliably detected. Our ComRed-based analysis of BMPR-complexes indicates that receptor distribution depends on cell polarization. The absence of a coordinated internalization after addition of BMP suggests that a rapid and continual recycling of BMPRs might occur. Receptor complexes formation and localization in cells induced by BMP might yield insights into the local regulation of different signaling pathways. Full article

In this paper, a robust hybrid watermarking method based on discrete wavelet transform (DWT), discrete cosine transform (DCT), and scale-invariant feature transformation (SIFT) is proposed. Indeed, it is of prime interest to develop robust feature-based image watermarking schemes to withstand both image processing attacks and geometric distortions while preserving good imperceptibility. To this end, a robust watermark is embedded in the DWT-DCT domain to withstand image processing manipulations, while SIFT is used to protect the watermark from geometric attacks. First, the watermark is embedded in the middle band of the discrete cosine transform (DCT) coefficients of the HL1 band of the discrete wavelet transform (DWT). Then, the SIFT feature points are registered to be used in the extraction process to correct the geometric transformations. Extensive experiments have been conducted to assess the effectiveness of the proposed scheme. The results demonstrate its high robustness against standard image processing attacks and geometric manipulations while preserving a high imperceptibility. Furthermore, it compares favorably with alternative methods. Full article

Limited navigation capabilities of many current robots and UAVs restricts their applications in GPS denied areas. Large aircraft with complex navigation systems rely on a variety of sensors including radio frequency aids and high performance inertial systems rendering them somewhat resistant to GPS denial. The rapid development of computer vision has seen cameras incorporated into small drones. Vision-based systems, consisting of one or more cameras, could arguably satisfy both size and weight constraints faced by UAVs. A new generation of thermal sensors is available that are lighter, smaller and widely available. Thermal sensors are a solution to enable navigation in difficult environments, including in low-light, dust or smoke. The purpose of this paper is to present a comprehensive literature review of thermal sensors integrated into navigation systems. Furthermore, the physics and characteristics of thermal sensors will also be presented to provide insight into challenges when integrating thermal sensors in place of conventional visual spectrum sensors. Full article

Many successful variational regularization methods employed to solve linear inverse problems in imaging applications (such as image deblurring, image inpainting, and computed tomography) aim at enhancing edges in the solution, and often involve non-smooth regularization terms (e.g., total variation). Such regularization methods can be treated as iteratively reweighted least squares problems (IRLS), which are usually solved by the repeated application of a Krylov projection method. This approach gives rise to an inner–outer iterative scheme where the outer iterations update the weights and the inner iterations solve a least squares problem with fixed weights. Recently, flexible or generalized Krylov solvers, which avoid inner–outer iterations by incorporating iteration-dependent weights within a single approximation subspace for the solution, have been devised to efficiently handle IRLS problems. Indeed, substantial computational savings are generally possible by avoiding the repeated application of a traditional Krylov solver. This paper aims to extend the available flexible Krylov algorithms in order to handle a variety of edge-enhancing regularization terms, with computationally convenient adaptive regularization parameter choice. In order to tackle both square and rectangular linear systems, flexible Krylov methods based on the so-called flexible Golub–Kahan decomposition are considered. Some theoretical results are presented (including a convergence proof) and numerical comparisons with other edge-enhancing solvers show that the new methods compute solutions of similar or better quality, with increased speedup. Full article

Although prostate cancer is one of the most common causes of mortality and morbidity in advancing-age males, early diagnosis improves prognosis and modifies the therapy of choice. The aim of this study was the evaluation of a combined radiomics and machine learning approach on a publicly available dataset in order to distinguish a clinically significant from a clinically non-significant prostate lesion. A total of 299 prostate lesions were included in the analysis. A univariate statistical analysis was performed to prove the goodness of the 60 extracted radiomic features in distinguishing prostate lesions. Then, a 10-fold cross-validation was used to train and test some models and the evaluation metrics were calculated; finally, a hold-out was performed and a wrapper feature selection was applied. The employed algorithms were Naïve bayes, K nearest neighbour and some tree-based ones. The tree-based algorithms achieved the highest evaluation metrics, with accuracies over 80%, and area-under-the-curve receiver-operating characteristics below 0.80. Combined machine learning algorithms and radiomics based on clinical, routine, multiparametric, magnetic-resonance imaging were demonstrated to be a useful tool in prostate cancer stratification. Full article

Table detection is a preliminary step in extracting reliable information from tables in scanned document images. We present CasTabDetectoRS, a novel end-to-end trainable table detection framework that operates on Cascade Mask R-CNN, including Recursive Feature Pyramid network and Switchable Atrous Convolution in the existing backbone architecture. By utilizing a comparativelyightweight backbone of ResNet-50, this paper demonstrates that superior results are attainable without relying on pre- and post-processing methods, heavier backbone networks (ResNet-101, ResNeXt-152), and memory-intensive deformable convolutions. We evaluate the proposed approach on five different publicly available table detection datasets. Our CasTabDetectoRS outperforms the previous state-of-the-art results on four datasets (ICDAR-19, TableBank, UNLV, and Marmot) and accomplishes comparable results on ICDAR-17 POD. Upon comparing with previous state-of-the-art results, we obtain a significant relative error reduction of $56.36\%$, $20\%$, $4.5\%$, and $3.5\%$ on the datasets of ICDAR-19, TableBank, UNLV, and Marmot, respectively. Furthermore, this paper sets a new benchmark by performing exhaustive cross-datasets evaluations to exhibit the generalization capabilities of the proposed method. Full article

Analyzing and understanding the movement of the mitral valve is of vital importance in cardiology, as the treatment and prevention of several serious heart diseases depend on it. Unfortunately, large amounts of noise as well as a highly varying image quality make the automatic tracking and segmentation of the mitral valve in two-dimensional echocardiographic videos challenging. In this paper, we present a fully automatic and unsupervised method for segmentation of the mitral valve in two-dimensional echocardiographic videos, independently of the echocardiographic view. We propose a bias-free variant of the robust non-negative matrix factorization (RNMF) along with a window-based localization approach, that is able to identify the mitral valve in several challenging situations. We improve the average f1-score on our dataset of 10 echocardiographic videos by 0.18 to a f1-score of 0.56. Full article

In this paper, we address the problem of activity estimation in passive gamma emission tomography (PGET) of spent nuclear fuel. Two different noise models are considered and compared, namely, the isotropic Gaussian and the Poisson noise models. The problem is formulated within a Bayesian framework as a linear inverse problem and prior distributions are assigned to the unknown model parameters. In particular, a Bernoulli-truncated Gaussian prior model is considered to promote sparse pin configurations. A Markov chain Monte Carlo (MCMC) method, based on a split and augmented Gibbs sampler, is then used to sample the posterior distribution of the unknown parameters. The proposed algorithm is first validated by simulations conducted using synthetic data, generated using the nominal models. We then consider more realistic data simulated using a bespoke simulator, whose forward model is non-linear and not available analytically. In that case, the linear models used are mis-specified and we analyse their robustness for activity estimation. The results demonstrate superior performance of the proposed approach in estimating the pin activities in different assembly patterns, in addition to being able to quantify their uncertainty measures, in comparison with existing methods. Full article

Atherosclerosis is a well-known disease leading to cardiovascular events, including myocardial infarction and ischemic stroke. These conditions lead to a high mortality rate, which explains the interest in their prevention, early detection, and treatment. Molecular imaging is able to shed light on the basic pathophysiological processes, such as inflammation, that cause the progression and instability of plaque. The most common radiotracers used in clinical practice can detect increased energy metabolism (FDG), macrophage number (somatostatin receptor imaging), the intensity of cell proliferation in the area (labeled choline), and microcalcifications (fluoride imaging). These radiopharmaceuticals, especially FDG and labeled sodium fluoride, can predict cardiovascular events. The limitations of molecular imaging in atherosclerosis include low uptake of highly specific tracers, possible overlap with other diseases of the vessel wall, and specific features of certain tracers’ physiological distribution. A common protocol for patient preparation, data acquisition, and quantification is needed in the area of atherosclerosis imaging research. Full article

Nowadays, computer vision relies heavily on convolutional neural networks (CNNs) to perform complex and accurate tasks. Among them, super-resolution CNNs represent a meaningful example, due to the presence of both convolutional (CONV) and transposed convolutional (TCONV) layers. While the former exploit multiply-and-accumulate (MAC) operations to extract features of interest from incoming feature maps (fmaps), the latter perform MACs to tune the spatial resolution of the received fmaps properly. The ever-growing real-time and low-power requirements of modern computer vision applications represent a stimulus for the research community to investigate the deployment of CNNs on well-suited hardware platforms, such as field programmable gate arrays (FPGAs). FPGAs are widely recognized as valid candidates for trading off computational speed and power consumption, thanks to their flexibility and their capability to also deal with computationally intensive models. In order to reduce the number of operations to be performed, this paper presents a novel hardware-oriented algorithm able to efficiently accelerate both CONVs and TCONVs. The proposed strategy was validated by employing it within a reconfigurable hardware accelerator purposely designed to adapt itself to different operating modes set at run-time. When characterized using the Xilinx XC7K410T FPGA device, the proposed accelerator achieved a throughput of up to 2022.2 GOPS and, in comparison to state-of-the-art competitors, it reached an energy efficiency up to 2.3 times higher, without compromising the overall accuracy. Full article

Grating-based phase-contrast and dark-field imaging systems create intensity modulations that are usually modeled with sinusoidal functions to extract transmission, differential-phase shift, and scatter information. Under certain system-related conditions, the modulations become non-sinusoidal and cause artifacts in conventional processing. To account for that, we introduce a piecewise-defined periodic polynomial function that resembles the physical signal formation process, modeling convolutions of binary periodic functions. Additionally, we extend the model with an iterative expectation-maximization algorithm that can account for imprecise grating positions during phase-stepping. We show that this approach can process a higher variety of simulated and experimentally acquired data, avoiding most artifacts. Full article

Image segmentation is an essential but critical component in low level vision, image analysis, pattern recognition, and now in robotic systems. In addition, it is one of the most challenging tasks in image processing and determines the quality of the final results of the image analysis. Colour based segmentation could hence offer more significant extraction of information as compared to intensity or texture based segmentation. In this work, we propose a new local or global method for multi-label segmentation that combines a random walk based model with a direct label assignment computed using a suitable colour distance. Our approach is a semi-automatic image segmentation technique, since it requires user interaction for the initialisation of the segmentation process. The random walk part involves a combinatorial Dirichlet problem for a weighted graph, where the nodes are the pixel of the image, and the positive weights are related to the distances between pixels: in this work we propose a novel colour distance for computing such weights. In the random walker model we assign to each pixel of the image a probability quantifying the likelihood that the node belongs to some subregion. The computation of the colour distance is pursued by employing the coordinates in a colour space (e.g., RGB, XYZ, YCbCr) of a pixel and of the ones in its neighbourhood (e.g., in a 8–neighbourhood). The segmentation process is, therefore, reduced to an optimisation problem coupling the probabilities from the random walker approach, and the similarity with respect the labelled pixels. A further investigation involves an adaptive preprocess strategy using a regression tree for learning suitable weights to be used in the computation of the colour distance. We discuss the properties of the new method also by comparing with standard random walk and $k-$means approaches. The experimental results carried on the White Blood Cell (WBC) dataset and GrabCut datasets show the remarkable performance of the proposed method in comparison with state-of-the-art methods, such as normalised random walk and normalised lazy random walk, with respect to segmentation quality and computational time. Moreover, it reveals to be very robust with respect to the presence of noise and to the choice of the colourspace. Full article

This paper presents a three-color balance adjustment for color constancy correction. White balancing is a typical adjustment for color constancy in an image, but there are still lighting effects on colors other than white. Cheng et al. proposed multi-color balancing to improve the performance of white balancing by mapping multiple target colors into corresponding ground truth colors. However, there are still three problems that have not been discussed: choosing the number of target colors, selecting target colors, and minimizing error which causes computational complexity to increase. In this paper, we first discuss the number of target colors for multi-color balancing. From our observation, when the number of target colors is greater than or equal to three, the best performance of multi-color balancing in each number of target colors is almost the same regardless of the number of target colors, and it is superior to that of white balancing. Moreover, if the number of target colors is three, multi-color balancing can be performed without any error minimization. Accordingly, we propose three-color balancing. In addition, the combination of three target colors is discussed to achieve color constancy correction. In an experiment, the proposed method not only outperforms white balancing but also has almost the same performance as Cheng’s method with 24 target colors. Full article

In the context of sensor-based data analysis, the compensation of image artifacts is a challenge. When the structures of interest are not clearly visible in an image, algorithms that can cope with artifacts are crucial for obtaining the desired information. Thereby, the high variation of artifacts, the combination of different types of artifacts, and their similarity to signals of interest are specific issues that have to be considered in the analysis. Despite the high generalization capability of deep learning-based approaches, their recent success was driven by the availability of large amounts of labeled data. Therefore, the provision of comprehensive labeled image data with different characteristics of image artifacts is of importance. At the same time, applying deep neural networks to problems with low availability of labeled data remains a challenge. This work presents a data-centric augmentation approach based on generative adversarial networks that augments the existing labeled data with synthetic artifacts generated from data not present in the training set. In our experiments, this augmentation leads to a more robust generalization in segmentation. Our method does not need additional labeling and does not lead to additional memory or time consumption during inference. Further, we find it to be more effective than comparable augmentations based on procedurally generated artifacts and the direct use of real artifacts. Building upon the improved segmentation results, we observe that our approach leads to improvements of 22% in the F1-score for an evaluated detection problem. Having achieved these results with an example sensor, we expect increased robustness against artifacts in future applications. Full article

This paper investigates the usefulness of multi-fractal analysis and local binary patterns (LBP) as texture descriptors for classifying mammogram images into different breast density categories. Multi-fractal analysis is also used in the pre-processing step to segment the region of interest (ROI). We use four multi-fractal measures and the LBP method to extract texture features, and to compare their classification performance in experiments. In addition, a feature descriptor combining multi-fractal features and multi-resolution LBP (MLBP) features is proposed and evaluated in this study to improve classification accuracy. An autoencoder network and principal component analysis (PCA) are used for reducing feature redundancy in the classification model. A full field digital mammogram (FFDM) dataset, INBreast, which contains 409 mammogram images, is used in our experiment. BI-RADS density labels given by radiologists are used as the ground truth to evaluate the classification results using the proposed methods. Experimental results show that the proposed feature descriptor based on multi-fractal features and LBP result in higher classification accuracy than using individual texture feature sets. Full article

Face recognition with wearable items has been a challenging task in computer vision and involves the problem of identifying humans wearing a face mask. Masked face analysis via multi-task learning could effectively improve performance in many fields of face analysis. In this paper, we propose a unified framework for predicting the age, gender, and emotions of people wearing face masks. We first construct FGNET-MASK, a masked face dataset for the problem. Then, we propose a multi-task deep learning model to tackle the problem. In particular, the multi-task deep learning model takes the data as inputs and shares their weight to yield predictions of age, expression, and gender for the masked face. Through extensive experiments, the proposed framework has been found to provide a better performance than other existing methods. Full article

Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis. Full article

Electric Network Frequency (ENF) is embedded in multimedia recordings if the recordings are captured with a device connected to power mains or placed near the power mains. It is exploited as a tool for multimedia authentication. ENF fluctuates stochastically around its nominal frequency at $50/60$ Hz. In indoor environments, luminance variations captured by video recordings can also be exploited for ENF estimation. However, the various textures and different levels of shadow and luminance hinder ENF estimation in static and non-static video, making it a non-trivial problem. To address this problem, a novel automated approach is proposed for ENF estimation in static and non-static digital video recordings. The proposed approach is based on the exploitation of areas with similar characteristics in each video frame. These areas, called superpixels, have a mean intensity that exceeds a specific threshold. The performance of the proposed approach is tested on various videos of real-life scenarios that resemble surveillance from security cameras. These videos are of escalating difficulty and span recordings from static ones to recordings, which exhibit continuous motion. The maximum correlation coefficient is employed to measure the accuracy of ENF estimation against the ground truth signal. Experimental results show that the proposed approach improves ENF estimation against the state-of-the-art, yielding statistically significant accuracy improvements. Full article

Photoacoustic (PA) imaging combines optical excitation with ultrasonic detection to achieve high-resolution imaging of biological samples. A high-energy pulsed laser is often used for imaging at multi-centimeter depths in tissue. These lasers typically have a low pulse repetition rate, so to acquire images in real-time, only one pulse of the laser can be used per image. This single pulse necessitates the use of many individual detectors and receive electronics to adequately record the resulting acoustic waves and form an image. Such requirements make many PA imaging systems both costly and complex. This investigation proposes and models a method of volumetric PA imaging using a state-of-the-art compressed sensing approach to achieve real-time acquisition of the initial pressure distribution (IPD) at a reduced level of cost and complexity. In particular, a single exposure of an optical image sensor is used to capture an entire Fabry–Pérot interferometric acoustic sensor. Time resolved encoding as achieved through spatial sweeping with a galvanometer. This optical system further makes use of a random binary mask to set a predetermined subset of pixels to zero, thus enabling recovery of the time-resolved signals. The Two-Step Iterative Shrinking and Thresholding algorithm is used to reconstruct the IPD, harnessing the sparsity naturally occurring in the IPD as well as the additional structure provided by the binary mask. We conduct experiments on simulated data and analyze the performance of our new approach. Full article