Journal of Imaging

This paper presents a unique approach for the dichotomy between useful and adverse variations of key-point descriptors, namely the identity and the expression variations in the descriptor (feature) space. The descriptors variations are learned from training examples. Based on labels of the training data, the equivalence relations among the descriptors are established. Both types of descriptor variations are represented by a graph embedded in the descriptor manifold. Invariant recognition is then conducted as a graph search problem. A heuristic graph search algorithm suitable for the recognition under this setup was devised. The proposed approach was tested on the FRGC v2.0, the Bosphorus and the 3D TEC datasets. It has shown to enhance the recognition performance, under expression variations, by considerable margins. Full article

Objective: The purpose of this study was to develop an automated method for performing quality control (QC) tests in magnetic resonance imaging (MRI) systems, investigate the effect of different definitions of QC parameters and its sensitivity with respect to variations in regions of interest (ROI) positioning, and validate the reliability of the automated method by comparison with results from manual evaluations. Materials and Methods: Magnetic Resonance imaging MRI used for acceptance and routine QC tests from five MRI systems were selected. All QC tests were performed using the American College of Radiology (ACR) MRI accreditation phantom. The only selection criterion was that in the same QC test, images from two identical sequential sequences should be available. The study was focused on four QC parameters: percent signal ghosting (PSG), percent image uniformity (PIU), signal-to-noise ratio (SNR), and SNR uniformity (SNRU), whose values are calculated using the mean signal and the standard deviation of ROIs defined within the phantom image or in the background. The variability of manual ROIs placement was emulated by the software using random variables that follow appropriate normal distributions. Results: Twenty-one paired sequences were employed. The automated test results for PIU were in good agreement with manual results. However, the PSG values were found to vary depending on the selection of ROIs with respect to the phantom. The values of SNR and SNRU also vary significantly, depending on the combination of the two out of the four standard rectangular ROIs. Furthermore, the methodology used for SNR and SNRU calculation also had significant effect on the results. Conclusions: The automated method standardizes the position of ROIs with respect to the ACR phantom image and allows for reproducible QC results. Full article

Nowadays, deep learning methods are employed in a broad range of research fields. The analysis and recognition of historical documents, as we survey in this work, is not an exception. Our study analyzes the papers published in the last few years on this topic from different perspectives: we first provide a pragmatic definition of historical documents from the point of view of the research in the area, then we look at the various sub-tasks addressed in this research. Guided by these tasks, we go through the different input-output relations that are expected from the used deep learning approaches and therefore we accordingly describe the most used models. We also discuss research datasets published in the field and their applications. This analysis shows that the latest research is a leap forward since it is not the simple use of recently proposed algorithms to previous problems, but novel tasks and novel applications of state of the art methods are now considered. Rather than just providing a conclusive picture of the current research in the topic we lastly suggest some potential future trends that can represent a stimulus for innovative research directions. Full article

Digital libraries offer access to a large number of handwritten historical documents. These documents are available as raw images and therefore their content is not searchable. A fully manual transcription is time-consuming and expensive while a fully automatic transcription is cheaper but not comparable in terms of accuracy. The performance of automatic transcription systems is strictly related to the composition of the training set. We propose a multi-step procedure that exploits a Keyword Spotting system and human validation for building up a training set in a time shorter than the one required by a fully manual procedure. The multi-step procedure was tested on a data set made up of 50 pages extracted from the Bentham collection. The palaeographer that transcribed the data set with the multi-step procedure instead of the fully manual procedure had a time gain of 52.54%. Moreover, a small size training set that allowed the keyword spotting system to show a precision value greater than the recall value was built with the multi-step procedure in a time equal to 35.25% of the time required for annotating the whole data set. Full article

This review concerns the challenges and perspectives of on-site non-invasive measurements applied to wall mosaics. Wall mosaics, during the centuries, decorated numerous buildings, nowadays being part of world cultural heritage. The preservation and maintenance of these valuable decorations are undoubtedly directly dependent on identifying possible problems that could affect their hidden structure. On-site non-invasive methods, using different contact or no-contact technologies, can offer support in this specific field of application. The choice of the appropriate technique or combination of different techniques depends, in general, on the depth of investigation, the resolution, the possibility to have direct contact with the surfaces or, on the contrary, limited accessibility of the wall mosaics due to their location (e.g., vaults), as well as deterioration problems, (e.g., voids, detachments, or humidity effects). This review paper provides a brief overview of selected recent studies regarding non-invasive methods applied to the analysis of wall mosaics. This review, discussing the assessment of advantages and limitations for each method here considered, also considers possible future developments of imaging techniques in this specific context for cultural heritage applications. Full article

Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) tomography provides a stack of images that represent serial slices of the sample. These images are displaced relatively to each other, and an alignment procedure is required. Traditional methods for alignment of a 3D image are based on a comparison of two adjacent slices. However, such algorithms are easily confused by anisotropy in the sample structure or even experiment geometry in the case of porous media. This may lead to significant distortions in the pore space geometry, if there are no stable fiducial marks in the frame. In this paper, we propose a new method, which meaningfully extends existing alignment procedures. Our technique allows the correction of random misalignments between slices and, at the same time, preserves the overall geometrical structure of the specimen. We consider displacements produced by existing alignment algorithms as a signal and decompose it into low and high-frequency components. Final transformations exclude slow variations and contain only high frequency variations that represent random shifts that need to be corrected. The proposed algorithm can operate with not only translations but also with arbitrary affine transformations. We demonstrate the performance of our approach on a synthetic dataset and two real FIB-SEM images of natural rock. Full article

Three-dimensional (3D) surface scans were carried out in order to determine the shapes of the upper sections of (skeletal) crania of adult Eurasian otters (Lutra lutra) from Great Britain. Landmark points were placed on these shapes using a graphical user interface (GUI) and distance measurements (i.e., the length, height, and width of the crania) were found by using the landmark points. Male otters had significantly larger skulls than females (P < 0.001). Differences in size also occurred by geographical area in Great Britain (P < 0.05). Multilevel Principal Components Analysis (mPCA) indicated that sex and geographical area explained 31.1% and 9.6% of shape variation in “unscaled” shape data and that they explained 17.2% and 9.7% of variation in “scaled” data. The first mode of variation at level 1 (sex) correctly reflected size changes between males and females for “unscaled” shape data. Modes at level 2 (geographical area) also showed possible changes in size and shape. Clustering by sex and geographical area was observed in standardized component scores. Such clustering in a cranial shape by geographical area might reflect genetic differences in otter populations in Great Britain, although other potentially confounding factors (e.g., population age-structure, diet, etc.) might also drive regional differences. This work provides a successful first test of the effectiveness of 3D surface scans and multivariate methods, such as mPCA, to study the cranial morphology of otters. Full article

With the exponential increase in new cases coupled with an increased mortality rate, cancer has ranked as the second most prevalent cause of death in the world. Early detection is paramount for suitable diagnosis and effective treatment of different kinds of cancers, but this is limited to the accuracy and sensitivity of available diagnostic imaging methods. Breast cancer is the most widely diagnosed cancer among women across the globe with a high percentage of total cancer deaths requiring an intensive, accurate, and sensitive imaging approach. Indeed, it is treatable when detected at an early stage. Hence, the use of state of the art computational approaches has been proposed as a potential alternative approach for the design and development of novel diagnostic imaging methods for breast cancer. Thus, this review provides a concise overview of past and present conventional diagnostics approaches in breast cancer detection. Further, we gave an account of several computational models (machine learning, deep learning, and robotics), which have been developed and can serve as alternative techniques for breast cancer diagnostics imaging. This review will be helpful to academia, medical practitioners, and others for further study in this area to improve the biomedical breast cancer imaging diagnosis. Full article

The measurement of night sky quality has become an important task in nature conservation. The primary device used for this task can be a calibrated digital camera. In addition, colour information can be derived from sky photography. In this paper, we provide a test on a concept to gather information about the possible sources of night sky brightness based on digital camera images. This method helps to understand changes in night sky quality due to natural and artificial changes in the environment. We demonstrate that a well-defined colour–colour diagram can differentiate between the different natural and artificial sources of night sky radiance. The colour information can be essential when interpreting long-term evolution of light pollution measurements. Full article

In this paper, a new method for the removal of Gaussian noise based on two types of prior information is described. The first type of prior information is internal, based on the similarities between the pixels in the noisy image, and the other is external, based on the index or pixel location in the image. The proposed method focuses on leveraging these two types of prior information to obtain tangible results. To this end, very similar patches are collected from the noisy image. This is done by sorting the image pixels in ascending order and then placing them in consecutive rows in a new two-dimensional image. Henceforth, a principal component analysis is applied on the patch matrix to help remove the small noisy components. Since the restored pixels are similar or close in values to those in the clean image, it is preferable to arrange them using indices similar to those of the clean pixels. Simulation experiments show that outstanding results are achieved, compared to other known methods, either in terms of image visual quality or peak signal to noise ratio. Specifically, once the proper indices are used, the proposed method achieves PSNR value better than the other well-known methods by >1.5 dB in all the simulation experiments. Full article

The move from paper to online is not only necessary for remote working, it is also significantly more sustainable. This trend has seen a rising need for the high-quality digitization of content from pages and whiteboards to sharable online material. However, capturing this information is not always easy nor are the results always satisfactory. Available scanning apps vary in their usability and do not always produce clean results, retaining surface imperfections from the page or whiteboard in their output images. CleanPage, a novel smartphone-based document and whiteboard scanning system, is presented. CleanPage requires one button-tap to capture, identify, crop, and clean an image of a page or whiteboard. Unlike equivalent systems, no user intervention is required during processing, and the result is a high-contrast, low-noise image with a clean homogenous background. Results are presented for a selection of scenarios showing the versatility of the design. CleanPage is compared with two market leader scanning apps using two testing approaches: real paper scans and ground-truth comparisons. These comparisons are achieved by a new testing methodology that allows scans to be compared to unscanned counterparts by using synthesized images. Real paper scans are tested using image quality measures. An evaluation of standard image quality assessments is included in this work, and a novel quality measure for scanned images is proposed and validated. The user experience for each scanning app is assessed, showing CleanPage to be fast and easier to use. Full article

This paper describes a methodology that extracts key morphological features from histological breast cancer images in order to automatically assess Tumour Cellularity (TC) in Neo-Adjuvant treatment (NAT) patients. The response to NAT gives information on therapy efficacy and it is measured by the residual cancer burden index, which is composed of two metrics: TC and the assessment of lymph nodes. The data consist of whole slide images (WSIs) of breast tissue stained with Hematoxylin and Eosin (H&E) released in the 2019 SPIE Breast Challenge. The methodology proposed is based on traditional computer vision methods (K-means, watershed segmentation, Otsu’s binarisation, and morphological operations), implementing colour separation, segmentation, and feature extraction. Correlation between morphological features and the residual TC after a NAT treatment was examined. Linear regression and statistical methods were used and twenty-two key morphological parameters from the nuclei, epithelial region, and the full image were extracted. Subsequently, an automated TC assessment that was based on Machine Learning (ML) algorithms was implemented and trained with only selected key parameters. The methodology was validated with the score assigned by two pathologists through the intra-class correlation coefficient (ICC). The selection of key morphological parameters improved the results reported over other ML methodologies and it was very close to deep learning methodologies. These results are encouraging, as a traditionally-trained ML algorithm can be useful when limited training data are available preventing the use of deep learning approaches. Full article

We present and compare the designs of three types of neutron microscopes for high-resolution neutron imaging. Like optical microscopes, and unlike standard neutron imaging instruments, these microscopes have both condenser and image-forming objective optics. The optics are glancing-incidence axisymmetric mirrors and therefore suitable for polychromatic neutron beams. The mirrors are designed to provide a magnification of 10 to achieve a spatial resolution of better than 10 $\mathsf{\mu }$m. The resolution of the microscopes is determined by the mirrors rather than by the L/Dratio as in conventional pinhole imaging, leading to possible dramatic improvements in the signal rate. We predict the increase in the signal rate by at least two orders of magnitude for very high-resolution imaging, which is always flux limited. Furthermore, in contrast to pinhole imaging, in the microscope, the samples are placed far from the detector to allow for a bulky sample environment without sacrificing spatial resolution. Full article

Quantitative phase imaging (QPI) techniques are widely used for the label-free examining of transparent biological samples. QPI techniques can be broadly classified into interference-based and interferenceless methods. The interferometric methods which record the complex amplitude are usually bulky with many optical components and use coherent illumination. The interferenceless approaches which need only the intensity distribution and works using phase retrieval algorithms have gained attention as they require lesser resources, cost, space and can work with incoherent illumination. With rapid developments in computational optical techniques and deep learning, QPI has reached new levels of applications. In this tutorial, we discuss one of the basic optical configurations of a lensless QPI technique based on the phase-retrieval algorithm. Simulative studies on QPI of thin, thick, and greyscale phase objects with assistive pseudo-codes and computational codes in Octave is provided. Binary phase samples with positive and negative resist profiles were fabricated using lithography, and a single plane and two plane phase objects were constructed. Light diffracted from a point object is modulated by phase samples and the corresponding intensity patterns are recorded. The phase retrieval approach is applied for 2D and 3D phase reconstructions. Commented codes in Octave for image acquisition and automation using a web camera in an open source operating system are provided. Full article

Collisionless media devoid of intrinsic stresses, for example, a dispersed phase in a multiphase medium, have a much wider variety of space-time structures and features formed in them than collisional media, for example, a carrier, gas, or liquid phase. This is a consequence of the fact that evolution in such media occurs in phase space, i.e., in a space of greater dimensions than the usual coordinate space. As a consequence, the process of the formation of features in collisionless media (clustering or vice versa, a loss of continuity) can occur primarily in the velocity space, which, in contrast to the features in the coordinate space (folds, caustics, or voids), is poorly observed directly. To identify such features, it is necessary to use visualization methods that allow us to consider, in detail, the evolution of the medium in the velocity space. This article is devoted to the development of techniques that allow visualizing the degree of anisotropy of the velocity fields of collisionless interpenetrating media. Simultaneously tracking the behavior of different fractions in such media is important, as their behavior can be significantly different. We propose three different techniques for visualizing the anisotropy of velocity fields using the example of two- and three-continuum dispersed media models. We proposed the construction of spatial distributions of eccentricity fields (scalar fields), or fields of principal directions of the velocity dispersion tensor (tensor fields). In the first case, we used some simple eccentricity functions for dispersion tensors for two fractions simultaneously, which we call surrogate entropy. In the second case, to visualize the anisotropy of the velocity fields of three fractions simultaneously, we used an ordered array (3-vector) of eccentricities for the color representation through decomposition in three basic colors. In the case of a multi-stream flow, we used cluster analysis methods to identify sections of a multi-stream flow (beams) and used glyphs to visualize the entire set of beams (vector-tensor fields). Full article

Deep learning (DL) convolutional neural networks (CNNs) have been rapidly adapted in very high spatial resolution (VHSR) satellite image analysis. DLCNN-based computer visions (CV) applications primarily aim for everyday object detection from standard red, green, blue (RGB) imagery, while earth science remote sensing applications focus on geo object detection and classification from multispectral (MS) imagery. MS imagery includes RGB and narrow spectral channels from near- and/or middle-infrared regions of reflectance spectra. The central objective of this exploratory study is to understand to what degree MS band statistics govern DLCNN model predictions. We scaffold our analysis on a case study that uses Arctic tundra permafrost landform features called ice-wedge polygons (IWPs) as candidate geo objects. We choose Mask RCNN as the DLCNN architecture to detect IWPs from eight-band Worldview-02 VHSR satellite imagery. A systematic experiment was designed to understand the impact on choosing the optimal three-band combination in model prediction. We tasked five cohorts of three-band combinations coupled with statistical measures to gauge the spectral variability of input MS bands. The candidate scenes produced high model detection accuracies for the F1 score, ranging between 0.89 to 0.95, for two different band combinations (coastal blue, blue, green (1,2,3) and green, yellow, red (3,4,5)). The mapping workflow discerned the IWPs by exhibiting low random and systematic error in the order of 0.17–0.19 and 0.20–0.21, respectively, for band combinations (1,2,3). Results suggest that the prediction accuracy of the Mask-RCNN model is significantly influenced by the input MS bands. Overall, our findings accentuate the importance of considering the image statistics of input MS bands and careful selection of optimal bands for DLCNN predictions when DLCNN architectures are restricted to three spectral channels. Full article

With increased use of light-weight materials with low factors of safety, non-destructive testing becomes increasingly important. Thanks to the advancement of infrared camera technology, pulse thermography is a cost efficient way to detect subsurface defects non-destructively. However, currently available evaluation algorithms have either a high computational cost or show poor performance if any geometry other than the most simple kind is surveyed. We present an extension of the thermographic signal reconstruction technique which can automatically segment and image defects from sound areas, while also estimating the defect depth, all with low computational cost. We verified our algorithm using real world measurements and compare our results to standard active thermography algorithms with similar computational complexity. We found that our algorithm can detect defects more accurately, especially when more complex geometries are examined. Full article

Recently, our world witnessed major events that attracted a lot of attention towards the importance of automatic crowd scene analysis. For example, the COVID-19 breakout and public events require an automatic system to manage, count, secure, and track a crowd that shares the same area. However, analyzing crowd scenes is very challenging due to heavy occlusion, complex behaviors, and posture changes. This paper surveys deep learning-based methods for analyzing crowded scenes. The reviewed methods are categorized as (1) crowd counting and (2) crowd actions recognition. Moreover, crowd scene datasets are surveyed. In additional to the above surveys, this paper proposes an evaluation metric for crowd scene analysis methods. This metric estimates the difference between calculated crowed count and actual count in crowd scene videos. Full article

Current point cloud extraction methods based on photogrammetry generate large amounts of spurious detections that hamper useful 3D mesh reconstructions or, even worse, the possibility of adequate measurements. Moreover, noise removal methods for point clouds are complex, slow and incapable to cope with semantic noise. In this work, we present body2vec, a model-based body segmentation tool that uses a specifically trained Neural Network architecture. Body2vec is capable to perform human body point cloud reconstruction from videos taken on hand-held devices (smartphones or tablets), achieving high quality anthropometric measurements. The main contribution of the proposed workflow is to perform a background removal step, thus avoiding the spurious points generation that is usual in photogrammetric reconstruction. A group of 60 persons were taped with a smartphone, and the corresponding point clouds were obtained automatically with standard photogrammetric methods. We used as a 3D silver standard the clean meshes obtained at the same time with LiDAR sensors post-processed and noise-filtered by expert anthropological biologists. Finally, we used as gold standard anthropometric measurements of the waist and hip of the same people, taken by expert anthropometrists. Applying our method to the raw videos significantly enhanced the quality of the results of the point cloud as compared with the LiDAR-based mesh, and of the anthropometric measurements as compared with the actual hip and waist perimeter measured by the anthropometrists. In both contexts, the resulting quality of body2vec is equivalent to the LiDAR reconstruction. Full article