Journal of Imaging — Open Access Journal

We present image processing algorithms for a new technique of ceramic proppant crush resistance characterization. To obtain the images of the proppant material before and after the test we used X-ray microtomography. We propose a watershed-based unsupervised algorithm for segmentation of proppant particles, as well as a set of parameters for the characterization of 3D particle size, shape, and porosity. An effective approach based on central geometric moments is described. The approach is used for calculation of particles’ form factor, compactness, equivalent ellipsoid axes lengths, and lengths of projections to these axes. Obtained grain size distribution and crush resistance fit the results of conventional test measured by sieves. However, our technique has a remarkable advantage over traditional laboratory method since it allows to trace the destruction at the level of individual particles and their fragments; it grants to analyze morphological features of fines. We also provide an example describing how the approach can be used for verification of statistical hypotheses about the correlation between particles’ parameters and their crushing under load. Full article

A system for inspecting flat panel displays (FPDs) acquires scanning images using multiline charge-coupled device (CCD) cameras and industrial machine vision. Optical filters are currently installed in front of these inspection systems to obtain high-quality images. However, the combination of optical filters required is determined manually and by using empirical methods; this is referred to as passive color control. In this study, active color control is proposed for inspecting FPDs. This inspection scheme requires the scanning of images, which is achieved using a mixed color light source and a mixing algorithm. The light source utilizes high-power light emitting diodes (LEDs) of multiple colors and a communication port to dim their level. Mixed light illuminates an active-matrix organic light-emitting diode (AMOLED) panel after passing through a beam expander and after being shaped into a line beam. The image quality is then evaluated using the Tenenbaum gradient after intensity calibration of the scanning images. The dimming levels are determined using the simplex search method which maximizes the image quality. The color of the light was varied after every scan of an AMOLED panel, and the variation was iterated until the image quality approached a local maximization. The number of scans performed was less than 225, while the number of dimming level combinations was 2048⁴. The proposed method can reduce manual tasks in setting-up inspection machines, and hence is useful for the inspection machines in FPD processes. Full article

Accurate mapping of weed distribution within a field is a first step towards effective weed management. The aim of this work was to improve the mapping of milk thistle (Silybum marianum) weed patches through unmanned aerial vehicle (UAV) images using auxiliary layers of information, such as spatial texture and estimated vegetation height from the UAV digital surface model. UAV multispectral images acquired in the visible and near-infrared parts of the spectrum were used as the main source of data, together with texture that was estimated for the image bands using a local variance filter. The digital surface model was created from structure from motion algorithms using the UAV image stereopairs. From this layer, the terrain elevation was estimated using a focal minimum filter followed by a low-pass filter. The plant height was computed by subtracting the terrain elevation from the digital surface model. Three classification algorithms (maximum likelihood, minimum distance and an object-based image classifier) were used to identify S. marianum from other vegetation using various combinations of inputs: image bands, texture and plant height. The resulting weed distribution maps were evaluated for their accuracy using field-surveyed data. Both texture and plant height have helped improve the accuracy of classification of S. marianum weed, increasing the overall accuracy of classification from 70% to 87% in 2015, and from 82% to 95% in 2016. Thus, as texture is easier to compute than plant height from a digital surface model, it may be preferable to be used in future weed mapping applications. Full article

The goal of block matching (BM) is to locate small patches of an image that are similar to a given patch or template. This can be done either in the spatial domain or, more efficiently, in a transform domain. Full search (FS) BM is an accurate, but computationally expensive procedure. Recently introduced orthogonal Haar transform (OHT)-based BM method significantly reduces the computational complexity of FS method. However, it cannot be used in applications where the patch size is not a power of two. In this paper, we generalize OHT-based BM to an arbitrary patch size, introducing a new BM algorithm based on a 2D orthonormal tree-structured Haar transform (OTSHT). Basis images of OHT are uniquely determined from the full balanced binary tree, whereas various OTSHTs can be constructed from any binary tree. Computational complexity of BM depends on a specific design of OTSHT. We compare BM based on OTSHTs to FS and OHT (for restricted patch sizes) within the framework of image denoising, using WNNM as a denoiser. Experimental results on eight grayscale test images corrupted by additive white Gaussian noise with five noise levels demonstrate that WNNM with OTSHT-based BM outperforms other methods both computationally and qualitatively. Full article

Digital tools as 3D (three-dimensional) modelling and imaging techniques are having an increasing role in many applicative fields, thanks to some significative features, such as their powerful communicative capacity, versatility of the results and non-invasiveness. These properties are very important in cultural heritage, and modern methodologies provide an efficient means for analyzing deeply and virtually rendering artworks without contact or damage. In this paper, we present two laser scanner prototypes based on the Imaging Topological Radar (ITR) technology developed at the ENEA Research Center of Frascati (RM, Italy) to obtain 3D models and IR images of medium/large targets with the use of laser sources without the need for scaffolding and independently from illumination conditions. The RGB-ITR (Red Green Blue-ITR) scanner employs three wavelengths in the visible range for three-dimensional color digitalization up to 30 m, while the IR-ITR (Infrared-ITR) system allows for layering inspection using one IR source for analyses. The functionalities and operability of the two systems are presented by showing the results of several case studies and laboratory tests. Full article

Modern agriculture is facing unique challenges in building a sustainable future for food production, in which the reliable detection of plantation threats is of critical importance. The breadth of existing information sources, and their equivalent sensors, can provide a wealth of data which, to be useful, must be transformed into actionable knowledge. Approaches based on Information Communication Technologies (ICT) have been shown to be able to help farmers and related stakeholders make decisions on problems by examining large volumes of data while assessing multiple criteria. In this paper, we address the automated identification (and count the instances) of the major threat of olive trees and their fruit, the Bactrocera Oleae (a.k.a. Dacus) based on images of the commonly used McPhail trap’s contents. Accordingly, we introduce the “Dacus Image Recognition Toolkit” (DIRT), a collection of publicly available data, programming code samples and web-services focused at supporting research aiming at the management the Dacus as well as extensive experimentation on the capability of the proposed dataset in identifying Dacuses using Deep Learning methods. Experimental results indicated performance accuracy (mAP) of 91.52% in identifying Dacuses in trap images featuring various pests. Moreover, the results also indicated a trade-off between image attributes affecting detail, file size and complexity of approaches and mAP performance that can be selectively used to better tackle the needs of each usage scenario. Full article

In many applications of tomography, the acquired data are limited in one or more ways due to unavoidable experimental constraints. In such cases, popular direct reconstruction algorithms tend to produce inaccurate images, and more accurate iterative algorithms often have prohibitively high computational costs. Using machine learning to improve the image quality of direct algorithms is a recently proposed alternative, for which promising results have been shown. However, previous attempts have focused on using encoder–decoder networks, which have several disadvantages when applied to large tomographic images, preventing wide application in practice. Here, we propose the use of the Mixed-Scale Dense convolutional neural network architecture, which was specifically designed to avoid these disadvantages, to improve tomographic reconstruction from limited data. Results are shown for various types of data limitations and object types, for both simulated data and large-scale real-world experimental data. The results are compared with popular tomographic reconstruction algorithms and machine learning algorithms, showing that Mixed-Scale Dense networks are able to significantly improve reconstruction quality even with severely limited data, and produce more accurate results than existing algorithms. Full article

In the image processing pipeline of almost every digital camera, there is a part for removing the influence of illumination on the colors of the image scene. Tuning the parameter values of an illumination estimation method for maximal accuracy requires calibrated images with known ground-truth illumination, but creating them for a given sensor is time-consuming. In this paper, the green stability assumption is proposed that can be used to fine-tune the values of some common illumination estimation methods by using only non-calibrated images. The obtained accuracy is practically the same as when training on calibrated images, but the whole process is much faster since calibration is not required and thus time is saved. The results are presented and discussed. The source code website is provided in Section Experimental Results. Full article

This article deals with visual cryptography. It consists of hiding a message in two key images (also called shares). The decryption of the message is obtained through human vision by superposition of the shares. In existing methods, the surface of key images is not visually pleasant and is not exploited for communicating textual or pictorial information. Presently, we propose a pictogram-based visual cryptography technique, which generates shares textured with customizable and aesthetic rendering. Moreover, robustness characteristics of this technique to the automated decoding of the secret message are presented. Experimental results show concrete personalized shares and their applicative potentials for security and creative domains. Full article

(1) Background: Evidence-based policymaking requires data about the local population’s socioeconomic status (SES) at detailed geographical level, however, such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however, these methods are also resource-intensive, since they require large volumes of manually labeled training data. (2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification, and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. (3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has $R^2=0.76$ and a correlation coefficient of $0.874$ with the true unemployment rate, while it achieves a mean absolute percentage error of $0.089$ and mean absolute error of $1.87$ on a held-out test set. Similar results are also obtained for other socioeconomic indicators, related to education level and occupational prestige. (4) Conclusions: The proposed methodology can be used to estimate SES indicators at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort. Full article

Congenital anomalies of the coronary ostia can lead to sudden death. A screening solution would be useful to prevent adverse outcomes for the affected individuals. To be considered for integration into clinical routine, such a procedure must meet strict constraints in terms of invasiveness, time and user interaction. Imaging must be fast and seamlessly integrable into the clinical process. Non-contrast enhanced coronary magnetic resonance angiography (MRA) is well suited for this. Furthermore, planar reformations proved effective to reduce the acquired volumetric datasets to 2D images. These usually require time consuming user interaction, though. To fulfill the aforementioned challenges, we present a fully automated solution for imaging and reformatting of the proximal coronary arteries which enables rapid screening of these. The proposed pipeline consists of: (I) highly accelerated single breath-hold MRA data acquisition, (II) coronary ostia detection and vessel centerline extraction, and (III) curved planar reformation of the proximal coronary arteries, as well as multiplanar reformation of the coronary ostia. The procedure proved robust and effective in ten volunteer data sets. Imaging of the proximal coronary arteries took $24\pm 5$ s and was successful within one breath-hold for all patients. The extracted centerlines achieve an overlap of 0.76 ± 0.18 compared to the reference standard and the average distance of the centerline points from the spherical surface for reformation was 1.1 ± 0.51 mm. The promising results encourage further experiments on patient data, particularly in coronary ostia anomaly screening. Full article

Watershed is a widespread technique for image segmentation. Many researchers apply the method implemented in open source libraries without a deep understanding of its characteristics and limitations. In the review, we describe benchmarking outcomes of six open-source marker-controlled watershed implementations for the segmentation of 2D and 3D images. Even though the considered solutions are based on the same algorithm by flooding having O(n)computational complexity, these implementations have significantly different performance. In addition, building of watershed lines grows processing time. High memory consumption is one more bottleneck for dealing with huge volumetric images. Sometimes, the usage of more optimal software is capable of mitigating the issues with the long processing time and insufficient memory space. We assume parallel processing is capable of overcoming the current limitations. However, the development of concurrent approaches for the watershed segmentation remains a challenging problem. Full article

Background Estimation is a common computer vision task, used for segmenting moving objects in video streams. This can be useful as a pre-processing step, isolating regions of interest for more complicated algorithms performing detection, recognition, and identification tasks, in order to reduce overall computation time. This is especially important in the context of embedded systems like smart cameras, which may need to process images with constrained computational resources. This work focuses on accelerating SuperBE, a superpixel-based background estimation algorithm that was designed for simplicity and reducing computational complexity while maintaining state-of-the-art levels of accuracy. We explore both software and hardware acceleration opportunities, converting the original algorithm into a greyscale, integer-only version, and using Hardware/Software Co-design to develop hardware acceleration components on FPGA fabric that assist a software processor. We achieved a 4.4× speed improvement with the software optimisations alone, and a 2× speed improvement with the hardware optimisations alone. When combined, these led to a 9× speed improvement on a Cyclone V System-on-Chip, delivering almost 38 fps on 320 × 240 resolution images. Full article

Reconstruction of photoacoustic (PA) images acquired with clinical ultrasound transducers is usually performed using the Delay and Sum (DAS) beamforming algorithm. Recently, a variant of DAS, referred to as Delay Multiply and Sum (DMAS) beamforming has been shown to provide increased contrast, signal-to-noise ratio (SNR) and resolution in PA imaging. The main reasons for the use of DAS beamforming in photoacoustics are its simple implementation, real-time capability, and the linearity of the beamformed image to the PA signal. This is crucial for the identification of different chromophores in multispectral PA applications. In contrast, current DMAS implementations are not responsive to the full spectrum of sound frequencies from a photoacoustic source and have not been shown to provide a reconstruction linear to the PA signal. Furthermore, due to its increased computational complexity, DMAS has not been shown yet to work in real-time. Here, we present an open-source real-time variant of the DMAS algorithm, signed DMAS (sDMAS), that ensures linearity in the original PA signal response while providing the increased image quality of DMAS. We show the applicability of sDMAS for multispectral PA applications, in vitro and in vivo. The sDMAS and reference DAS algorithms were integrated in the open-source Medical Imaging Interaction Toolkit (MITK) and are available as real-time capable implementations. Full article

Deceit occurs in daily life and, even from an early age, children can successfully deceive their parents. Therefore, numerous book and psychological studies have been published to help people decipher the facial cues to deceit. In this study, we tackle the problem of deceit detection by analyzing eye movements: blinks, saccades and gaze direction. Recent psychological studies have shown that the non-visual saccadic eye movement rate is higher when people lie. We propose a fast and accurate framework for eye tracking and eye movement recognition and analysis. The proposed system tracks the position of the iris, as well as the eye corners (the outer shape of the eye). Next, in an offline analysis stage, the trajectory of these eye features is analyzed in order to recognize and measure various cues which can be used as an indicator of deception: the blink rate, the gaze direction and the saccadic eye movement rate. On the task of iris center localization, the method achieves within pupil localization in 91.47% of the cases. For blink localization, we obtained an accuracy of 99.3% on the difficult EyeBlink8 dataset. In addition, we proposed a novel metric, the normalized blink rate deviation to stop deceitful behavior based on blink rate. Using this metric and a simple decision stump, the deceitful answers from the Silesian Face database were recognized with an accuracy of 96.15%. Full article

Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset (Chinese Academy of Sciences Micro-expression II) are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP (Local Binary Patterns from Three Orthogonal Planes), HOOF (Histograms of Oriented Optical Flow) and HOG 3D (3D Histogram of Oriented Gradient) feature descriptors. The experiments are evaluated on two benchmark FACS (Facial Action Coding System) coded datasets: CASME II and SAMM (A Spontaneous Micro-Facial Movement). The best result achieves 86.35% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition. Full article

High-resolution hyperspectral images are in great demand but hard to acquire due to several existing fundamental and technical limitations. A practical way around this is to fuse multiple multiband images of the same scene with complementary spatial and spectral resolutions. We propose an algorithm for fusing an arbitrary number of coregistered multiband, i.e., panchromatic, multispectral, or hyperspectral, images through estimating the endmember and their abundances in the fused image. To this end, we use the forward observation and linear mixture models and formulate an appropriate maximum-likelihood estimation problem. Then, we regularize the problem via a vector total-variation penalty and the non-negativity/sum-to-one constraints on the endmember abundances and solve it using the alternating direction method of multipliers. The regularization facilitates exploiting the prior knowledge that natural images are mostly composed of piecewise smooth regions with limited abrupt changes, i.e., edges, as well as coping with potential ill-posedness of the fusion problem. Experiments with multiband images constructed from real-world hyperspectral images reveal the superior performance of the proposed algorithm in comparison with the state-of-the-art algorithms, which need to be used in tandem to fuse more than two multiband images. Full article

Many existing natural scene statistics-based no reference image quality assessment (NR IQA) algorithms employ univariate parametric distributions to capture the statistical inconsistencies of bandpass distorted image coefficients. Here, we propose a multivariate model of natural image coefficients expressed in the bandpass spatial domain that has the potential to capture higher order correlations that may be induced by the presence of distortions. We analyze how the parameters of the multivariate model are affected by different distortion types, and we show their ability to capture distortion-sensitive image quality information. We also demonstrate the violation of Gaussianity assumptions that occur when locally estimating the energies of distorted image coefficients. Thus, we propose a generalized Gaussian-based local contrast estimator as a way to implement non-linear local gain control, which facilitates the accurate modeling of both pristine and distorted images. We integrate the novel approach of generalized contrast normalization with multivariate modeling of bandpass image coefficients into a holistic NR IQA model, which we refer to as multivariate generalized contrast normalization (MVGCN). We demonstrate the improved performance of MVGCN on quality-relevant tasks on multiple imaging modalities, including visible light image quality prediction and task success prediction on distorted X-ray images. Full article

This research presents the idea of a novel fully-Convolutional Neural Network (CNN)-based model for probabilistic pixel-wise segmentation, titled Encoder-decoder-based CNN for Road-Scene Understanding (ECRU). Lately, scene understanding has become an evolving research area, and semantic segmentation is the most recent method for visual recognition. Among vision-based smart systems, the driving assistance system turns out to be a much preferred research topic. The proposed model is an encoder-decoder that performs pixel-wise class predictions. The encoder network is composed of a VGG-19 layer model, while the decoder network uses 16 upsampling and deconvolution units. The encoder of the network has a very flexible architecture that can be altered and trained for any size and resolution of images. The decoder network upsamples and maps the low-resolution encoder’s features. Consequently, there is a substantial reduction in the trainable parameters, as the network recycles the encoder’s pooling indices for pixel-wise classification and segmentation. The proposed model is intended to offer a simplified CNN model with less overhead and higher performance. The network is trained and tested on the famous road scenes dataset CamVid and offers outstanding outcomes in comparison to similar early approaches like FCN and VGG16 in terms of performance vs. trainable parameters. Full article