Search Results (22)

Pansharpening is a traditional image fusion problem where the reference image (or ground truth) is not accessible. Machine-learning-based algorithms designed for this task require an extensive optimization phase of network parameters, which must be performed using unsupervised learning techniques. The learning phase can either rely on a companion problem where ground truth is available, such as by reproducing the task at a lower scale or using a pretext task, or it can use a reference-free cost function. This study focuses on the latter approach, where performance depends not only on the accuracy of the quality measure but also on the mathematical properties of these measures, which may introduce challenges related to computational complexity and optimization. The evaluation of the most recognized no-reference image quality measures led to the proposal of a novel criterion, the Regression-based QNR (RQNR), which has not been previously used. To mitigate computational challenges, an approximate version of the relevant indices was employed, simplifying the optimization of the cost functions. The effectiveness of the proposed cost functions was validated through the reduced-resolution assessment protocol applied to a public dataset (PairMax) containing images of diverse regions of the Earth’s surface. Full article

In low-dose cone beam computed tomography (CT), the insufficient number of photons inevitably results in noise, which reduces the accuracy of disease diagnosis. One approach to improving the image quality of CT images acquired using a low-dose protocol involves the utilization of a reconstruction algorithm that efficiently reduces noise. In this study, we modeled the Feldkamp–Davis–Kress (FDK) algorithm using various filters and projection angles and applied it to the reconstruction process using CT simulation. To quantitatively evaluate the quality of the reconstruction images, we measured the coefficient of variation (COV), and signal-to-noise ratio (SNR) in the air, brain, and bone regions to evaluate the noise level. Furthermore, we calculated root mean square error (RMSE), universal image quality index (UQI), and blind/referenceless image spatial quality evaluator (BRISQUE) as similarity and no-reference evaluation. The Hann filter of the FDK algorithm showed superior performance in terms of COV, SNR, RMSE, and UQI compared to the other filters. In addition, when analyzing the COV and SNR results, we observed that image quality increased significantly at projection angles smaller than approximately 2.8°. Moreover, based on BRISQUE results, we confirm that the Shepp–Logan filter exhibited the most superior performance. In conclusion, we believe that the application of the Hann filter in the FDK reconstruction process offers significant advantages in improving the image quality acquired under a low-dose protocol, and we expect that our study will be a preliminary study of no-reference evaluation of CT reconstruction images. Full article

In hyperspectral image (HSI) reconstruction tasks, due to the lack of ground truth in real imaging processes, models are usually trained and validated on simulation datasets and then tested on real measurements captured by real HSI imaging systems. However, due to the gap between the simulation imaging process and the real imaging process, the best model validated on the simulation dataset may fail on real measurements. To obtain the best model for the real-world task, it is crucial to design a suitable no-reference HSI quality assessment metric to reflect the reconstruction performance of different models. In this paper, we propose a novel no-reference HSI quality assessment metric via ranking feature learning (R-NHSIQA), which calculates the Wasserstein distance between the distribution of the deep features of the reconstructed HSIs and the benchmark distribution. Additionally, by introducing the spectral self-attention mechanism, we propose a Spectral Transformer (S-Transformer) to extract the spatial-spectral representative deep features of HSIs. Furthermore, to extract quality-sensitive deep features, we use quality ranking as a pre-training task to enhance the representation capability of the S-Transformer. Finally, we introduce the Wasserstein distance to measure the distance between the distribution of the deep features and the benchmark distribution, improving the assessment capacity of our method, even with non-overlapping distributions. The experimental results demonstrate that the proposed metric yields consistent results with multiple full-reference image quality assessment (FR-IQA) metrics, validating the idea that the proposed metric can serve as a substitute for FR-IQA metrics in real-world tasks. Full article

Unmanned Aerial Vehicle (UAV) inspection of transmission channels in mountainous areas is susceptible to non-homogeneous fog, such as up-slope fog and advection fog, which causes crucial portions of transmission lines or towers to become fuzzy or even wholly concealed. This paper presents a Dual Attention Level Feature Fusion Multi-Patch Hierarchical Network (DAMPHN) for single image defogging to address the bad quality of cross-level feature fusion in Fast Deep Multi-Patch Hierarchical Networks (FDMPHN). Compared with FDMPHN before improvement, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) of DAMPHN are increased by 0.3 dB and 0.011 on average, and the Average Processing Time (APT) of a single picture is shortened by 11%. Additionally, compared with the other three excellent defogging methods, the PSNR and SSIM values DAMPHN are increased by 1.75 dB and 0.022 on average. Then, to mimic non-homogeneous fog, we combine the single picture depth information with 3D Berlin noise to create the UAV-HAZE dataset, which is used in the field of UAV power assessment. The experiment demonstrates that DAMPHN offers excellent defogging results and is competitive in no-reference and full-reference assessment indices. Full article

When image quality is evaluated, the human visual system (HVS) infers the details in the image through its internal generative mechanism. In this process, the HVS integrates both local and global information about the image, utilizes contextual information to restore the original image information, and compares it with the distorted image information for image quality evaluation. Inspired by this mechanism, a no-reference image quality assessment method is proposed based on a multitask image restoration network. The multitask image restoration network generates a pseudo-reference image as the main task and produces a structural similarity index measure map as an auxiliary task. By mutually promoting the two tasks, a higher-quality pseudo-reference image is generated. In addition, when predicting the image quality score, both the quality restoration features and the difference features between the distorted and reference images are used, thereby fully utilizing the information from the pseudo-reference image. In order to facilitate the model’s ability to extract both global and local features, we introduce a multi-scale feature fusion module. Experimental results demonstrate that the proposed method achieves excellent performance on both synthetically and authentically distorted databases. Full article

No-reference segmentation quality evaluation aims to evaluate the quality of image segmentation without any reference image during the application process. It usually depends on certain quality criteria to describe a good segmentation with some prior knowledge. Therefore, there is a need for a precise description of the objects in the segmentation and an integration of the representation in the evaluation process. In this paper, from the perspective of understanding the semantic relationship between the original image and the segmentation results, we propose a feature contrastive learning method. This method can enhance the performance of no-reference segmentation quality evaluations and be applied in semantic segmentation scenarios. By learning the pixel-level similarity between the original image and the segmentation result, a contrastive learning step is performed in the feature space. In addition, a class activation map (CAM) is used to guide the evaluation, making the score more consistent with the human visual judgement. Experiments were conducted on the PASCAL VOC2012 dataset, with segmentation results obtained by state-of-the-art (SoA) segmentation methods. We adopted two meta-measure criteria to validate the efficiency of the proposed method. Compared with other no-reference evaluation methods, our method achieves a higher accuracy which is comparable to the supervised evaluation methods and partly even exceeds them. Full article

New CMOS imaging sensor (CIS) techniques in smartphones have helped user-generated content dominate our lives over traditional DSLRs. However, tiny sensor sizes and fixed focal lengths also lead to more grainy details, especially for zoom photos. Moreover, multi-frame stacking and post-sharpening algorithms would produce zigzag textures and over-sharpened appearances, for which traditional image-quality metrics may over-estimate. To solve this problem, a real-world zoom photo database is first constructed in this paper, which includes 900 tele-photos from 20 different mobile sensors and ISPs. Then we propose a novel no-reference zoom quality metric which incorporates the traditional estimation of sharpness and the concept of image naturalness. More specifically, for the measurement of image sharpness, we are the first to combine the total energy of the predicted gradient image with the entropy of the residual term under the framework of free-energy theory. To further compensate for the influence of over-sharpening effect and other artifacts, a set of model parameters of mean subtracted contrast normalized (MSCN) coefficients are utilized as the natural statistics representatives. Finally, these two measures are combined linearly. Experimental results on the zoom photo database demonstrate that our quality metric can achieve SROCC and PLCC over 0.91, while the performance of single sharpness or naturalness index is around 0.85. Moreover, compared with the best tested general-purpose and sharpness models, our zoom metric outperforms them by 0.072 and 0.064 in SROCC, respectively. Full article

Thermal vision-based devices are nowadays used in a number of industries, ranging from the automotive industry, surveillance, navigation, fire detection, and rescue missions to precision agriculture. This work describes the development of a low-cost imaging device based on thermographic technology. The proposed device uses a miniature microbolometer module, a 32-bit ARM microcontroller, and a high-accuracy ambient temperature sensor. The developed device is capable of enhancing RAW high dynamic thermal readings obtained from the sensor using a computationally efficient image enhancement algorithm and presenting its visual result on the integrated OLED display. The choice of microcontroller, rather than the alternative System on Chip (SoC), offers almost instantaneous power uptime and extremely low power consumption while providing real-time imaging of an environment. The implemented image enhancement algorithm employs the modified histogram equalization, where the ambient temperature sensor helps the algorithm enhance both background objects near ambient temperature and foreground objects (humans, animals, and other heat sources) that actively emit heat. The proposed imaging device was evaluated on a number of environmental scenarios using standard no-reference image quality measures and comparisons against the existing state-of-the-art enhancement algorithms. Qualitative results obtained from the survey of 11 subjects are also provided. The quantitative evaluations show that, on average, images acquired by the developed camera provide better perception quality in 75% of tested cases. According to qualitative evaluations, images acquired by the developed camera provide better perception quality in 69% of tested cases. The obtained results verify the usability of the developed low-cost device for a range of applications where thermal imaging is needed. Full article

The purpose of the no-reference image quality assessment (NR-IQA) is to measure perceived image quality based on subjective judgments; however, due to the lack of a clean reference image, this is a complicated and unresolved challenge. Massive new IQA datasets have facilitated the creation of deep learning-based image quality measurements. We present a unique model to handle the NR-IQA challenge in this research by employing a hybrid strategy that leverages from pre-trained CNN model and the unified learning mechanism that extracts both local and non-local characteristics from the input patch. The deep analysis of the proposed framework shows that the model uses features and a mechanism that improves the monotonicity relationship between objective and subjective ratings. The intermediary goal was mapped to a quality score using a regression architecture. To extract various feature maps, a deep architecture with an adaptive receptive field was used. Analyses of this biggest NR-IQA benchmark datasets demonstrate that the suggested technique outperforms current state-of-the-art NR-IQA measures. Full article

Establishing an accurate objective evaluation metric of image sharpness is crucial for image analysis, recognition and quality measurement. In this review, we highlight recent advances in no-reference image quality assessment research, divide the reported algorithms into four groups (spatial domain-based methods, spectral domain-based methods, learning-based methods and combination methods) and outline the advantages and disadvantages of each method group. Furthermore, we conduct a brief bibliometric study with which to provide an overview of the current trends from 2013 to 2021 and compare the performance of representative algorithms on public datasets. Finally, we describe the shortcomings and future challenges in the current studies. Full article

Provisioning the stereoscopic 3D (S3D) video transmission services of admissible quality in a wireless environment is an immense challenge for video service providers. Unlike for 2D videos, a widely accepted No-reference objective model for assessing transmitted 3D videos that explores the Human Visual System (HVS) appropriately has not been developed yet. Distortions perceived in 2D and 3D videos are significantly different due to the sophisticated manner in which the HVS handles the dissimilarities between the two different views. In real-time video transmission, viewers only have the distorted or receiver end content of the original video acquired through the communication medium. In this paper, we propose a No-reference quality assessment method that can estimate the quality of a stereoscopic 3D video based on HVS. By evaluating perceptual aspects and correlations of visual binocular impacts in a stereoscopic movie, the approach creates a way for the objective quality measure to assess impairments similarly to a human observer who would experience the similar material. Firstly, the disparity is measured and quantified by the region-based similarity matching algorithm, and then, the magnitude of the edge difference is calculated to delimit the visually perceptible areas of an image. Finally, an objective metric is approximated by extracting these significant perceptual image features. Experimental analysis with standard S3D video datasets demonstrates the lower computational complexity for the video decoder and comparison with the state-of-the-art algorithms shows the efficiency of the proposed approach for 3D video transmission at different quantization (QP 26 and QP 32) and loss rate (1% and 3% packet loss) parameters along with the perceptual distortion features. Full article

Compression is a way of encoding digital data so that it takes up less storage and requires less network bandwidth to be transmitted, which is currently an imperative need for iris recognition systems due to the large amounts of data involved, while deep neural networks trained as image auto-encoders have recently emerged a promising direction for advancing the state-of-the-art in image compression, yet the generalizability of these schemes to preserve the unique biometric traits has been questioned when utilized in the corresponding recognition systems. For the first time, we thoroughly investigate the compression effectiveness of DSSLIC, a deep-learning-based image compression model specifically well suited for iris data compression, along with an additional deep-learning based lossy image compression technique. In particular, we relate Full-Reference image quality as measured in terms of Multi-scale Structural Similarity Index (MS-SSIM) and Local Feature Based Visual Security (LFBVS), as well as No-Reference images quality as measured in terms of the Blind Reference-less Image Spatial Quality Evaluator (BRISQUE), to the recognition scores as obtained by a set of concrete recognition systems. We further compare the DSSLIC model performance against several state-of-the-art (non-learning-based) lossy image compression techniques including: the ISO standard JPEG2000, JPEG, H.265 derivate BPG, HEVC, VCC, and AV1 to figure out the most suited compression algorithm which can be used for this purpose. The experimental results show superior compression and promising recognition performance of the model over all other techniques on different iris databases. Full article

An imaging system has natural statistics that reflect its intrinsic characteristics. For example, the gradient histogram of a visible light image generally obeys a heavy-tailed distribution, and its restoration considers natural statistics. Thermal imaging cameras detect infrared radiation, and their signal processors are specialized according to the optical and sensor systems. Thermal images, also known as long wavelength infrared (LWIR) images, suffer from distinct degradations of LWIR sensors and residual nonuniformity (RNU). However, despite the existence of various studies on the statistics of thermal images, thermal image processing has seldom attempted to incorporate natural statistics. In this study, natural statistics of thermal imaging sensors are derived, and an optimization method for restoring thermal images is proposed. To verify our hypothesis about the thermal images, high-frequency components of thermal images from various datasets are analyzed with various measures (correlation coefficient, histogram intersection, chi-squared test, Bhattacharyya distance, and Kullback–Leibler divergence), and generalized properties are derived. Furthermore, cost functions accommodating the validated natural statistics are designed and minimized by a pixel-wise optimization method. The proposed algorithm has a specialized structure for thermal images and outperforms the conventional methods. Several image quality assessments are employed for quantitatively demonstrating the performance of the proposed method. Experiments with synthesized images and real-world images are conducted, and the results are quantified by reference image assessments (peak signal-to-noise ratio and structural similarity index measure) and no-reference image assessments (Roughness (Ro) and Effective Roughness (ERo) indices). Full article

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images. Full article

The estimation of image quality and noise perception still remains an important issue in various image processing applications. It has also become a hot topic in the field of photo-realistic computer graphics where noise is inherent in the calculation process. Unlike natural-scene images, however, a reference image is not available for computer-generated images. Thus, classic methods to assess noise quantity and stopping criterion during the rendering process are not usable. This is particularly important in the case of global illumination methods based on stochastic techniques: They provide photo-realistic images which are, however, corrupted by stochastic noise. This noise can be reduced by increasing the number of paths, as proved by Monte Carlo theory, but the problem of finding the right number of paths that are required in order to ensure that human observers cannot perceive any noise is still open. Until now, the features taking part in the human evaluation of image quality and the remaining perceived noise are not precisely known. Synthetic image generation tends to be very expensive and the produced datasets are high-dimensional datasets. In that case, finding a stopping criterion using a learning framework is a challenging task. In this paper, a new method for characterizing computational noise for computer generated images is presented. The noise is represented by the entropy of the singular value decomposition of each block composing an image. These Singular Value Decomposition (SVD)-entropy values are then used as input to a recurrent neural network architecture model in order to extract image noise and in predicting a visual convergence threshold of different parts of any image. Thus a new no-reference image quality assessment is proposed using the relation between SVD-Entropy and perceptual quality, based on a sequence of distorted images. Experiments show that the proposed method, compared with experimental psycho-visual scores, demonstrates a good consistency between these scores and stopping criterion measures that we obtain. Full article