Digital Signal, Image and Video Processing for Emerging Multimedia Technology

Recent developments in image/video-based deep learning technology have enabled new services in the field of multimedia and recognition technology [1,2,3]. The technologies underlying the development of these recognition and emerging services are based on essential signal and image processing algorithms. In addition, the recent realistic media services, mixed reality, augmented reality and virtual reality media services also require very high-definition media creation, personalization, and transmission technologies, and this demand continues to grow [4,5]. To accommodate these needs, international standardization and industry are studying various digital signal and image processing technologies to provide a variety of new or future media services. In this issue, we present high-quality papers that study advanced signal/image processing and video data processing, including deep learning approaches.

Twenty-three papers relating to digital signal, image and video processing for emerging multimedia technology have been published in this Special Issue. They deal with some advanced issues regarding signal/image processing and video data processing, including deep learning approaches, such as the convolution neural network (CNN) and generative adversarial networks (GANs).

Classifying and recognizing objects is very important task in the image and video signal processing area. In [6], Tsai, M.-F et al. not only proposed multiple feature dependency detection to identify key parts of pets (mouth and tail), but also combined the meaning of the pet’s bark (growl and cry) to identify the pet’s mood and state. Nguyen, K et al. presented an evaluation of state-of-the-art deep-learning detectors including Faster Regional CNN (Faster R-CNN), Region-based Fully Convolutional Networks (RFCN), Scale Normalization for Image Pyramids with Efficient Resampling (SNIPER), Single-Shot Detector (SSD), YOLO (You Only Look Once), RetinaNet, and CenterNet for the object detection in videos captured by drones [7]. Additionally, the impact of age and gender in sentiment analysis was explored as this could help e-commerce retailers to market their products based on specific demographics by Kumar, S et al. [8]. To make a clean image, Wu, C et al. proposed an image text deblurring method based on a generative adversarial network. The model of the algorithm consists of two generative adversarial networks, combined with Wasserstein distance, using a combination of adversarial loss and perceptual loss on unpaired datasets to train the network model to restore the captured blurred images into clear and natural image [9]. In [10], the authors proposed a new and efficient method for the detection and recognition of moving objects in a sequence of images captured from a UAV, in real time and in a real environment. Kang, S et al. proposed a convolutional neural network (CNN)-based steganalytic method that allowed ternary classification to simultaneously identify WOW and UNIWARD, which were representative adaptive image steganographic algorithms [11]. A novel smoke detection algorithm that reduces false positive detection using spatial and temporal features based on deep learning from factory installed surveillance cameras was suggested by Lee, Y et al. [12]. Gómez Blas, N et al. reported a study and implementation of a convolutional neural network to identify and recognize humpback whale specimens by processing their tail patterns [13].

In the field of learning mechanism design, Xu, Y. et al. investigated a method named NL2LL to collect the underexposure images and the corresponding normal exposure images by adjusting camera settings in the “normal” level of light during the daytime [14]. An approach to hyperparameter optimization for the objective function in machine learning was proposed by Kim, Y. et al. [15]. In [16], Xia, S.et al. suggested an effective method for exploring discriminative regions of the scene image using the gradient-weighted class activation mapping (Grad-CAM) technique and weakly supervised information to generate the attention map (AM) of scene images.

For video processing, Yan, T. et al. proposed a novel ρ domain rate control algorithm for multiview high efficiency video coding (MV-HEVC) [17]. In addition, Maturana-Espinosa, J.C. et al. proposed two rate-allocation algorithms which provided reconstructions that were progressive in quality [18]. They used Optimized Sub-band Layers Allocation (OSLA) and Estimated-Slope sub-band Layers Allocation (ESLA). To gain more coding efficiency, a perspective affine motion compensation (PAMC) method which can cope with more complex motions such as shear and shape distortion was proposed by Choi, Y.-J. et al. [19].

As the advanced theory of digital signal and image processing, Maqsood, S. et al. proposed a novel multiscale image fusion system based on contrast enhancement, spatial gradient information and multiscale image matting to extract the focused region information from multiple source images [20]. In [21], an edge-preserving nonlinear filter was proposed to reduce multiplicative noise by using a filter structure based on mathematical morphology by Ince, I.F. et al. This method was called the minimum index of dispersion (MID) filter. Zhu, Y. et al. suggested an altered adaptive algorithm on block-compressive sensing (BCS) by using saliency and error analysis [22]. In [23], Benavides-Álvarez, C. et al. implemented a new strategy called Wiener-Granger Causality theory based on self-content images extracted using a Content-Based Image Retrieval (CBIR) methodology, for classifying natural scenery images. A scale-invariant deep neural network model based on wavelets for single image super-resolution (SISR) was proposed by Sahito, F. et al. [24].

In the field of security and data protection, Khan, A.N. et al. suggested an efficient separable reversible data hiding scheme over a homomorphically encrypted image that assures privacy preservation of the contents in the cloud environment [25]. In addition, Lee, J.Y. improved the performance of embedding capacity using PEE, inter-component prediction, and allowable pixel ranges. Inter-component prediction utilized a strong correlation between the texture image and the depth map in 3D video [26]. For CCTV video security, a character order preserving (COP)-transformation technique that allows the secure protection of video meta-data was proposed by Kim, J. et al. [27].

Lastly, Lee, J. et al. developed a wrist-mounted dive computer, so called DiverPAD, for underwater drawing and writing. For the framework design, firmware, communication protocol, user interface (UI), and underwater touchscreen functions were designed and integrated on DiverPAD [28].

I hope that the technical papers published in this Special Issue can help researchers and readers to understand the emerging theories and technologies in the field of digital signal, image, and video signal processing.