Advances in Machine Learning

Since its inception as a branch of Artificial Intelligence, Machine Learning (ML) has flourished in recent years. A variety of approaches have been proposed based on concepts from related fields including statistics, mathematics, cognitive science, information theory, control theory, philosophy, biology, and computer science.

This Special Issue consists of sixteen papers introducing advances in ML with diverse applications, including three papers on Single Image Super-Resolution (SISR), five papers on different deep learning models, two papers on meta or ensemble learning, two papers on hyperparameter optimization (HPO), two papers on data privacy, and two papers on applications using ML.

To provide a brief introduction to the papers, three papers [1,2,3] deal with the SISR task: In [1], a new image super-resolution framework, BSR-DUL, is proposed using a deep unsupervised neural network for accurate prediction of high-resolution image and the degradation operation applied from the low-resolution image without any prior knowledge. Ref. [2] introduces a multi-path deep convolutional neural network (CNN), MCISIR, with residual and inception blocks, Leakly ReLU function, combination of deconvolution and upsampling layers, and without the batch normalization and max-pooling layers, and reports improved performance in terms of signal-to-noise ratio and computation speed. Ref. [3] proposes a deep residual dense network, DRDN, which exploits the residual in the residual dense block and constructs networks of residual and dense blocks with skip connections in various depths.

Several deep learning models are presented: Ref. [4] designs a network intrusion detection model, DLNID, that combines an attention mechanism, a bidirectional long short-term memory (LSTM), and an adaptive synthetic sampling, and demonstrates improved performance for severely imbalanced data. Ref. [5] proposes a multi-level deep learning model for disease recognition in potato leaves which consists of two levels of image segmentation and classification, where the classification level is implemented by a new CNN model outperforming existing methods with fewer parameters. Ref. [6] designs a progressive deep neural network to learn new classes without forgetting old ones through a novel incremental learning technique that trains the network only for new classes when they are introduced and then fine-tunes the network instead of training the entire network, requiring less computational overhead and structural changes. Ref. [7] presents a spatial-temporal deep learning network for traffic flow forecasting, ST-TrafficNet, based on both a multi-diffusion (of attentive and bidirectional) convolution block and a stacked LSTM block to capture spatial and temporal features, respectively. Ref. [8] deals with medical image data for tumor segmentation based on a new hybrid model for mobile devices, RMU-Net, which adds residual blocks to the architecture of MobileNetV2 (as an encoder) and makes use of upsampling layers of U-Net (as a decoder).

This SI includes two papers on meta/ensemble learning: Ref. [9] proposes a new meta-learning method for unsupervised outlier detection that attempts to combine the best of individual algorithms through ensemble voting and to identify the most informative algorithm for a given dataset through unsupervised feature selection. Ref. [10] is an experimental paper verifying the effectiveness of boosting, in terms of classification accuracy, with two representative decision tree learning algorithms (C5.0 and CART) applied to real-world bank-customer data.

In addition, there are a couple of papers on HPO: Ref. [11] introduces a greedy k-fold cross validation method that vastly reduces the average time required to find the best-performing model with or without a computational budget, and experimentally verifies its improved performance over existing methods. Another paper, Ref. [12] develops Hermes, a GPU scheduling framework for accelerating HPO in deep learning clusters by time-sharing between deep learning jobs and prioritizing jobs with more promising hyperparameter combinations.

There are two papers in this SI that deal with data privacy: Ref. [13] proposes a privacy-preserving semi-generative adversarial network, PPSGAN, that selectively adds noise to class-independent features of each image (anoymization) to enable the processed image to maintain its original class label (utility), based on the generative adversarial network model. Ref. [14] offers an application of differential privacy to reinforcement learning, and it presents a differentially private actor-critic approach preserving the privacy of the actor and its eligibility trace while training on private or sensitive data, and demonstrates its application with Patient Treatment Progression and Taxi datasets where individual privacy is of the utmost importance.

The last few papers in this SI are on practical applications: Ref. [15] applies ML models to the Channel Quality Indicator prediction task for mobile networks, and suggests XGBoost as the best performing model and introduces several features that improves classification accuracy. Ref. [16] attempts to use bioacoustics and to accurately classify birds as monosyllabic or multisyllabic with ML, proposing a collection of Perceptual, Descriptive, and Harmonic Features (PDHF) used with artificial neural networks.

As ML is becoming more prevalent in almost all areas, novel ideas and research directions can be fostered. We hope that this SI will be a useful introduction and an example to people in all fields by helping them carry out research and derive new technology solutions in various real-world applications.