Submit to Entropy Review for Entropy Propose a Special Issue

Journal Browser

Statistical Machine Learning with High-Dimensional Data and Image Analysis

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (26 April 2023) | Viewed by 16724

Share This Special Issue

Special Issue Editor

Dr. Lei Wang

E-Mail Website
Guest Editor

Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
Interests: machine learning; computer vision; image transform; 3D reconstruction

Special Issue Information

Dear Colleagues,

Statistical machine learning methods have been widely used for the analysis of high-dimensional structured data and images. Many algorithms have been developed aiming to build a model based on sample data from various areas, such as medicine, web documents, remote sensing, and multimedia data. To handle high-dimensional data, sparse signal reconstruction, low-rank matrix recovery, and principal component analysis have been studied. How to solve the convex or non-convex optimization questions still need to be researched, as well as their theory for new kinds of structured data and images.

The applications of statistical machine learning to high-dimensional data always have some difficulties, such as non-modularity and instability, which restrict their effectiveness in real-world scenes. Emerging technologies have provided new resolutions based on deep neural networks for large-scale datasets. However, the interpretability of the networks is not as good as that of statistical machine learning algorithms. The information entropy has not been well explored in deep learning. Contributions addressing any of these issues are very welcome.

This Special Issue aims to be a forum for the presentation of new techniques of statistical machine learning for high-dimensional data. In particular, the analysis and interpretation of real-world data or images based on machine learning or deep learning fall within the scope of this Special Issue.

Dr. Lei Wang
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

statistical machine learning
high-dimensional data
image processing
data mining
pattern recognition
deep learning
big data
computer vision
multimedia

Published Papers (6 papers)

Download All Papers

Research

Jump to: Review

12 pages, 1948 KiB

Open AccessArticle

Hybrid Granularities Transformer for Fine-Grained Image Recognition

by Ying Yu and Jinghui Wang

Entropy 2023, 25(4), 601; https://doi.org/10.3390/e25040601 - 1 Apr 2023

Viewed by 1123

Abstract

Many current approaches for image classification concentrate solely on the most prominent features within an image, but in fine-grained image recognition, even subtle features can play a significant role in model classification. In addition, the large variations in the same class and small differences between different categories that are unique to fine-grained image recognition pose a great challenge for the model to extract discriminative features between different categories. Therefore, we aim to present two lightweight modules to help the network discover more detailed information in this paper. (1) Patches Hidden Integrator (PHI) module randomly selects patches from images and replaces them with patches from other images of the same class. It allows the network to glean diverse discriminative region information and prevent over-reliance on a single feature, which can lead to misclassification. Additionally, it does not increase the training time. (2) Consistency Feature Learning (CFL) aggregates patch tokens from the last layer, mining local feature information and fusing it with the class token for classification. CFL also utilizes inconsistency loss to force the network to learn common features in both tokens, thereby guiding the network to focus on salient regions. We conducted experiments on three datasets, CUB-200-2011, Stanford Dogs, and Oxford 102 Flowers. We achieved experimental results of 91.6%, 92.7%, and 99.5%, respectively, achieving a competitive performance compared to other works. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

23 pages, 1967 KiB

Open AccessArticle

An Order Reduction Design Framework for Higher-Order Binary Markov Random Fields

by Zhuo Chen, Hongyu Yang and Yanli Liu

Entropy 2023, 25(3), 535; https://doi.org/10.3390/e25030535 - 20 Mar 2023

Viewed by 860

Abstract

The order reduction method is an important approach to optimize higher-order binary Markov random fields (HoMRFs), which are widely used in information theory, machine learning and image analysis. It transforms an HoMRF into an equivalent and easier reduced first-order binary Markov random field (RMRF) by elaborately setting the coefficients and auxiliary variables of RMRF. However, designing order reduction methods is difficult, and no previous study has investigated this design issue. In this paper, we propose an order reduction design framework to study this problem for the first time. Through study, we find that the design difficulty mainly lies in that the coefficients and variables of RMRF must be set simultaneously. Therefore, the proposed framework decomposes the design difficulty into two processes, and each process mainly considers the coefficients or auxiliary variables of RMRF. Some valuable properties are also proven. Based on our framework, a new family of 14 order reduction methods is provided. Experiments, such as synthetic data and image denoising, demonstrate the superiority of our method. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

17 pages, 2488 KiB

Open AccessArticle

Image Adversarial Example Generation Method Based on Adaptive Parameter Adjustable Differential Evolution

by Zhiyi Lin, Changgen Peng, Weijie Tan and Xing He

Entropy 2023, 25(3), 487; https://doi.org/10.3390/e25030487 - 10 Mar 2023

Viewed by 1330

Abstract

Adversarial example generation techniques for neural network models have exploded in recent years. In the adversarial attack scheme for image recognition models, it is challenging to achieve a high attack success rate with very few pixel modifications. To address this issue, this paper proposes an adversarial example generation method based on adaptive parameter adjustable differential evolution. The method realizes the dynamic adjustment of the algorithm performance by adjusting the control parameters and operation strategies of the adaptive differential evolution algorithm, while searching for the optimal perturbation. Finally, the method generates adversarial examples with a high success rate, modifying just a very few pixels. The attack effectiveness of the method is confirmed in CIFAR10 and MNIST datasets. The experimental results show that our method has a greater attack success rate than the One Pixel Attack based on the conventional differential evolution. In addition, it requires significantly less perturbation to be successful compared to global or local perturbation attacks, and is more resistant to perception and detection. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

14 pages, 4428 KiB

Open AccessArticle

Papaver somniferum and Papaver rhoeas Classification Based on Visible Capsule Images Using a Modified MobileNetV3-Small Network with Transfer Learning

by Jin Zhu, Chuanhui Zhang and Changjiang Zhang

Entropy 2023, 25(3), 447; https://doi.org/10.3390/e25030447 - 3 Mar 2023

Cited by 4 | Viewed by 4050

Abstract

Traditional identification methods for Papaver somniferum and Papaver rhoeas (PSPR) consume much time and labor, require strict experimental conditions, and usually cause damage to the plant. This work presents a novel method for fast, accurate, and nondestructive identification of PSPR. First, to fill the gap in the PSPR dataset, we construct a PSPR visible capsule image dataset. Second, we propose a modified MobileNetV3-Small network with transfer learning, and we solve the problem of low classification accuracy and slow model convergence due to the small number of PSPR capsule image samples. Experimental results demonstrate that the modified MobileNetV3-Small is effective for fast, accurate, and nondestructive PSPR classification. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

18 pages, 4222 KiB

Open AccessArticle

Research on Insulator Defect Detection Based on an Improved MobilenetV1-YOLOv4

by Shanyong Xu, Jicheng Deng, Yourui Huang, Liuyi Ling and Tao Han

Entropy 2022, 24(11), 1588; https://doi.org/10.3390/e24111588 - 2 Nov 2022

Cited by 15 | Viewed by 1787

Abstract

Insulator devices are important for transmission lines, and defects such as insulator bursting and string loss affect the safety of transmission lines. In this study, we aim to investigate the problems of slow detection speed and low efficiency of traditional insulator defect detection algorithms, and to improve the accuracy of insulator fault identification and the convenience of daily work; therefore, we propose an insulator defect detection algorithm based on an improved MobilenetV1-YOLOv4. First, the backbone feature extraction network of YOLOv4 ‘Backbone’ is replaced with the lightweight module Mobilenet-V1. Second, the scSE attention mechanism is introduced in stages of preliminary feature extraction and enhanced feature extraction, sequentially. Finally, the depthwise separable convolution substitutes the 3 × 3 convolution of the enhanced feature extraction network to reduce the overall number of network parameters. The experimental results show that the weight of the improved algorithm is 57.9 MB, which is 62.6% less than that obtained by the MobilenetV1-YOLOv4 model; the average accuracy of insulator defect detection is improved by 0.26% and reaches 98.81%; and the detection speed reaches 190 frames per second with an increase of 37 frames per second. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

Review

Jump to: Research

23 pages, 1208 KiB

Open AccessReview

Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review

by Man-Fai Wong, Shangxin Guo, Ching-Nam Hang, Siu-Wai Ho and Chee-Wei Tan

Entropy 2023, 25(6), 888; https://doi.org/10.3390/e25060888 - 1 Jun 2023

Cited by 5 | Viewed by 6406

Abstract

This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software naturalness, have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection. Notable examples of such applications include the GitHub Copilot powered by OpenAI’s Codex and DeepMind AlphaCode. This paper presents an overview of the major LLMs and their applications in downstream tasks related to AI-assisted programming. Furthermore, it explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications, with a discussion on extending AI-assisted programming capabilities to Apple’s Xcode for mobile software development. This paper also presents the challenges of and opportunities for incorporating NLP techniques with software naturalness, empowering developers with advanced coding assistance and streamlining the software development process. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures