Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Towards the Understanding of Kernel and Neural Learning Methods via...
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Towards the Understanding of Kernel and Neural Learning Methods via Information-Theoretic Learning (ITL)

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (30 December 2021) | Viewed by 6796

Special Issue Editors

Prof. Dr. Lluís A. Belanche Muñoz

E-Mail Website
Guest Editor
School of Computer Science, Universitat Politècnica de Catalunya, 08034 Barcelona, Spain
Interests: feed-forward neural networks; support vector machines; kernel functions; similarity measures
Dr. Bruno Ordozgoiti

E-Mail Website
Guest Editor
Department of Computer Science, Aalto University, FI-00076 Aalto, Finland
Interests: graph mining; approximation algorithms; matrix factorization; clustering; kernel methods

Special Issue Information

Dear Colleagues,

Kernel methods and neural networks have achieved great popularity as tools in pattern recognition and machine learning for the identification of nonlinear systems. The kernel function is a very flexible container under which to express knowledge about the problem as well as to capture the meaningful relations in input space. The choice of a proper kernel for a given problem is both an open issue and an appealing chance to boost research. Kernel methods are single-layer machines and usually involve solving a tractable convex problem. Contrary to neural models, they are able to handle nonvectorial data directly, leading to a higher (input) expressive power.

On the other hand, multilayer neural networks have experienced a rebirth in the data analysis field, displaying impressive results, extending even to classical artificial intelligence domains such as game playing, computer vision, natural language, and speech processing. The versatility of such methods has led deep (semi)-parametric models to overtake well-established methods like classical statistical techniques.

In this Special Issue, we aim to bring together contributions towards the understanding of such single and deep structures from the analysis of their information content, as well as contributions that apply or analyze such structures from inferential, probabilistic, or information-theoretic points of view. An additional natural field of research is given by their hybridization, which can be done in many fruitful ways. Many ideas from the deep learning field can be transferred to the kernel method framework and vice versa.

Of particular interest are works oriented towards increasing our current understanding on the following topics:

  • Analysis of single or deep learning structures from an information-theoretic point of view;
  • Links or dualities between ITL methods and kernel or neuronal methods;
  • Characterization of the learning process from an information-theoretic point of view;
  • Use of ITL methods to analyze the implications of the kernel or neuron model for the success of learning methods;
  • Links between kernel or neuron model design and ITL methods for special or new application domains;
  • New ideas to bridge the gap between the fields of deep and kernel learning;
  • New ideas or tools to increase the understanding of their respective weak and strong points.

The scope of the contributions is rather broad, including theoretical studies and practical applications to any kind of machine learning or statistical task, such as regression, classification, system identification, unsupervised learning, density estimation, clustering, etc.

Prof. Dr. Lluís A. Belanche-Muñoz
Dr. Bruno Ordozgoiti-Rubio
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Neural networks
  • Deep learning
  • Information theory of learning structures
  • Integration of data sources
  • Kernel methods
  • Kernel functions
  • Neuron models
  • Integration of learning methods

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

39 pages, 1240 KiB  
Article
Analysis of Kernel Matrices via the von Neumann Entropy and Its Relation to RVM Performances
by Lluís A. Belanche-Muñoz and Małgorzata Wiejacha
Entropy 2023, 25(1), 154; https://doi.org/10.3390/e25010154 - 12 Jan 2023
Viewed by 1275
Abstract
Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others [...] Read more.
Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model’s fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited. Full article
(This article belongs to the Special Issue Towards the Understanding of Kernel and Neural Learning Methods via Information-Theoretic Learning (ITL))
►▼ Show Figures

Figure 1

10 pages, 380 KiB  
Article
Cross-Modality Person Re-Identification Based on Heterogeneous Center Loss and Non-Local Features
by Chengmei Han, Peng Pan, Aihua Zheng and Jin Tang
Entropy 2021, 23(7), 919; https://doi.org/10.3390/e23070919 - 20 Jul 2021
Cited by 3 | Viewed by 2184
Abstract
Cross-modality person re-identification is the study of images of people matching under different modalities (RGB modality, IR modality). Given one RGB image of a pedestrian collected under visible light in the daytime, cross-modality person re-identification aims to determine whether the same pedestrian appears [...] Read more.
Cross-modality person re-identification is the study of images of people matching under different modalities (RGB modality, IR modality). Given one RGB image of a pedestrian collected under visible light in the daytime, cross-modality person re-identification aims to determine whether the same pedestrian appears in infrared images (IR images) collected by infrared cameras at night, and vice versa. Cross-modality person re-identification can solve the task of pedestrian recognition in low light or at night. This paper aims to improve the degree of similarity for the same pedestrian in two modalities by improving the feature expression ability of the network and designing appropriate loss functions. To implement our approach, we introduce a deep neural network structure combining heterogeneous center loss (HC loss) and a non-local mechanism. On the one hand, this can heighten the performance of feature representation of the feature learning module, and, on the other hand, it can improve the similarity of cross-modality within the class. Experimental data show that the network achieves excellent performance on SYSU-MM01 datasets. Full article
(This article belongs to the Special Issue Towards the Understanding of Kernel and Neural Learning Methods via Information-Theoretic Learning (ITL))
►▼ Show Figures

Figure 1

15 pages, 462 KiB  
Article
Weighted Mutual Information for Aggregated Kernel Clustering
by Nezamoddin N. Kachouie and Meshal Shutaywi
Entropy 2020, 22(3), 351; https://doi.org/10.3390/e22030351 - 18 Mar 2020
Cited by 5 | Viewed by 2644
Abstract
Background: A common task in machine learning is clustering data into different groups based on similarities. Clustering methods can be divided in two groups: linear and nonlinear. A commonly used linear clustering method is K-means. Its extension, kernel K-means, is a non-linear technique [...] Read more.
Background: A common task in machine learning is clustering data into different groups based on similarities. Clustering methods can be divided in two groups: linear and nonlinear. A commonly used linear clustering method is K-means. Its extension, kernel K-means, is a non-linear technique that utilizes a kernel function to project the data to a higher dimensional space. The projected data will then be clustered in different groups. Different kernels do not perform similarly when they are applied to different datasets. Methods: A kernel function might be relevant for one application but perform poorly to project data for another application. In turn choosing the right kernel for an arbitrary dataset is a challenging task. To address this challenge, a potential approach is aggregating the clustering results to obtain an impartial clustering result regardless of the selected kernel function. To this end, the main challenge is how to aggregate the clustering results. A potential solution is to combine the clustering results using a weight function. In this work, we introduce Weighted Mutual Information (WMI) for calculating the weights for different clustering methods based on their performance to combine the results. The performance of each method is evaluated using a training set with known labels. Results: We applied the proposed Weighted Mutual Information to four data sets that cannot be linearly separated. We also tested the method in different noise conditions. Conclusions: Our results show that the proposed Weighted Mutual Information method is impartial, does not rely on a single kernel, and performs better than each individual kernel specially in high noise. Full article
(This article belongs to the Special Issue Towards the Understanding of Kernel and Neural Learning Methods via Information-Theoretic Learning (ITL))
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop