E-Mail Alert

Add your e-mail address to receive forthcoming issues of this journal:

Journal Browser

Journal Browser

Special Issue "Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets"

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (30 January 2016)

Special Issue Editor

Guest Editor
Prof. Dr. Andreas Holzinger

Medical University Graz and Graz University of Technology, Austria
Website1 | Website2 | E-Mail
Phone: +43 316 385 13883
Interests: Machine Learning; Knowledge Extraction; Health Informatics

Special Issue Information

Dear Colleagues,

In the real world, we are confronted, not only with complex and high-dimensional data sets, but also usually with noisy, incomplete, and uncertain data, where the application of traditional methods of knowledge discovery and data mining always entail the danger of modeling artifacts. Originally, information entropy was introduced by Shannon (1949), as a measure of uncertainty in data. Up to the present, many different types of entropy methods with a large number of different purposes and possible application areas have emerged. In this Special Issue we are seeking papers discussing advances in the application of learning algorithms and entropy for use in knowledge discovery and data mining, to discover unknowns in complex data sets, e.g., for biomarker discovery in biomedical data sets.

Prof. Dr. Andreas Holzinger
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1500 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Machine Learning
  • Knowledge Discovery
  • Entropy-based Data Mining

Published Papers (13 papers)

View options order results:
result details:
Displaying articles 1-13
Export citation of selected articles as:

Research

Open AccessArticle Distribution Entropy Boosted VLAD for Image Retrieval
Entropy 2016, 18(8), 311; doi:10.3390/e18080311
Received: 26 February 2016 / Revised: 12 July 2016 / Accepted: 16 August 2016 / Published: 24 August 2016
PDF Full-text (1057 KB) | HTML Full-text | XML Full-text
Abstract
Several recent works have shown that aggregating local descriptors to generate global image representation results in great efficiency for retrieval and classification tasks. The most popular method following this approach is VLAD (Vector of Locally Aggregated Descriptors). We present a novel image presentation
[...] Read more.
Several recent works have shown that aggregating local descriptors to generate global image representation results in great efficiency for retrieval and classification tasks. The most popular method following this approach is VLAD (Vector of Locally Aggregated Descriptors). We present a novel image presentation called Distribution Entropy Boosted VLAD (EVLAD), which extends the original vector of locally aggregated descriptors. The original VLAD adopts only residuals to depict the distribution information of every visual word and neglects other statistical clues, so its discriminative power is limited. To address this issue, this paper proposes the use of the distribution entropy of each cluster as supplementary information to enhance the search accuracy. To fuse two feature sources organically, two fusion methods after a new normalization stage meeting power law are also investigated, which generate identically sized and double-sized vectors as the original VLAD. We validate our approach in image retrieval and image classification experiments. Experimental results demonstrate the effectiveness of our algorithm. Full article
Figures

Figure 1

Open AccessArticle Voice Activity Detection Using Fuzzy Entropy and Support Vector Machine
Entropy 2016, 18(8), 298; doi:10.3390/e18080298
Received: 27 April 2016 / Revised: 27 July 2016 / Accepted: 8 August 2016 / Published: 12 August 2016
Cited by 1 | PDF Full-text (4576 KB) | HTML Full-text | XML Full-text
Abstract
This paper proposes support vector machine (SVM) based voice activity detection using FuzzyEn to improve detection performance under noisy conditions. The proposed voice activity detection (VAD) uses fuzzy entropy (FuzzyEn) as a feature extracted from noise-reduced speech signals to train an SVM model
[...] Read more.
This paper proposes support vector machine (SVM) based voice activity detection using FuzzyEn to improve detection performance under noisy conditions. The proposed voice activity detection (VAD) uses fuzzy entropy (FuzzyEn) as a feature extracted from noise-reduced speech signals to train an SVM model for speech/non-speech classification. The proposed VAD method was tested by conducting various experiments by adding real background noises of different signal-to-noise ratios (SNR) ranging from −10 dB to 10 dB to actual speech signals collected from the TIMIT database. The analysis proves that FuzzyEn feature shows better results in discriminating noise and corrupted noisy speech. The efficacy of the SVM classifier was validated using 10-fold cross validation. Furthermore, the results obtained by the proposed method was compared with those of previous standardized VAD algorithms as well as recently developed methods. Performance comparison suggests that the proposed method is proven to be more efficient in detecting speech under various noisy environments with an accuracy of 93.29%, and the FuzzyEn feature detects speech efficiently even at low SNR levels. Full article
Figures

Open AccessArticle How Is a Data-Driven Approach Better than Random Choice in Label Space Division for Multi-Label Classification?
Entropy 2016, 18(8), 282; doi:10.3390/e18080282
Received: 1 February 2016 / Revised: 12 July 2016 / Accepted: 19 July 2016 / Published: 30 July 2016
PDF Full-text (2969 KB) | HTML Full-text | XML Full-text
Abstract
We propose using five data-driven community detection approaches from social networks to partition the label space in the task of multi-label classification as an alternative to random partitioning into equal subsets as performed by RAkELd. We evaluate modularity-maximizing using
[...] Read more.
We propose using five data-driven community detection approaches from social networks to partition the label space in the task of multi-label classification as an alternative to random partitioning into equal subsets as performed by RAkELd. We evaluate modularity-maximizing using fast greedy and leading eigenvector approximations, infomap, walktrap and label propagation algorithms. For this purpose, we propose to construct a label co-occurrence graph (both weighted and unweighted versions) based on training data and perform community detection to partition the label set. Then, each partition constitutes a label space for separate multi-label classification sub-problems. As a result, we obtain an ensemble of multi-label classifiers that jointly covers the whole label space. Based on the binary relevance and label powerset classification methods, we compare community detection methods to label space divisions against random baselines on 12 benchmark datasets over five evaluation measures. We discover that data-driven approaches are more efficient and more likely to outperform RAkELd than binary relevance or label powerset is, in every evaluated measure. For all measures, apart from Hamming loss, data-driven approaches are significantly better than RAkELd ( α = 0 . 05 ), and at least one data-driven approach is more likely to outperform RAkELd than a priori methods in the case of RAkELd’s best performance. This is the largest RAkELd evaluation published to date with 250 samplings per value for 10 values of RAkELd parameter k on 12 datasets published to date. Full article
Figures

Open AccessArticle A PUT-Based Approach to Automatically Extracting Quantities and Generating Final Answers for Numerical Attributes
Entropy 2016, 18(6), 235; doi:10.3390/e18060235
Received: 14 January 2016 / Revised: 12 June 2016 / Accepted: 12 June 2016 / Published: 22 June 2016
PDF Full-text (604 KB) | HTML Full-text | XML Full-text
Abstract
Automatically extracting quantities and generating final answers for numerical attributes is very useful in many occasions, including question answering, image processing, human-computer interaction, etc. A common approach is to learn linguistics templates or wrappers and employ some algorithm or model to generate a
[...] Read more.
Automatically extracting quantities and generating final answers for numerical attributes is very useful in many occasions, including question answering, image processing, human-computer interaction, etc. A common approach is to learn linguistics templates or wrappers and employ some algorithm or model to generate a final answer. However, building linguistics templates or wrappers is a tough task for builders. In addition, linguistics templates or wrappers are domain-dependent. To make the builder escape from building linguistics templates or wrappers, we propose a new approach to final answer generation based on Predicates-Units Table (PUT), a mini domain-independent knowledge base. It is deserved to point out that, in the following cases, quantities are not represented well. Quantities are absent of units. Quantities are perhaps wrong for a given question. Even if all of them are represented well, their units are perhaps inconsistent. These cases have a strong impact on final answer solving. One thousand nine hundred twenty-six real queries are employed to test the proposed method, and the experimental results show that the average correctness ratio of our approach is 87.1%. Full article
Figures

Open AccessArticle Stimuli-Magnitude-Adaptive Sample Selection for Data-Driven Haptic Modeling
Entropy 2016, 18(6), 222; doi:10.3390/e18060222
Received: 19 April 2016 / Revised: 1 June 2016 / Accepted: 2 June 2016 / Published: 7 June 2016
PDF Full-text (1897 KB) | HTML Full-text | XML Full-text | Supplementary Files
Abstract
Data-driven haptic modeling is an emerging technique where contact dynamics are simulated and interpolated based on a generic input-output matching model identified by data sensed from interaction with target physical objects. In data-driven modeling, selecting representative samples from a large set of data
[...] Read more.
Data-driven haptic modeling is an emerging technique where contact dynamics are simulated and interpolated based on a generic input-output matching model identified by data sensed from interaction with target physical objects. In data-driven modeling, selecting representative samples from a large set of data in a way that they can efficiently and accurately describe the whole dataset has been a long standing problem. This paper presents a new algorithm for the sample selection where the variances of output are observed for selecting representative input-output samples in order to ensure the quality of output prediction. The main idea is that representative pairs of input-output are chosen so that the ratio of the standard deviation to the mean of the corresponding output group does not exceed an application-dependent threshold. This output- and standard deviation-based sample selection is very effective in applications where the variance or relative error of the output should be kept within a certain threshold. This threshold is used for partitioning the input space using Binary Space Partitioning-tree (BSP-tree) and k-means algorithms. We apply the new approach to data-driven haptic modeling scenario where the relative error of the output prediction result should be less than a perceptual threshold. For evaluation, the proposed algorithm is compared to two state-of-the-art sample selection algorithms for regression tasks. Four kinds of haptic related behavior–force datasets are tested. The results showed that the proposed algorithm outperformed the others in terms of output-approximation quality and computational complexity. Full article
Figures

Open AccessArticle A Conjecture Regarding the Extremal Values of Graph Entropy Based on Degree Powers
Entropy 2016, 18(5), 183; doi:10.3390/e18050183
Received: 14 March 2016 / Revised: 10 May 2016 / Accepted: 10 May 2016 / Published: 13 May 2016
Cited by 1 | PDF Full-text (700 KB) | HTML Full-text | XML Full-text
Abstract
Many graph invariants have been used for the construction of entropy-based measures to characterize the structure of complex networks. The starting point has been always based on assigning a probability distribution to a network when using Shannon’s entropy. In particular, Cao et al.
[...] Read more.
Many graph invariants have been used for the construction of entropy-based measures to characterize the structure of complex networks. The starting point has been always based on assigning a probability distribution to a network when using Shannon’s entropy. In particular, Cao et al. (2014 and 2015) defined special graph entropy measures which are based on degrees powers. In this paper, we obtain some lower and upper bounds for these measures and characterize extremal graphs. Moreover we resolve one part of a conjecture stated by Cao et al. Full article
Open AccessArticle Estimation of Tsunami Bore Forces on a Coastal Bridge Using an Extreme Learning Machine
Entropy 2016, 18(5), 167; doi:10.3390/e18050167
Received: 7 September 2015 / Revised: 26 November 2015 / Accepted: 9 December 2015 / Published: 28 April 2016
Cited by 1 | PDF Full-text (4680 KB) | HTML Full-text | XML Full-text
Abstract
This paper proposes a procedure to estimate tsunami wave forces on coastal bridges through a novel method based on Extreme Learning Machine (ELM) and laboratory experiments. This research included three water depths, ten wave heights, and four bridge models with a variety of
[...] Read more.
This paper proposes a procedure to estimate tsunami wave forces on coastal bridges through a novel method based on Extreme Learning Machine (ELM) and laboratory experiments. This research included three water depths, ten wave heights, and four bridge models with a variety of girders providing a total of 120 cases. The research was designed and adapted to estimate tsunami bore forces including horizontal force, vertical uplift and overturning moment on a coastal bridge. The experiments were carried out on 1:40 scaled concrete bridge models in a wave flume with dimensions of 24 m × 1.5 m × 2 m. Two six-axis load cells and four pressure sensors were installed to the base plate to measure forces. In the numerical procedure, estimation and prediction results of the ELM model were compared with Genetic Programming (GP) and Artificial Neural Networks (ANNs) models. The experimental results showed an improvement in predictive accuracy, and capability of generalization could be achieved by the ELM approach in comparison with GP and ANN. Moreover, results indicated that the ELM models developed could be used with confidence for further work on formulating novel model predictive strategy for tsunami bore forces on a coastal bridge. The experimental results indicated that the new algorithm could produce good generalization performance in most cases and could learn thousands of times faster than conventional popular learning algorithms. Therefore, it can be conclusively obtained that utilization of ELM is certainly developing as an alternative approach to estimate the tsunami bore forces on a coastal bridge. Full article
Open AccessArticle Finding Influential Users in Social Media Using Association Rule Learning
Entropy 2016, 18(5), 164; doi:10.3390/e18050164
Received: 30 January 2016 / Revised: 12 April 2016 / Accepted: 22 April 2016 / Published: 27 April 2016
Cited by 5 | PDF Full-text (396 KB) | HTML Full-text | XML Full-text
Abstract
Influential users play an important role in online social networks since users tend to have an impact on one other. Therefore, the proposed work analyzes users and their behavior in order to identify influential users and predict user participation. Normally, the success of
[...] Read more.
Influential users play an important role in online social networks since users tend to have an impact on one other. Therefore, the proposed work analyzes users and their behavior in order to identify influential users and predict user participation. Normally, the success of a social media site is dependent on the activity level of the participating users. For both online social networking sites and individual users, it is of interest to find out if a topic will be interesting or not. In this article, we propose association learning to detect relationships between users. In order to verify the findings, several experiments were executed based on social network analysis, in which the most influential users identified from association rule learning were compared to the results from Degree Centrality and Page Rank Centrality. The results clearly indicate that it is possible to identify the most influential users using association rule learning. In addition, the results also indicate a lower execution time compared to state-of-the-art methods. Full article
Open AccessArticle An Informed Framework for Training Classifiers from Social Media
Entropy 2016, 18(4), 130; doi:10.3390/e18040130
Received: 18 January 2016 / Revised: 22 March 2016 / Accepted: 28 March 2016 / Published: 9 April 2016
PDF Full-text (6825 KB) | HTML Full-text | XML Full-text | Supplementary Files
Abstract
Extracting information from social media has become a major focus of companies and researchers in recent years. Aside from the study of the social aspects, it has also been found feasible to exploit the collaborative strength of crowds to help solve classical machine
[...] Read more.
Extracting information from social media has become a major focus of companies and researchers in recent years. Aside from the study of the social aspects, it has also been found feasible to exploit the collaborative strength of crowds to help solve classical machine learning problems like object recognition. In this work, we focus on the generally underappreciated problem of building effective datasets for training classifiers by automatically assembling data from social media. We detail some of the challenges of this approach and outline a framework that uses expanded search queries to retrieve more qualified data. In particular, we concentrate on collaboratively tagged media on the social platform Flickr, and on the problem of image classification to evaluate our approach. Finally, we describe a novel entropy-based method to incorporate an information-theoretic principle to guide our framework. Experimental validation against well-known public datasets shows the viability of this approach and marks an improvement over the state of the art in terms of simplicity and performance. Full article
Open AccessArticle The Effect of Threshold Values and Weighting Factors on the Association between Entropy Measures and Mortality after Myocardial Infarction in the Cardiac Arrhythmia Suppression Trial (CAST)
Entropy 2016, 18(4), 129; doi:10.3390/e18040129
Received: 28 January 2016 / Revised: 23 March 2016 / Accepted: 31 March 2016 / Published: 8 April 2016
Cited by 5 | PDF Full-text (683 KB) | HTML Full-text | XML Full-text | Supplementary Files
Abstract
Heart rate variability (HRV) is a non-invasive measurement based on the intervals between normal heart beats that characterize cardiac autonomic function. Decreased HRV is associated with increased risk of cardiovascular events. Characterizing HRV using only moment statistics fails to capture abnormalities in regulatory
[...] Read more.
Heart rate variability (HRV) is a non-invasive measurement based on the intervals between normal heart beats that characterize cardiac autonomic function. Decreased HRV is associated with increased risk of cardiovascular events. Characterizing HRV using only moment statistics fails to capture abnormalities in regulatory function that are important aspects of disease risk. Thus, entropy measures are a promising approach to quantify HRV for risk stratification. The purpose of this study was to investigate this potential for approximate, corrected approximate, sample, fuzzy, and fuzzy measure entropy and its dependency on the parameter selection. Recently, published parameter sets and further parameter combinations were investigated. Heart rate data were obtained from the "Cardiac Arrhythmia Suppression Trial (CAST) RR Interval Sub-Study Database" (Physionet). Corresponding outcomes and clinical data were provided by one of the investigators. The use of previously-reported parameter sets on the pre-treatment data did not significantly add to the identification of patients at risk for cardiovascular death on follow-up. After arrhythmia suppression treatment, several parameter sets predicted outcomes for all patients and patients without coronary artery bypass grafting (CABG). The strongest results were seen using the threshold parameter as a multiple of the data’s standard deviation ( r = 0 . 2 · σ ). Approximate and sample entropy provided significant hazard ratios for patients without CABG and without diabetes for an entropy maximizing threshold approximation. Additional parameter combinations did not improve the results for pre-treatment data. The results of this study illustrate the influence of parameter selection on entropy measures’ potential for cardiovascular risk stratification and support the potential use of entropy measures in future studies. Full article
Open AccessArticle Feature Selection of Power Quality Disturbance Signals with an Entropy-Importance-Based Random Forest
Entropy 2016, 18(2), 44; doi:10.3390/e18020044
Received: 4 November 2015 / Revised: 3 January 2016 / Accepted: 18 January 2016 / Published: 28 January 2016
Cited by 1 | PDF Full-text (2212 KB) | HTML Full-text | XML Full-text
Abstract
Power quality signal feature selection is an effective method to improve the accuracy and efficiency of power quality (PQ) disturbance classification. In this paper, an entropy-importance (EnI)-based random forest (RF) model for PQ feature selection and disturbance classification is proposed. Firstly, 35 kinds
[...] Read more.
Power quality signal feature selection is an effective method to improve the accuracy and efficiency of power quality (PQ) disturbance classification. In this paper, an entropy-importance (EnI)-based random forest (RF) model for PQ feature selection and disturbance classification is proposed. Firstly, 35 kinds of signal features extracted from S-transform (ST) with random noise are used as the original input feature vector of RF classifier to recognize 15 kinds of PQ signals with six kinds of complex disturbance. During the RF training process, the classification ability of different features is quantified by EnI. Secondly, without considering the features with zero EnI, the optimal perturbation feature subset is obtained by applying the sequential forward search (SFS) method which considers the classification accuracy and feature dimension. Then, the reconstructed RF classifier is applied to identify disturbances. According to the simulation results, the classification accuracy is higher than that of other classifiers, and the feature selection effect of the new approach is better than SFS and sequential backward search (SBS) without EnI. With the same feature subset, the new method can maintain a classification accuracy above 99.7% under the condition of 30 dB or above, and the accuracy under 20 dB is 96.8%. Full article
Open AccessArticle Using Multidimensional ADTPE and SVM for Optical Modulation Real-Time Recognition
Entropy 2016, 18(1), 30; doi:10.3390/e18010030
Received: 26 November 2015 / Revised: 6 January 2016 / Accepted: 11 January 2016 / Published: 16 January 2016
Cited by 1 | PDF Full-text (2211 KB) | HTML Full-text | XML Full-text
Abstract
Based on the feature extraction of multidimensional asynchronous delay-tap plot entropy (ADTPE) and multiclass classification of support vector machine (SVM), we propose a method for recognition of multiple optical modulation formats and various data rates. We firstly present the algorithm of multidimensional ADTPE,
[...] Read more.
Based on the feature extraction of multidimensional asynchronous delay-tap plot entropy (ADTPE) and multiclass classification of support vector machine (SVM), we propose a method for recognition of multiple optical modulation formats and various data rates. We firstly present the algorithm of multidimensional ADTPE, which is extracted from asynchronous delay sampling pairs of modulated optical signal. Then, a multiclass SVM is utilized for fast and accurate classification of several widely-used optical modulation formats. In addition, a simple real-time recognition scheme is designed to reduce the computation time. Compared to the existing method based on asynchronous delay-tap plot (ADTP), the theoretical analysis and simulation results show that our recognition method can effectively enhance the tolerance of transmission impairments, obtaining relatively high accuracy. Finally, it is further demonstrated that the proposed method can be integrated in an optical transport network (OTN) with flexible expansion. Through simply adding the corresponding sub-SVM module in the digital signal processer (DSP), arbitrary new modulation formats can be recognized with high recognition accuracy in a short response time. Full article
Open AccessArticle Fruit Classification by Wavelet-Entropy and Feedforward Neural Network Trained by Fitness-Scaled Chaotic ABC and Biogeography-Based Optimization
Entropy 2015, 17(8), 5711-5728; doi:10.3390/e17085711
Received: 22 May 2015 / Revised: 22 July 2015 / Accepted: 28 July 2015 / Published: 7 August 2015
Cited by 47 | PDF Full-text (1374 KB) | HTML Full-text | XML Full-text
Abstract
Fruit classification is quite difficult because of the various categories and similar shapes and features of fruit. In this work, we proposed two novel machine-learning based classification methods. The developed system consists of wavelet entropy (WE), principal component analysis (PCA), feedforward neural network
[...] Read more.
Fruit classification is quite difficult because of the various categories and similar shapes and features of fruit. In this work, we proposed two novel machine-learning based classification methods. The developed system consists of wavelet entropy (WE), principal component analysis (PCA), feedforward neural network (FNN) trained by fitness-scaled chaotic artificial bee colony (FSCABC) and biogeography-based optimization (BBO), respectively. The K-fold stratified cross validation (SCV) was utilized for statistical analysis. The classification performance for 1653 fruit images from 18 categories showed that the proposed “WE + PCA + FSCABC-FNN” and “WE + PCA + BBO-FNN” methods achieve the same accuracy of 89.5%, higher than state-of-the-art approaches: “(CH + MP + US) + PCA + GA-FNN ” of 84.8%, “(CH + MP + US) + PCA + PSO-FNN” of 87.9%, “(CH + MP + US) + PCA + ABC-FNN” of 85.4%, “(CH + MP + US) + PCA + kSVM” of 88.2%, and “(CH + MP + US) + PCA + FSCABC-FNN” of 89.1%. Besides, our methods used only 12 features, less than the number of features used by other methods. Therefore, the proposed methods are effective for fruit classification. Full article

Journal Contact

MDPI AG
Entropy Editorial Office
St. Alban-Anlage 66, 4052 Basel, Switzerland
E-Mail: 
Tel. +41 61 683 77 34
Fax: +41 61 302 89 18
Editorial Board
Contact Details Submit to Entropy Edit a special issue Review for Entropy
logo
loading...
Back to Top