Entropy — Open Access Journal

Ship-radiated noise signal has a lot of nonlinear, non-Gaussian, and nonstationary information characteristics, which can reflect the important signs of ship performance. This paper proposes a novel feature extraction technique for ship-radiated noise based on improved intrinsic time-scale decomposition (IITD) and multiscale dispersion entropy (MDE). The proposed feature extraction technique is named IITD-MDE. First, IITD is applied to decompose the ship-radiated noise signal into a series of intrinsic scale components (ISCs). Then, we select the ISC with the main information through the correlation analysis, and calculate the MDE value as feature vectors. Finally, the feature vectors are input into the support vector machine (SVM) for ship classification. The experimental results indicate that the recognition rate of the proposed technique reaches 86% accuracy. Therefore, compared with the other feature extraction methods, the proposed method provides a new solution for classifying different types of ships effectively. Full article

In this paper, a new method for the solution of distribution problem in a fuzzy setting is presented. It consists of two phases. In the first of them, the problem is formulated as the classical, fully fuzzy transportation problem. A new, straightforward numerical method for solving this problem is proposed. This method is implemented using the $\alpha$ -cut approximation of fuzzy values and the probability approach to interval comparison. The method allows us to provide the straightforward fuzzy extension of a simplex method. It is important that the results are fuzzy values. To validate our approach, these results were compared with those obtained using the competing method and those we got using the Monte–Carlo method. In the second phase, the results obtained in the first one (the fuzzy profit) are used as the natural constraints on the parameters of multiobjective task. In our approach to the solution of distribution problem, the fuzzy local criteria based on the overall profit and contracts breaching risks are used. The particular local criteria are aggregated with the use of most popular aggregation modes. To obtain a compromise solution, the compromise general criterion is introduced, which is the aggregation of aggregating modes with the use of level-2 fuzzy sets. As the result, a new two phase method for solving the fuzzy, nonlinear, multiobjective distribution problem aggregating the fuzzy local criteria based on the overall profit and contracts breaching risks has been developed. Based on the comparison of the results obtained using our method with those obtained by competing one, and on the results of the sensitivity analysis, we can conclude that the method may be successfully used in applications. Numerical examples illustrate the proposed method. Full article

Synthetic data has been shown to be effective in training state-of-the-art neural machine translation (NMT) systems. Because the synthetic data is often generated by back-translating monolingual data from the target language into the source language, it potentially contains a lot of noise—weakly paired sentences or translation errors. In this paper, we propose a novel approach to filter this noise from synthetic data. For each sentence pair of the synthetic data, we compute a semantic similarity score using bilingual word embeddings. By selecting sentence pairs according to these scores, we obtain better synthetic parallel data. Experimental results on the IWSLT 2017 Korean→English translation task show that despite using much less data, our method outperforms the baseline NMT system with back-translation by up to 0.72 and 0.62 Bleu points for tst2016 and tst2017, respectively. Full article

In order to respond to environmental signals, cells often use small molecular circuits to transmit information about their surroundings. Recently, motivated by specific examples in signaling and gene regulation, a body of work has focused on the properties of circuits that function out of equilibrium and dissipate energy. We briefly review the probabilistic measures of information and dissipation and use simple models to discuss and illustrate trade-offs between information and dissipation in biological circuits. We find that circuits with non-steady state initial conditions can transmit more information at small readout delays than steady state circuits. The dissipative cost of this additional information proves marginal compared to the steady state dissipation. Feedback does not significantly increase the transmitted information for out of steady state circuits but does decrease dissipative costs. Lastly, we discuss the case of bursty gene regulatory circuits that, even in the fast switching limit, function out of equilibrium. Full article

We study the statistical underpinnings of life, in particular its increase in order and complexity over evolutionary time. We question some common assumptions about the thermodynamics of life. We recall that contrary to widespread belief, even in a closed system entropy growth can accompany an increase in macroscopic order. We view metabolism in living things as microscopic variables directly driven by the second law of thermodynamics, while viewing the macroscopic variables of structure, complexity and homeostasis as mechanisms that are entropically favored because they open channels for entropy to grow via metabolism. This perspective reverses the conventional relation between structure and metabolism, by emphasizing the role of structure for metabolism rather than the converse. Structure extends in time, preserving information along generations, particularly in the genetic code, but also in human culture. We argue that increasing complexity is an inevitable tendency for systems with these dynamics and explain this with the notion of metastable states, which are enclosed regions of the phase-space that we call “bubbles,” and channels between these, which are discovered by random motion of the system. We consider that more complex systems inhabit larger bubbles (have more available states), and also that larger bubbles are more easily entered and less easily exited than small bubbles. The result is that the system entropically wanders into ever-larger bubbles in the foamy phase space, becoming more complex over time. This formulation makes intuitive why the increase in order/complexity over time is often stepwise and sometimes collapses catastrophically, as in biological extinction. Full article

Static standing postural stability has been measured by multiscale entropy (MSE), which is used to measure complexity. In this study, we used the average entropy (AE) to measure the static standing postural stability, as AE is a good measure of disorder. The center of pressure (COP) trajectories were collected from 11 subjects under four kinds of balance conditions, from stable to unstable: bipedal with open eyes, bipedal with closed eyes, unipedal with open eyes, and unipedal with closed eyes. The AE, entropy of entropy (EoE), and MSE methods were used to analyze these COP data, and EoE was found to be a good measure of complexity. The AE of the 11 subjects sequentially increased by 100% as the balance conditions progressed from stable to unstable, but the results of EoE and MSE did not follow this trend. Therefore, AE, rather than EoE or MSE, is a good measure of static standing postural stability. Furthermore, the comparison of EoE and AE plots exhibited an inverted U curve, which is another example of a complexity versus disorder inverted U curve. Full article

Entropy is an uncertainty measure of random variables which mathematically represents the prospective quantity of the information. In this paper, we mainly focus on the estimation for the parameters and entropy of an Inverse Weibull distribution under progressive first-failure censoring using classical (Maximum Likelihood) and Bayesian methods. For Bayesian approaches, the Bayesian estimates are obtained based on both asymmetric (General Entropy, Linex) and symmetric (Squared Error) loss functions. Due to the complex form of Bayes estimates, we cannot get an explicit solution. Therefore, the Lindley method as well as Importance Sampling procedure is applied. Furthermore, using Importance Sampling method, the Highest Posterior Density credible intervals of entropy are constructed. As a comparison, the asymptotic intervals of entropy are also gained. Finally, a simulation study is implemented and a real data set analysis is performed to apply the previous methods. Full article

In this work, we study the problem of inferring a discrete probability distribution using both expert knowledge and empirical data. This is an important issue for many applications where the scarcity of data prevents a purely empirical approach. In this context, it is common to rely first on an a priori from initial domain knowledge before proceeding to an online data acquisition. We are particularly interested in the intermediate regime, where we do not have enough data to do without the initial a priori of the experts, but enough to correct it if necessary. We present here a novel way to tackle this issue, with a method providing an objective way to choose the weight to be given to experts compared to data. We show, both empirically and theoretically, that our proposed estimator is always more efficient than the best of the two models (expert or data) within a constant. Full article

The high fidelity generation of strongly entangled states of many particles, such as cat states, is a particularly demanding challenge. One approach is to drive the system, within a certain final time, as adiabatically as possible, in order to avoid the generation of unwanted excitations. However, excitations can also be generated by the presence of dissipative effects such as dephasing. Here we compare the effectiveness of Local Adiabatic and the FAst QUasi ADiabatic protocols in achieving a high fidelity for a target superposition state both with and without dephasing. In particular, we consider trapped ions set-ups in which each spin interacts with all the others with the uniform coupling strength or with a power-law coupling. In order to mitigate the effects of dephasing, we complement the adiabatic protocols with dynamical decoupling and we test its effectiveness. The protocols we study could be readily implemented with state-of-the-art techniques. Full article

Costa et. al (Frontiers in Physiology (2017) 8255) proved that abnormal features of heart rate variability (HRV) can be discerned by the presence of particular patterns in a signal of time intervals between subsequent heart contractions, called RR intervals. In the following, the statistics of these patterns, quantified using entropic tools, are explored in order to uncover the specifics of the dynamics of heart contraction based on RR intervals. The 33 measures of HRV (standard and new ones) were estimated from four hour nocturnal recordings obtained from 181 healthy people of different ages and analyzed with the machine learning methods. The validation of the methods was based on the results obtained from shuffled data. The exploratory factor analysis provided five factors driving the HRV. We hypothesize that these factors could be related to the commonly assumed physiological sources of HRV: (i) activity of the vagal nervous system; (ii) dynamical balance in the autonomic nervous system; (iii) sympathetic activity; (iv) homeostatic stability; and (v) humoral effects. In particular, the indices describing patterns: their total volume, as well as their distribution, showed important aspects of the organization of the ANS control: the presence or absence of a strong correlation between the patterns’ indices, which distinguished the original rhythms of people from their shuffled representatives. Supposing that the dynamic organization of RR intervals is age dependent, classification with the support vector machines was performed. The classification results proved to be strongly dependent on the parameters of the methods used, therefore determining that the age group was not obvious. Full article

Cross-efficiency evaluation is an effective methodology for discriminating among a set of decision-making units (DMUs) through both self- and peer-evaluation methods. This evaluation technique is usually used for data envelopment analysis (DEA) models with constant returns to scale due to the fact that negative efficiencies never happen in this case. For cases of variable returns to scale (VRSs), the evaluation may generate negative cross-efficiencies. However, when the production technology is known to be VRS, a VRS model must be used. In this case, negative efficiencies may occur. Negative efficiencies are unreasonable and cause difficulties in calculating the final cross-efficiency. In this paper, we propose a cross-efficiency evaluation method, with the technology of VRS. The cross-efficiency intervals of DMUs were derived from the associated aggressive and benevolent formulations. More importantly, the proposed approach does not produce negative efficiencies. For comparison of DMUs with their cross-efficiency intervals, a numerical index is required. Since the concept of entropy is an effective tool to measure the uncertainty, this concept was employed to build an index for ranking DMUs with cross efficiency intervals. A real-case example was used to illustrate the approach proposed in this paper. Full article

Given the recent replacement of value-at-risk as the regulatory standard measure of risk with expected shortfall (ES) undertaken by the Basel Committee on Banking Supervision, it is imperative that ES gives correct estimates for the value of expected levels of losses in crisis situations. However, the measurement of ES is affected by a lack of observations in the tail of the distribution. While kernel-based smoothing techniques can be used to partially circumvent this problem, in this paper we propose a simple nonparametric tail measure of risk based on information entropy and compare its backtesting performance with that of other standard ES models. Full article

Quality of sleep can be assessed by analyzing the cyclic alternating pattern, a long-lasting periodic activity that is composed of two alternate electroencephalogram patterns, which is considered to be a marker of sleep instability. Experts usually score this pattern through a visual examination of each one-second epoch of an electroencephalogram signal, a repetitive and time-consuming task that is prone to errors. To address these issues, a home monitoring device was developed for automatic scoring of the cyclic alternating pattern by analyzing the signal from one electroencephalogram derivation. Three classifiers, specifically, two recurrent networks (long short-term memory and gated recurrent unit) and one one-dimension convolutional neural network, were developed and tested to determine which was more suitable for the cyclic alternating pattern phase’s classification. It was verified that the network based on the long short-term memory attained the best results with an average accuracy, sensitivity, specificity and area under the receiver operating characteristic curve of, respectively, 76%, 75%, 77% and 0.752. The classified epochs were then fed to a finite state machine to determine the cyclic alternating pattern cycles and the performance metrics were 76%, 71%, 84% and 0.778, respectively. The performance achieved is in the higher bound of the experts’ expected agreement range and considerably higher than the inter-scorer agreement of multiple experts, implying the usability of the device developed for clinical analysis. Full article

The size of the largest binary single deletion code has been unknown for more than 50 years. It is known that Varshamov–Tenengolts (VT) code is an optimum single deletion code for block length $n\le 10$ ; however, only a few upper bounds of the size of single deletion code are proposed for larger n. We provide improved upper bounds using Mixed Integer Linear Programming (MILP) relaxation technique. Especially, we show the size of single deletion code is smaller than or equal to 173 when the block length n is 11. In the second half of the paper, we propose a conjecture that is equivalent to the long-lasting conjecture that “VT code is optimum for all n”. This equivalent formulation of the conjecture contains small sub-problems that can be numerically verified. We provide numerical results that support the conjecture. Full article

The entropy rate h of a natural language quantifies the complexity underlying the language. While recent studies have used computational approaches to estimate this rate, their results rely fundamentally on the performance of the language model used for prediction. On the other hand, in 1951, Shannon conducted a cognitive experiment to estimate the rate without the use of any such artifact. Shannon’s experiment, however, used only one subject, bringing into question the statistical validity of his value of $h=1.3$ bits per character for the English language entropy rate. In this study, we conducted Shannon’s experiment on a much larger scale to reevaluate the entropy rate h via Amazon’s Mechanical Turk, a crowd-sourcing service. The online subjects recruited through Mechanical Turk were each asked to guess the succeeding character after being given the preceding characters until obtaining the correct answer. We collected 172,954 character predictions and analyzed these predictions with a bootstrap technique. The analysis suggests that a large number of character predictions per context length, perhaps as many as ${10}^3$ , would be necessary to obtain a convergent estimate of the entropy rate, and if fewer predictions are used, the resulting h value may be underestimated. Our final entropy estimate was $h\approx 1.22$ bits per character. Full article

We consider heat conduction in a superlattice with mobile defects, which reduce the thermal conductivity of the material. If the defects may be dragged by the heat flux, and if they are stopped at the interfaces of the superlattice, it is seen that the effective thermal resistance of the layers will depend on the heat flux. Thus, the concentration dependence of the transport coefficients plus the mobility of the defects lead to a strongly nonlinear behavior of heat transport, which may be used in some cases as a basis for thermal transistors. Full article

The motor imagery-based brain-computer interface (BCI) using electroencephalography (EEG) has been receiving attention from neural engineering researchers and is being applied to various rehabilitation applications. However, the performance degradation caused by motor imagery EEG with very low single-to-noise ratio faces several application issues with the use of a BCI system. In this paper, we propose a novel motor imagery classification scheme based on the continuous wavelet transform and the convolutional neural network. Continuous wavelet transform with three mother wavelets is used to capture a highly informative EEG image by combining time-frequency and electrode location. A convolutional neural network is then designed to both classify motor imagery tasks and reduce computation complexity. The proposed method was validated using two public BCI datasets, BCI competition IV dataset 2b and BCI competition II dataset III. The proposed methods were found to achieve improved classification performance compared with the existing methods, thus showcasing the feasibility of motor imagery BCI. Full article

Integrated Information Theory proposes a measure of conscious activity ( $\Phi$ ), characterised as the irreducibility of a dynamical system to the sum of its components. Due to its computational cost, current versions of the theory (IIT 3.0) are difficult to apply to systems larger than a dozen units, and, in general, it is not well known how integrated information scales as systems grow larger in size. In this article, we propose to study the scaling behaviour of integrated information in a simple model of a critical phase transition: an infinite-range kinetic Ising model. In this model, we assume a homogeneous distribution of couplings to simplify the computation of integrated information. This simplified model allows us to critically review some of the design assumptions behind the measure and connect its properties with well-known phenomena in phase transitions in statistical mechanics. As a result, we point to some aspects of the mathematical definitions of IIT that 3.0 fail to capture critical phase transitions and propose a reformulation of the assumptions made by integrated information measures. Full article

Low-temperature heat utilization can be applied to waste heat from industrial processes or renewable energy sources such as geothermal and ocean energy. The most common low-temperature waste-heat recovery technology is the organic Rankine cycle (ORC). However, the phase change of ORC working fluid for the heat extraction process causes a pinch-point problem, and the heat recovery cannot be efficiently used. To improve heat extraction and power generation, this study explored the cycle characteristics of the trilateral flash cycle (TFC) in a low-temperature heat source. A pinch-point-based methodology was developed for studying the optimal design point and operating conditions and for optimizing working fluid evaporation temperature and mass flow rate. According to the simulation results, the TFC system can recover more waste heat than ORC under the same operating conditions. The net power output of the TFC was approximately 30% higher than ORC but at a cost of higher pump power consumption. Additionally, the TFC was superior to ORC with an extremely low-temperature heat source (<80 °C), and the ideal efficiency was approximately 3% at the highest work output condition. The TFC system is economically beneficial for waste-heat recovery for low-temperature heat sources. Full article