Entropy

Violence detection aims to locate violent content in video frames. Improving the accuracy of violence detection is of great importance for security. However, the current methods do not make full use of the multi-modal vision and audio information, which affects the accuracy of violence detection. We found that the violence detection accuracy of different kinds of videos is related to the change of optical flow. With this in mind, we propose an optical flow-aware-based multi-modal fusion network (OAMFN) for violence detection. Specifically, we use three different fusion strategies to fully integrate multi-modal features. First, the main branch concatenates RGB features and audio features and the optical flow branch concatenates optical flow features with RGB features and audio features, respectively. Then, the cross-modal information fusion module integrates the features of different combinations and applies weights to them to capture cross-modal information in audio and video. After that, the channel attention module extracts valuable information by weighting the integration features. Furthermore, an optical flow-aware-based score fusion strategy is introduced to fuse features of different modalities from two branches. Compared with methods on the XD-Violence dataset, our multi-modal fusion network yields APs that are 83.09% and 1.4% higher than those of the state-of-the-art methods in offline detection, and 78.09% and 4.42% higher than those of the state-of-the-art methods in online detection. Full article

In the double carbon background, riding the wind of new energy vehicles and the battery high nickelization, nickel resources rise along with the trend. In recent years, due to the influence of geopolitical conflicts and emergencies, as well as the speculation and control of international capital with its advantages and rules, the world may face price and security supply risks to a certain extent. Therefore, to obtain the most objective trade redistribution strategy, this paper first constructs the nickel material trade network, identifies the core trading countries and the main trade relations of nickel material trade, and finds that the flow of nickel material mainly occurred between a few countries. On this basis, a trade redistribution model is constructed based on the maximum entropy principle. Taking Indonesia, the largest exporter, and the largest trade relationship (Indonesia exports to China) as examples, the nickel material redistribution between countries when different supply risks occur are simulated. The results can provide an important reference for national resource recovery after the risk of the nickel trade. Full article

This paper considers a multivariate time series model for stock prices in the stock market. A multivariate heterogeneous autoregressive (HAR) model is adopted with exponentially decaying coefficients. This model is not only suitable for multivariate data with strong cross-correlation and long memory, but also represents a common structure of the joint data in terms of decay rates. Tests are proposed to identify the existence of the decay rates in the multivariate HAR model. The null limiting distributions are established as the standard Brownian bridge and are proven by means of a modified martingale central limit theorem. Simulation studies are conducted to assess the performance of tests and estimates. Empirical analysis with joint datasets of U.S. stock prices illustrates that the proposed model outperforms the conventional HAR models via OLSE and LASSO with respect to residual errors. Full article

Error detection is a critical step in data cleaning. Most traditional error detection methods are based on rules and external information with high cost, especially when dealing with large-scaled data. Recently, with the advances of deep learning, some researchers focus their attention on learning the semantic distribution of data for error detection; however, the low error rate in real datasets makes it hard to collect negative samples for training supervised deep learning models. Most of the existing deep-learning-based error detection algorithms solve the class imbalance problem by data augmentation. Due to the inadequate sampling of negative samples, the features learned by those methods may be biased. In this paper, we propose an AEGAN (Auto-Encoder Generative Adversarial Network)-based deep learning model named SAT-GAN (Self-Attention Generative Adversarial Network) to detect errors in relational datasets. Combining the self-attention mechanism with the pre-trained language model, our model can capture semantic features of the dataset, specifically the functional dependency between attributes, so that no rules or constraints are needed for SAT-GAN to identify inconsistent data. For the lack of negative samples, we propose to train our model via zero-shot learning. As a clean-data tailored model, SAT-GAN tries to recognize error data as outliers by learning the latent features of clean data. In our evaluation, SAT-GAN achieves an average $F_1$-score of 0.95 on five datasets, which yields at least 46.2% $F_1$-score improvement over rule-based methods and outperforms state-of-the-art deep learning approaches in the absence of rules and negative samples. Full article

Metrological methods for word learning list tests can be developed with an information theoretical approach extending earlier simple syntax studies. A classic Brillouin entropy expression is applied to the analysis of the Rey’s Auditory Verbal Learning Test RAVLT (immediate recall), where more ordered tasks—with less entropy—are easier to perform. The findings from three case studies are described, including 225 assessments of the NeuroMET2 cohort of persons spanning a cognitive spectrum from healthy older adults to patients with dementia. In the first study, ordinality in the raw scores is compensated for, and item and person attributes are separated with the Rasch model. In the second, the RAVLT IR task difficulty, including serial position effects (SPE), particularly Primacy and Recency, is adequately explained (Pearson’s correlation $R=0.80$) with construct specification equations (CSE). The third study suggests multidimensionality is introduced by SPE, as revealed through goodness-of-fit statistics of the Rasch analyses. Loading factors common to two kinds of principal component analyses (PCA) for CSE formulation and goodness-of-fit logistic regressions are identified. More consistent ways of defining and analysing memory task difficulties, including SPE, can maintain the unique metrological properties of the Rasch model and improve the estimates and understanding of a person’s memory abilities on the path towards better-targeted and more fit-for-purpose diagnostics. Full article

A color image encryption algorithm based on double fractional order chaotic neural network (CNN), interlaced dynamic deoxyribonucleic acid (DNA) encoding and decoding, zigzag confusion, bidirectional bit-level diffusion and convolution operation is proposed. Firstly, two fractional order chaotic neural networks (CNNs) are proposed to explore the application of fractional order CNN in image encryption. Meanwhile, spectral entropy (SE) algorithm shows that the sequence generated by the proposed fractional order CNNs has better randomness. Secondly, a DNA encoding and decoding encryption scheme with evolutionary characteristics is adopted. In addition, convolution operation is utilized to improve the key sensitivity. Finally, simulation results and security analysis illustrate that the proposed algorithm has high security performance and can withstand classical cryptanalysis attacks. Full article

We characterise the geometry of the statistical Roegenian manifold that arises from the equilibrium distribution of an income of noninteracting identical economic actors. The main results for ideal income are included in three subsections: partition function in distribution, scalar curvature, and geodesics. Although this system displays no phase transition, its analysis provides an enlightening contrast with the results of Van der Waals Income in Roegenian Economics, where we shall examine the geometry of the economic Van der Waals income, which does exhibit a “monetary policy as liquidity—income” transition. Here we focus on three subsections: canonical partition function, economic limit, and information geometry of the economic Van der Waals manifold. Full article

In the process of disease identification, as the number of diseases increases, the collection of both diseases and symptoms becomes larger. However, existing computer-aided diagnosis systems do not completely solve the dimensional disaster caused by the increasing data set. To address the above problems, we propose methods of using symptom filtering and a weighted network with the goal of deeper processing of the collected symptom information. Symptom filtering is similar to a filter in signal transmission, which can filter the collected symptom information, further reduce the dimensional space of the system, and make the important symptoms more prominent. The weighted network, on the other hand, mines deeper disease information by modeling the channels of symptom information, amplifying important information, and suppressing unimportant information. Compared with existing hierarchical reinforcement learning models, the feature extraction methods proposed in this paper can help existing models improve their accuracy by more than 10%. Full article

The varied cognitive abilities and rich adaptive behaviors enabled by the animal nervous system are often described in terms of information processing. This framing raises the issue of how biological neural circuits actually process information, and some of the most fundamental outstanding questions in neuroscience center on understanding the mechanisms of neural information processing. Classical information theory has long been understood to be a natural framework within which information processing can be understood, and recent advances in the field of multivariate information theory offer new insights into the structure of computation in complex systems. In this review, we provide an introduction to the conceptual and practical issues associated with using multivariate information theory to analyze information processing in neural circuits, as well as discussing recent empirical work in this vein. Specifically, we provide an accessible introduction to the partial information decomposition (PID) framework. PID reveals redundant, unique, and synergistic modes by which neurons integrate information from multiple sources. We focus particularly on the synergistic mode, which quantifies the “higher-order” information carried in the patterns of multiple inputs and is not reducible to input from any single source. Recent work in a variety of model systems has revealed that synergistic dynamics are ubiquitous in neural circuitry and show reliable structure–function relationships, emerging disproportionately in neuronal rich clubs, downstream of recurrent connectivity, and in the convergence of correlated activity. We draw on the existing literature on higher-order information dynamics in neuronal networks to illustrate the insights that have been gained by taking an information decomposition perspective on neural activity. Finally, we briefly discuss future promising directions for information decomposition approaches to neuroscience, such as work on behaving animals, multi-target generalizations of PID, and time-resolved local analyses. Full article

On 31 December 2019, a cluster of pneumonia cases of unknown etiology was reported in Wuhan (China). The cases were declared to be Coronavirus Disease 2019 (COVID-19) by the World Health Organization (WHO). COVID-19 has been defined as SARS Coronavirus 2 (SARS-CoV-2). Some countries, e.g., Italy, France, and the United Kingdom (UK), have been subjected to frequent restrictions for preventing the spread of infection, contrary to other ones, e.g., the United States of America (USA) and Sweden. The restrictions afflicted the evolution of trends with several perturbations that destabilized its normal evolution. Globally, $R_t$ has been used to estimate time-varying reproduction numbers during epidemics. Methods: This paper presents a solution based on Deep Learning (DL) for the analysis and forecasting of epidemic trends in new positive cases of SARS-CoV-2 (COVID-19). It combined a neural network (NN) and an $R_t$ estimation by adjusting the data produced by the output layer of the NN on the related $R_t$ estimation. Results: Tests were performed on datasets related to the following countries: Italy, the USA, France, the UK, and Sweden. Positive case registration was retrieved between 24 February 2020 and 11 January 2022. Tests performed on the Italian dataset showed that our solution reduced the Mean Absolute Percentage Error (MAPE) by 28.44%, 39.36%, 22.96%, 17.93%, 28.10%, and 24.50% compared to other ones with the same configuration but that were based on the LSTM, GRU, RNN, ARIMA (1,0,3), and ARIMA (7,2,4) models, or an NN without applying the $R_t$ as a corrective index. It also reduced MAPE by 17.93%, the Mean Absolute Error (MAE) by 34.37%, and the Root Mean Square Error (RMSE) by 43.76% compared to the same model without the adjustment performed by the $R_t$. Furthermore, it allowed an average MAPE reduction of 5.37%, 63.10%, 17.84%, and 14.91% on the datasets related to the USA, France, the UK, and Sweden, respectively. Full article

Many security-related scenarios including cryptography depend on the random generation of passwords, permutations, Latin squares, CAPTCHAs and other types of non-numerical entities. Random generation of each entity type is a different problem with different solutions. This study is an attempt at a unified solution for all of the mentioned problems. This paper is the first of its kind to pose, formulate, analyze and solve the problem of random object generation as the general problem of generating random non-numerical entities. We examine solving the problem via connecting it to the well-studied random number generation problem. To this end, we highlight the challenges and propose solutions for each of them. We explain our method using a case study; random Latin square generation. Full article

In order to further improve the accuracy of fault identification of rolling bearings, a fault diagnosis method based on the modified particle swarm optimization (MPSO) algorithm optimized least square support vector machine (LSSVM), combining parameter optimization variational mode decomposition (VMD) and multi-scale permutation entropy (MPE), was proposed. Firstly, to solve the problem of insufficient decomposition and mode mixing caused by the improper selection of mode component K and penalty factor α in VMD algorithm, the whale optimization algorithm (WOA) was used to optimize the penalty factor and mode component number in the VMD algorithm, and the optimal parameter combination (K, α) was obtained. Secondly, the optimal parameter combination (K, α) was used for the VMD of the rolling bearing vibration signal to obtain several intrinsic mode functions (IMFs). According to the Pearson correlation coefficient (PCC) criterion, the optimal IMF component was selected, and its optimal multi-scale permutation entropy was calculated to form the feature set. Finally, K-fold cross-validation was used to train the MPSO-LSSVM model, and the test set was input into the trained model for identification. The experimental results show that compared with PSO-SVM, LSSVM, and PSO-LSSVM, the MPSO-LSSVM fault diagnosis model has higher recognition accuracy. At the same time, compared with VMD-SE, VMD-MPE, and PSO-VMD-MPE, WOA-VMD-MPE can extract more accurate features. Full article

(1) Background: in recent years, a lot of the research of statistical methods focused on the classification problem in presence of imprecise data. A particular case of imprecise data is the interval-valued data. Following this research line, in this work a new hierarchical classification technique for multivariate interval-valued data is suggested for diagnosis of the breast cancer; (2) Methods: an unsupervised hierarchical classification method for imprecise multivariate data (called HC-ID) is performed for diagnosis of breast cancer (i.e., to discriminate between benign or malignant masses) and the results have been compared with the conventional (unsupervised) hierarchical classification approach (HC); (3) Results: the application on real data shows that the HC-ID procedure performs better HC procedure in terms of accuracy (HC-ID = 0.80, HC = 0.66) and sensitivity (HC-ID = 0.61, HC = 0.08). In the results obtained by the usual procedure, there is a high degree of false-negative (i.e., benign cancer diagnosis in malignant status) affected by the high degree of variability (i.e., uncertainty) characterizing the worst data. Full article

This editorial is intended to provide a brief history of the application of Information Theory to the fields of Computational Biology and Bioinformatics; to succinctly summarize the current state of associated research, and open challenges; and to describe the scope of the invited content for this Special Issue of the journal Entropy with the theme of “Information Theory in Computational Biology” [...] Full article

Based on the canonical correlation analysis, we derive series representations of the probability density function (PDF) and the cumulative distribution function (CDF) of the information density of arbitrary Gaussian random vectors as well as a general formula to calculate the central moments. Using the general results, we give closed-form expressions of the PDF and CDF and explicit formulas of the central moments for important special cases. Furthermore, we derive recurrence formulas and tight approximations of the general series representations, which allow efficient numerical calculations with an arbitrarily high accuracy as demonstrated with an implementation in Python publicly available on GitLab. Finally, we discuss the (in)validity of Gaussian approximations of the information density. Full article

The energy loss of the multi-stage centrifugal pump was investigated by numerical analysis using the entropy generation method with the RNG k-ε turbulence model. Entropy generation due to time-averaged motion and velocity fluctuation was mainly considered. It was found that the entropy generation of guide vanes and impellers account for 71.2% and 23.3% of the total entropy generation under the designed flow condition. The guide vanes are the main hydraulic loss domains and their entropy generation is about 9 W/K, followed by impellers. There are vortices at the tongue of the guide vane inlet as well as flow separations in the impellers, which lead to entropy generation. The fluid impacts the outer surface of the guide vanes, resulting in the increase in entropy generation. There are refluxes near the guide vane tongues which also increase the entropy generation of this part. The entropy generation distribution of the guide vanes and impellers was investigated, which found that the positive guide vane has more entropy generation compared with the reverse guide. The entropy generation of the blade suction surface is higher compared with the pressure surface. This study indicated that the entropy generation method has distinct advantages in the assessment of hydraulic loss. Full article

In this paper, a theoretical numerical analysis of the thermodynamics second law in ammonia/ethylene counter-flow diffusion flames is carried out. The combustion process, which includes heat and mass transfer, as well as a chemical reaction, is simulated based on a detailed chemical reaction model. Entropy generation and exergy loss due to various reasons in ammonia/ethylene and argon/ethylene flames are calculated. The effects of ammonia addition on the thermodynamics efficiency of combustion are investigated. Based on thermodynamics analysis, a parameter, the lowest emission of pollutant (LEP), is proposed to establish a relationship between the available work and pollutant emissions produced during the combustion process. Chemical reaction paths are also analyzed by combining the chemical entropy generation, and some important chemical reactions and substances are identified. The numerical results reveal that ammonia addition has a significant enhancement on heat transfer and chemical reaction in the flames, and the total exergy loss rate increases slightly at first and then decreases with an increase in ammonia concentration. Considering the factors of thermodynamic efficiency, the emissions of CO₂ and NOx reach a maximum when ammonia concentration is near 10% and 30%, respectively. In terms of the chemical reaction path analysis, ammonia pyrolysis and nitrogen production increase significantly, while ethylene pyrolysis and carbon monoxide production decrease when ammonia is added to hydrocarbon diffusion flames. Full article

The aim of this study is to assess and compare changes in regularity in the 36 European and the U.S. stock market indices within major turbulence periods. Two periods are investigated: the Global Financial Crisis in 2007–2009 and the COVID-19 pandemic outbreak in 2020–2021. The proposed research hypothesis states that entropy of an equity market index decreases during turbulence periods, which implies that regularity and predictability of a stock market index returns increase in such cases. To capture sequential regularity in daily time series of stock market indices, the Sample Entropy algorithm (SampEn) is used. Changes in the SampEn values before and during the particular turbulence period are estimated. The empirical findings are unambiguous and confirm no reason to reject the research hypothesis. Moreover, additional formal statistical analyses indicate that the SampEn results are similar both for developed and emerging European economies. Furthermore, the rolling-window procedure is utilized to assess the evolution of SampEn over time. Full article