Entropy

Some problems of describing biological systems with the use of entropy as a measure of the complexity of these systems are considered. Entropy is studied both for the organism as a whole and for its parts down to the molecular level. Correlation of actions of various parts of the whole organism, intercellular interactions and control, as well as cooperativity on the microlevel lead to a more complex structure and lower statistical entropy. For a multicellular organism, entropy is much lower than entropy for the same mass of a colony of unicellular organisms. Cooperativity always reduces the entropy of the system; a simple example of ligand binding to a macromolecule carrying two reaction centers shows how entropy is consistent with the ambiguity of the result in the Bernoulli test scheme. Particular attention is paid to the qualitative and quantitative relationship between the entropy of the system and the cooperativity of ligand binding to macromolecules. A kinetic model of metabolism. corresponding to Schrödinger’s concept of the maintenance biosystems by “negentropy feeding”, is proposed. This model allows calculating the nonequilibrium local entropy and comparing it with the local equilibrium entropy inherent in non-living matter. Full article

Inverted exponentiated Rayleigh distribution is a widely used and important continuous lifetime distribution, which plays a key role in lifetime research. The joint progressively type-II censoring scheme is an effective method used in the quality evaluation of products from different assembly lines. In this paper, we study the statistical inference of inverted exponentiated Rayleigh distribution based on joint progressively type-II censored data. The likelihood function and maximum likelihood estimates are obtained firstly by adopting Expectation-Maximization algorithm. Then, we calculate the observed information matrix based on the missing value principle. Bootstrap-p and Bootstrap-t methods are applied to get confidence intervals. Bayesian approaches under square loss function and linex loss function are provided respectively to derive the estimates, during which the importance sampling method is introduced. Finally, the Monte Carlo simulation and real data analysis are performed for further study. Full article

For a large ensemble of complex systems, a Many-System Problem (MSP) studies how heterogeneity constrains and hides structural mechanisms, and how to uncover and reveal hidden major factors from homogeneous parts. All member systems in an MSP share common governing principles of dynamics, but differ in idiosyncratic characteristics. A typical dynamic is found underlying response features with respect to covariate features of quantitative or qualitative data types. Neither all-system-as-one-whole nor individual system-specific functional structures are assumed in such response-vs-covariate (Re–Co) dynamics. We developed a computational protocol for identifying various collections of major factors of various orders underlying Re–Co dynamics. We first demonstrate the immanent effects of heterogeneity among member systems, which constrain compositions of major factors and even hide essential ones. Secondly, we show that fuller collections of major factors are discovered by breaking heterogeneity into many homogeneous parts. This process further realizes Anderson’s “More is Different” phenomenon. We employ the categorical nature of all features and develop a Categorical Exploratory Data Analysis (CEDA)-based major factor selection protocol. Information theoretical measurements—conditional mutual information and entropy—are heavily used in two selection criteria: C1—confirmable and C2—irreplaceable. All conditional entropies are evaluated through contingency tables with algorithmically computed reliability against the finite sample phenomenon. We study one artificially designed MSP and then two real collectives of Major League Baseball (MLB) pitching dynamics with 62 slider pitchers and 199 fastball pitchers, respectively. Finally, our MSP data analyzing techniques are applied to resolve a scientific issue related to the Rosenberg Self-Esteem Scale. Full article

As a data augmentation method, masking word is commonly used in many natural language processing tasks. However, most mask methods are based on rules and are not related to downstream tasks. In this paper, we propose a novel masking word generator, named Actor-Critic Mask Model (ACMM), which can adaptively adjust the mask strategy according to the performance of downstream tasks. In order to demonstrate the effectiveness of the method, we conducted experiments on two causal event extraction datasets. Experiment results show that, compared with various rule-based masking methods, the masked sentences generated by our proposed method can significantly enhance the generalization of the model and improve the model performance. Full article

Minimal rotating thermodynamic systems are addressed. Particle m placed into the rotating symmetrical double-well potential (bowl), providing binary logical system is considered. The condition providing the transfer of the particle from one frictionless half-well to another, and, in this way, the possibility to record 1 bit of information is derived. The procedure of recording turns out to be irreversible; it is impossible to return the particle to its initial state under rotation about the same axis. The same rotating double-well system exerted to the thermal noise is considered. A minimal rotating thermal engine built of the rotating chamber, movable partition, and the particle confined within the chamber is treated. Rotation of the system displaces the partition, thus enabling erasing of one bit information. Erasing of 1 bit of information is due to the inertia (centrifugal force) acting on the partition. Isothermal expansion of the “minimal gas” expectedly gives rise to the Landauer bound. Compression of the “gas” with the rotation around the same axis is impossible and demands the additional axis of rotation. The interrelation between the possibility of recording/erasing information and the symmetry of the system is considered. Full article

There has been a lot of interest in sufficient dimension reduction (SDR) methodologies, as well as nonlinear extensions in the statistics literature. The SDR methodology has previously been motivated by several considerations: (a) finding data-driven subspaces that capture the essential facets of regression relationships; (b) analyzing data in a `model-free’ manner. In this article, we develop an approach to interpreting SDR techniques using information theory. Such a framework leads to a more assumption-lean understanding of what SDR methods do and also allows for some connections to results in the information theory literature. Full article

The interval-valued q-rung dual hesitant linguistic (IVq-RDHL) sets are widely used to express the evaluation information of decision makers (DMs) in the process of multi-attribute decision-making (MADM). However, the existing MADM method based on IVq-RDHL sets has obvious shortcomings, i.e., the operational rules of IVq-RDHL values have some weaknesses and the existing IVq-RDHL aggregation operators are incapable of dealing with some special decision-making situations. In this paper, by analyzing these drawbacks, we then propose the operations for IVq-RDHL values based on a linguistic scale function. After it, we present novel aggregation operators for IVq-RDHL values based on the power Hamy mean and introduce the IVq-RDHL power Hamy mean operator and IVq-RDHL power weighted Hamy mean operator. Properties of these new aggregation operators are also studied. Based on these foundations, we further put forward a MADM method, which is more reasonable and rational than the existing one. Our proposed method not only provides a series of more reasonable operational laws but also offers a more powerful manner to fuse attribute values. Finally, we apply the new MADM method to solve the practical problem of patient admission evaluation. The performance and advantages of our method are illustrated in the comparative analysis with other methods. Full article

In recent years, the pedestrian detection technology of a single 2D image has been dramatically improved. When the scene becomes very crowded, the detection performance will deteriorate seriously and cannot meet the requirements of autonomous driving perception. With the introduction of the multi-view method, the task of pedestrian detection in crowded or fuzzy scenes has been significantly improved and has become a widely used method in autonomous driving. In this paper, we construct a double-branch feature fusion structure, the first branch adopts a lightweight structure, the second branch further extracts features and gets the feature map obtained from each layer. At the same time, the receptive field is enlarged by expanding convolution. To improve the speed of the model, the keypoint is used instead of the entire object for regression without an NMS post-processing operation. Meanwhile, the whole model can be learned from end to end. Even in the presence of many people, the method can still perform better on accuracy and speed. In the standard of Wildtrack and MultiviewX dataset, the accuracy and running speed both perform better than the state-of-the-art model, which has great practical significance in the autonomous driving field. Full article

In order to reduce maintenance costs and avoid safety accidents, it is of great significance to carry out fault prediction to reasonably arrange maintenance plans for rotating mechanical equipment. At present, the relevant research mainly focuses on fault diagnosis and remaining useful life (RUL) predictions, which cannot provide information on the specific health condition and fault types of rotating mechanical equipment in advance. In this paper, a novel three-stage fault prediction method is presented to realize the identification of the degradation period and the type of failure simultaneously. Firstly, based on the vibration signals from multiple sensors, a convolutional neural network (CNN) and long short-term memory (LSTM) network are combined to extract the spatiotemporal features of the degradation period and fault type by means of the cross-entropy loss function. Then, to predict the degradation trend and the type of failure, the attention-bidirectional (Bi)-LSTM network is used as the regression model to predict the future trend of features. Furthermore, the predicted features are given to the support vector classification (SVC) model to identify the specific degradation period and fault type, which can eventually realize a comprehensive fault prediction. Finally, the NSF I/UCR Center for Intelligent Maintenance Systems (IMS) dataset is used to verify the feasibility and efficiency of the proposed fault prediction method. Full article

This study focuses on the full-form model-free adaptive controller (FFMFAC) for SISO discrete-time nonlinear systems, and proposes enhanced FFMFAC. The proposed technique design incorporates long short-term memory neural networks (LSTMs) and fuzzy neural networks (FNNs). To be more precise, LSTMs are utilized to adjust vital parameters of the FFMFAC online. Additionally, due to the high nonlinear approximation capabilities of FNNs, pseudo gradient (PG) values of the controller are estimated online. EFFMFAC is characterized by utilizing the measured I/O data for the online training of all introduced neural networks and does not involve offline training and specific models of the controlled system. Finally, the rationality and superiority are verified by two simulations and a supporting ablation analysis. Five individual performance indices are given, and the experimental findings show that EFFMFAC outperforms all other methods. Especially compared with the FFMFAC, EFFMFAC reduces the $RMSE$ by 21.69% and 11.21%, respectively, proving it to be applicable for SISO discrete-time nonlinear systems. Full article

We discuss hypothesis testing and compare different theories in light of observed or experimental data as fundamental endeavors in the sciences. Issues associated with the p-value approach and null hypothesis significance testing are reviewed, and the Bayesian alternative based on the Bayes factor is introduced, along with a review of computational methods and sensitivity related to prior distributions. We demonstrate how Bayesian testing can be practically implemented in several examples, such as the t-test, two-sample comparisons, linear mixed models, and Poisson mixed models by using existing software. Caveats and potential problems associated with Bayesian testing are also discussed. We aim to inform researchers in the many fields where Bayesian testing is not in common use of a well-developed alternative to null hypothesis significance testing and to demonstrate its standard implementation. Full article

As a powerful tool for measuring complexity and randomness, multivariate multi-scale permutation entropy (MMPE) has been widely applied to the feature representation and extraction of multi-channel signals. However, MMPE still has some intrinsic shortcomings that exist in the coarse-grained procedure, and it lacks the precise estimation of entropy value. To address these issues, in this paper a novel non-linear dynamic method named composite multivariate multi-scale permutation entropy (CMMPE) is proposed, for optimizing insufficient coarse-grained process in MMPE, and thus to avoid the loss of information. The simulated signals are used to verify the validity of CMMPE by comparing it with the often-used MMPE method. An intelligent fault diagnosis method is then put forward on the basis of CMMPE, Laplacian score (LS), and bat optimization algorithm-based support vector machine (BA-SVM). Finally, the proposed fault diagnosis method is utilized to analyze the test data of rolling bearings and is then compared with the MMPE, multivariate multi-scale multiscale entropy (MMFE), and multi-scale permutation entropy (MPE) based fault diagnosis methods. The results indicate that the proposed fault diagnosis method of rolling bearing can achieve effective identification of fault categories and is superior to comparative methods. Full article

Since the grand partition function $Z_q$ for the so-called q-particles (i.e., quons), $q\in (-1,1)$, cannot be computed by using the standard 2nd quantisation technique involving the full Fock space construction for $q=0$, and its q-deformations for the remaining cases, we determine such grand partition functions in order to obtain the natural generalisation of the Plank distribution to $q\in [-1,1]$. We also note the (non) surprising fact that the right grand partition function concerning the Boltzmann case (i.e., $q=0$) can be easily obtained by using the full Fock space 2nd quantisation, by considering the appropriate correction by the Gibbs factor $1/n!$ in the n term of the power series expansion with respect to the fugacity z. As an application, we briefly discuss the equations of the state for a gas of free quons or the condensation phenomenon into the ground state, also occurring for the Bose-like quons $q\in (0,1)$. Full article

This research aims to compare the performance of ARIMA as a linear model with that of the combination of ARIMA and GARCH family models to forecast S&P500 log returns in order to construct algorithmic investment strategies on this index. We used the data collected from Yahoo Finance with daily frequency for the period from 1 January 2000 to 31 December 2019. By using a rolling window approach, we compared ARIMA with the hybrid models to examine whether hybrid ARIMA-SGARCH and ARIMA-EGARCH can really reflect the specific time-series characteristics and have better predictive power than the simple ARIMA model. In order to assess the precision and quality of these models in forecasting, we compared their equity lines, their forecasting error metrics (MAE, MAPE, RMSE, MAPE), and their performance metrics (annualized return compounded, annualized standard deviation, maximum drawdown, information ratio, and adjusted information ratio). The main contribution of this research is to show that the hybrid models outperform ARIMA and the benchmark (Buy&Hold strategy on S&P500 index) over the long term. These results are not sensitive to varying window sizes, the type of distribution, and the type of the GARCH model. Full article

Driven by the emerging mission-critical and data-intensive applications in industrial intelligent manufacturing, the software-defined network (SDN) based fiber-wireless access network (FiWi) is attracting considerable attention thanks to its capability of central control and large bandwidth. However, the heterogeneity of the network leads to new challenges, since the packet loss can be caused either by the poor channel quality of wireless links or network component failures. A novel and adaptive mechanism combining sparse random linear network coding with parallel transmission (SNC-PT) is proposed to achieve the fault-tolerance against high packet loss rate and any network element malfunction. We illustrate the benefits of using the SNC-PT mechanism to improve fault tolerance by characterizing the network performance with respect to the completion time and goodput along with its relationship to channel quality and node failures. We show that significant performance gains can be obtained in comparison with conventional uncoded transmission based on transmission control protocol (TCP). The simulation results show that the SNC-PT mechanism is fault-tolerant, while it can significantly shorten the data transmission completion time to at least 12% of the baseline and increase the goodput by about 10% compared to other coding schemes such as random linear network coding. Full article

Fine-grained image retrieval aims at searching relevant images among fine-grained classes given a query. The main difficulty of this task derives from the small interclass distinction and the large intraclass variance of fine-grained images, posing severe challenges to the methods that only resort to global or local features. In this paper, we propose a novel fine-grained image retrieval method, where global–local aware feature representation is learned. Specifically, the global feature is extracted by selecting the most relevant deep descriptors. Meanwhile, we explore the intrinsic relationship of different parts via the frequent pattern mining, thus obtaining the representative local feature. Further, an aggregation feature that learns global–local aware feature representation is designed. Consequently, the discriminative ability among different fine-grained classes is enhanced. We evaluate the proposed method on five popular fine-grained datasets. Extensive experimental results demonstrate that the performance of fine-grained image retrieval is improved with the proposed global–local aware representation. Full article

Background: For the kinetic models used in contrast-based medical imaging, the assignment of the arterial input function named AIF is essential for the estimation of the physiological parameters of the tissue via solving an optimization problem. Objective: In the current study, we estimate the AIF relayed on the modified maximum entropy method. The effectiveness of several numerical methods to determine kinetic parameters and the AIF is evaluated—in situations where enough information about the AIF is not available. The purpose of this study is to identify an appropriate method for estimating this function. Materials and Methods: The modified algorithm is a mixture of the maximum entropy approach with an optimization method, named the teaching-learning method. In here, we applied this algorithm in a Bayesian framework to estimate the kinetic parameters when specifying the unique form of the AIF by the maximum entropy method. We assessed the proficiency of the proposed method for assigning the kinetic parameters in the dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), when determining AIF with some other parameter-estimation methods and a standard fixed AIF method. A previously analyzed dataset consisting of contrast agent concentrations in tissue and plasma was used. Results and Conclusions: We compared the accuracy of the results for the estimated parameters obtained from the MMEM with those of the empirical method, maximum likelihood method, moment matching (“method of moments”), the least-square method, the modified maximum likelihood approach, and our previous work. Since the current algorithm does not have the problem of starting point in the parameter estimation phase, it could find the best and nearest model to the empirical model of data, and therefore, the results indicated the Weibull distribution as an appropriate and robust AIF and also illustrated the power and effectiveness of the proposed method to estimate the kinetic parameters. Full article

In the paper, discrete-time multi-agent systems under Denial-of-Service (DoS) attacks are considered. Since in the presence of DoS attacks the stability of the whole system may be disturbed, sufficient stability conditions for the multi-agent system under DoS attacks are delivered. The consensus problem for the special case of the considered system under DoS attacks is also examined by delivering sufficient conditions. Theoretical considerations are illustrated by numerical examples. Full article