Algorithms, Volume 15, Issue 1 (January 2022) – 26 articles

Meta-analysis is an indispensable tool for synthesizing statistical results obtained from individual studies. Recently, non-Bayesian estimators for individual means were proposed by applying three methods: the James–Stein (JS) shrinkage estimator, isotonic regression estimator, and pretest (PT) estimator. In order to make these methods available to users, we develop a new R package meta.shrinkage. Our package can compute seven estimators (named JS, JS+, RML, RJS, RJS+, PT, and GPT). We introduce this R package along with the usage of the R functions and the “average-min-max” steps for the pool-adjacent violators algorithm. We conduct Monte Carlo simulations to validate the proposed R package to ensure that the package can work properly in a variety of scenarios. We also analyze a data example to show the ability of the R package. Full article

Semantic code retrieval is the task of retrieving relevant codes based on natural language queries. Although it is related to other information retrieval tasks, it needs to bridge the gaps between the language used in the code (which is usually syntax-specific and logic-specific) and the natural language which is more suitable for describing ambiguous concepts and ideas. Existing approaches study code retrieval in a natural language for a specific programming language, however it is unwieldy and often requires a large amount of corpus for each language when dealing with multilingual scenarios.Using knowledge distillation of six existing monolingual Teacher Models to train one Student Model—MPLCS (Multi-Programming Language Code Search), this paper proposed a method to support multi-programing language code search tasks. MPLCS has the ability to incorporate multiple languages into one model with low corpus requirements. MPLCS can study the commonality between different programming languages and improve the recall accuracy for small dataset code languages. As for Ruby used in this paper, MPLCS improved its MRR score by 20 to 25%. In addition, MPLCS can compensate the low recall accuracy of monolingual models when perform language retrieval work on other programming languages. And in some cases, MPLCS’ recall accuracy can even outperform the recall accuracy of monolingual models when they perform language retrieval work on themselves. Full article

In the past two decades, metaheuristic optimisation algorithms (MOAs) have been increasingly popular, particularly in logistic, science, and engineering problems. The fundamental characteristics of such algorithms are that they are dependent on a parameter or a strategy. Some online and offline strategies are employed in order to obtain optimal configurations of the algorithms. Adaptive operator selection is one of them, and it determines whether or not to update a strategy from the strategy pool during the search process. In the field of machine learning, Reinforcement Learning (RL) refers to goal-oriented algorithms, which learn from the environment how to achieve a goal. On MOAs, reinforcement learning has been utilised to control the operator selection process. However, existing research fails to show that learned information may be transferred from one problem-solving procedure to another. The primary goal of the proposed research is to determine the impact of transfer learning on RL and MOAs. As a test problem, a set union knapsack problem with 30 separate benchmark problem instances is used. The results are statistically compared in depth. The learning process, according to the findings, improved the convergence speed while significantly reducing the CPU time. Full article

To overcome the shortcomings of the harmony search algorithm, such as its slow convergence rate and poor global search ability, a reward population-based differential genetic harmony search algorithm is proposed. In this algorithm, a population is divided into four ordinary sub-populations and one reward sub-population, for each of which the evolution strategy of the differential genetic harmony search is used. After the evolution, the population with the optimal average fitness is combined with the reward population to produce a new reward population. During an experiment, tests were conducted first on determining the value of the harmony memory size (HMS) and the harmony memory consideration rate (HMCR), followed by an analysis of the effect of their values on the performance of the proposed algorithm. Then, six benchmark functions were selected for the experiment, and a comparison was made on the calculation results of the standard harmony memory search algorithm, reward population harmony search algorithm, differential genetic harmony algorithm, and reward population-based differential genetic harmony search algorithm. The result suggests that the reward population-based differential genetic harmony search algorithm has the merits of a strong global search ability, high solving accuracy, and satisfactory stability. Full article

A general crossword grid generation is considered an NP-complete problem and theoretically it could be a good candidate to be used by cryptography algorithms. In this article, we propose a new algorithm for generating perfect crosswords grids (with no black boxes) that relies on using tries data structures, which are very important for reducing the time for finding the solutions, and offers good opportunity for parallelisation, too. The algorithm uses a special tries representation and it is very efficient, but through parallelisation the performance is improved to a level that allows the solution to be obtained extremely fast. The experiments were conducted using a dictionary of almost 700,000 words, and the solutions were obtained using the parallelised version with an execution time in the order of minutes. We demonstrate here that finding a perfect crossword grid could be solved faster than has been estimated before, if we use tries as supporting data structures together with parallelisation. Still, if the size of the dictionary is increased by a lot (e.g., considering a set of dictionaries for different languages—not only for one), or through a generalisation to a 3D space or multidimensional spaces, then the problem still could be investigated for a possible usage in cryptography. Full article

Analysis of high-dimensional data, with more features ($p$) than observations ($N$) ($p>N$), places significant demand in cost and memory computational usage attributes. Feature selection can be used to reduce the dimensionality of the data. We used a graph-based approach, principal component analysis (PCA) and recursive feature elimination to select features for classification from RNAseq datasets from two lung cancer datasets. The selected features were discretized for association rule mining where support and lift were used to generate informative rules. Our results show that the graph-based feature selection improved the performance of sequential minimal optimization (SMO) and multilayer perceptron classifiers (MLP) in both datasets. In association rule mining, features selected using the graph-based approach outperformed the other two feature-selection techniques at a support of 0.5 and lift of 2. The non-redundant rules reflect the inherent relationships between features. Biological features are usually related to functions in living systems, a relationship that cannot be deduced by feature selection and classification alone. Therefore, the graph-based feature-selection approach combined with rule mining is a suitable way of selecting and finding associations between features in high-dimensional RNAseq data. Full article

Complex networks usually consist of dense-connected cliques, which are defined as communities. A community structure is a reflection of the local characteristics existing in the network topology, this makes community detection become an important research field to reveal the internal structural characteristics of networks. In this article, an information-based community detection approach MINC-NRL is proposed, which can be applied to both overlapping and non-overlapping community detection. MINC-NRL introduces network representation learning (NRL) to represent the target network as vectors, then generates a community evolution process based on these vectors to reduce the search space, and finally, finds the best community partition in this process using mutual information between network and communities (MINC). Experiments on real-world and synthetic data sets verifies the effectiveness of the approach in community detection, both on non-overlapping and overlapping tasks. Full article

Parameter optimization in the field of control engineering has always been a research topic. This paper studies the parameter optimization of an active disturbance rejection controller. The parameter optimization problem in controller design can be summarized as a nonlinear optimization problem with constraints. It is often difficult and complicated to solve the problem directly, and meta-heuristic algorithms are suitable for this problem. As a relatively new method, the ant-lion optimization algorithm has attracted much attention and study. The contribution of this work is proposing an adaptive ant-lion algorithm, namely differential step-scaling ant-lion algorithm, to optimize parameters of the active disturbance rejection controller. Firstly, a differential evolution strategy is introduced to increase the diversity of the population and improve the global search ability of the algorithm. Then the step scaling method is adopted to ensure that the algorithm can obtain higher accuracy in a local search. Comparison with existing optimizers is conducted for different test functions with different qualities, the results show that the proposed algorithm has advantages in both accuracy and convergence speed. Simulations with different algorithms and different indexes are also carried out, the results show that the improved algorithm can search better parameters for the controllers. Full article

In this paper, we consider the $\lambda$-model for an arbitrary-order Cayley tree that has a disordered phase. Such a phase corresponds to a splitting Gibbs measure with free boundary conditions. In communication theory, such a measure appears naturally, and its extremality is related to the solvability of the non-reconstruction problem. In general, the disordered phase is not extreme; hence, it is natural to find a condition for their extremality. In the present paper, we present certain conditions for the extremality of the disordered phase of the $\lambda$-model. Full article

Analog circuits play an important role in modern electronic systems. Aiming to accurately diagnose the faults of analog circuits, this paper proposes a novel variant of a convolutional neural network, namely, a multi-scale convolutional neural network with a selective kernel (MSCNN-SK). In MSCNN-SK, a multi-scale average difference layer is developed to compute multi-scale average difference sequences, and then these sequences are taken as the input of the model, which enables it to mine potential fault characteristics. In addition, a dynamic convolution kernel selection mechanism is introduced to adaptively adjust the receptive field, so that the feature extraction ability of MSCNN-SK is enhanced. Based on two well-known fault diagnosis circuits, comparison experiments are conducted, and experimental results show that our proposed method achieves higher performance. Full article

In this article we present a class of mixed Poisson regression models with varying dispersion arising from non-conjugate to the Poisson mixing distributions for modelling overdispersed claim counts in non-life insurance. The proposed family of models combined with the adopted modelling framework can provide sufficient flexibility for dealing with different levels of overdispersion. For illustrative purposes, the Poisson-lognormal regression model with regression structures on both its mean and dispersion parameters is employed for modelling claim count data from a motor insurance portfolio. Maximum likelihood estimation is carried out via an expectation-maximization type algorithm, which is developed for the proposed family of models and is demonstrated to perform satisfactorily. Full article

Parameter identification is an important research topic with a variety of applications in industrial and environmental problems. Usually, a functional has to be minimized in conjunction with parameter identification; thus, there is a certain similarity between the parameter identification and optimization. A number of rigorous and efficient algorithms for optimization problems were developed in recent decades for the case of a convex functional. In the case of a non-convex functional, the metaheuristic algorithms dominate. This paper discusses an optimization method called modified bee colony algorithm (MBC), which is a modification of the standard bees algorithm (SBA). The SBA is inspired by a particular intelligent behavior of honeybee swarms. The algorithm is adapted for the parameter identification of reaction-dominated pore-scale transport when a non-convex functional has to be minimized. The algorithm is first checked by solving a few benchmark problems, namely finding the minima for Shekel, Rosenbrock, Himmelblau and Rastrigin functions. A statistical analysis was carried out to compare the performance of MBC with the SBA and the artificial bee colony (ABC) algorithm. Next, MBC is applied to identify the three parameters in the Langmuir isotherm, which is used to describe the considered reaction. Here, 2D periodic porous media were considered. The simulation results show that the MBC algorithm can be successfully used for identifying admissible sets for the reaction parameters in reaction-dominated transport characterized by low Pecklet and high Damkholer numbers. Finite element approximation in space and implicit time discretization are exploited to solve the direct problem. Full article

This paper proposes a toolbox for simulating the effective integration of renewable energy sources into distribution systems. The toolbox uses four heuristic methods: the particle swarm optimization (PSO) method, and three recently developed methods, namely Gray Wolf Optimization (GWO), Ant Lion Optimization (ALO), and Whale Optimization Algorithm (WOA), for the efficient operation of power distribution systems. The toolbox consists of two main functionalities. The first one allows the user to select the test system to be solved (33-, 69-, or 141-bus test systems), the locations of the distributed generators (DGs), and the voltage regulators. In addition, the user selects the daily active power output profiles of the DGs, and the tool solves the voltage deviation problem for the specified time of day. The second functionality involves the simulation of energy storage systems and provides the optimal daily power output of the resources. With this program, a graphical user interface (GUI) allows users to select the test system, the optimization method to be used, the number of DGs and locations, the locations and number of battery energy storage systems (BESSs), and the tap changer locations. With the simple user interface, the user can manage the distribution system simulation and see the results by making appropriate changes to the test systems. Full article

A scheme for parallel computation of the two-dimensional Edwards—Anderson model based on the transfer matrix approach is proposed. Free boundary conditions are considered. The method may find application in calculations related to spin glasses and in quantum simulators. Performance data are given. The scheme of parallelisation for various numbers of threads is tested. Application to a quantum computer simulator is considered in detail. In particular, a parallelisation scheme of work of quantum computer simulator. Full article

The ductility and strength of reinforced concrete (RC) columns could be noticeably improved by replacing steel bars with polymeric bars. Despite the previous research on RC columns, most of those studies focused only on the lateral load capacity of this structural member and were mainly costly experimental studies. However, this paper is concentrated on the previously occurred damages to the reinforced columns in the previous earthquakes. Subsequently, finite element analysis has been performed to examine 24 models including the various shapes of RC columns. In employing the plastic behavior of steel, carbon fiber-reinforced polymer (CFRP), and glass fiber reinforced polymer (GFRP) bars, the bilinear hardening has been considered. To capture both compressive and tensile behavior of the concrete, the concrete damage plasticity model has been implemented. Furthermore, the optimization technique is used for CFRP models to compare with other models. In this paper, the parameters of energy, seismic factor, stiffness, and ductility have been computed using the method proposed by the authors. This suggested method is considered to compare the results from each parameter. Finite element results of steel bars are compared with carbon and glass models. The results show the stiffness of models is improved by CFRP bars, while the energy absorption and ductility factor are enhanced with steel bars. Moreover, GFRP bars can enhance the seismic factor. The reduction of column stiffness to almost half would occur in some rectangular cross-section columns. Full article

Convolutional neural networks (CNNs) have powerful representation learning capabilities by automatically learning and extracting features directly from inputs. In classification applications, CNN models are typically composed of: convolutional layers, pooling layers, and fully connected (FC) layer(s). In a chain-based deep neural network, the FC layers contain most of the parameters of the network, which affects memory occupancy and computational complexity. For many real-world problems, speeding up inference time is an important matter because of the hardware design implications. To deal with this problem, we propose the replacement of the FC layers with a Hopfield neural network (HNN). The proposed architecture combines both a CNN and an HNN: A pretrained CNN model is used for feature extraction, followed by an HNN, which is considered as an associative memory that saves all features created by the CNN. Then, to deal with the limitation of the storage capacity of the HNN, the proposed work uses multiple HNNs. To optimize this step, the knapsack problem formulation is proposed, and a genetic algorithm (GA) is used solve it. According to the results obtained on the Noisy MNIST Dataset, our work outperformed the state-of-the-art algorithms. Full article

Given a monotone ordered multi-dimensional real array A and a real value k, an important question in computation is to establish if k is a member of A by sequentially searching A by comparing k with some of its entries. This search problem and its known results are surveyed, including the case when A has sizes not necessarily equal. Worst case search algorithms for various types of arrays of finite dimension and sizes are reported. Each algorithm has order strictly less than the product of the sizes of the array. Present challenges and open problems in the area are also presented. Full article

Hyper-heuristics comprise a set of approaches that are motivated (at least in part) by the objective of intelligently combining heuristic methods to solve hard optimization problems. Ant colony optimization (ACO) algorithms have been proven to deal with Dynamic Optimization Problems (DOPs) properly. Despite the good results obtained by the integration of local search operators with ACO, little has been done to tackle DOPs. In this research, one of the most reliable ACO schemes, the $\mathcal{MAX}$-$\mathcal{MIN}$ Ant System ($\mathcal{MM}$AS), has been integrated with advanced and effective local search operators, resulting in an innovative hyper-heuristic. The local search operators are the Lin–Kernighan (LK) and the Unstringing and Stringing (US) heuristics, and they were intelligently chosen to improve the solution obtained by ACO. The proposed method aims to combine the adaptation capabilities of ACO for DOPs and the good performance of the local search operators chosen in an adaptive way and based on their performance, creating in this way a hyper-heuristic. The travelling salesman problem (TSP) was the base problem to generate both symmetric and asymmetric dynamic test cases. Experiments have shown that the $\mathcal{MM}$AS provides good initial solutions to the local search operators and the hyper-heuristic creates a robust and effective method for the vast majority of test cases. Full article

The problem of finite-time control for singular linear semi-Markov jump systems (SMJSs) with unknown transition rates is considered in this paper. By designing a new semi-positive definite Lyapunov-like function, state feedback controller design methods are given that allow closed-loop singular linear SMJSs to be regular, impulse-free and stochastically finite-time-stable without external disturbance, and stochastically finite-time bounded with external disturbance. The obtained conditions are expressed by a set of strict matrix inequalities, which can be simplified to a set of linear matrix inequalities by a one dimensional search for a scalar. Two numerical examples are given to illustrate the effectiveness of proposed method. Full article

In the paper, the author deals with modeling the stochastic behavior of ordinary patch antennas in terms of the mean and standard deviation of their reflection coefficient |S₁₁| under the geometrical uncertainty associated with their manufacturing process. The Artificial Neural Network is used to model the stochastic reflection coefficient of the antennas. The Polynomial Chaos Expansion and FDTD computations are used to obtain the training and testing data for the Artificial Neural Network. For the first time, the author uses his analytical transformations to reduce the required number of highly time-consuming FDTD simulations for a given set of nominal values of the design parameters of the ordinary patch antenna. An analysis is performed for n257 and n258 frequency bands (24.5–28.7 GHz). The probability distributions of the design parameters are extracted from the measurement results obtained for a series of manufactured patch antenna arrays for three different frequencies in the C, X, and Ka bands. Patch antennas are chosen as the subject of the scientific analysis in this paper because of the popularity of the patch antennas in the scientific literature concerning antennas, as well as because of a simple form of these antennas that is reflected in the time required for computation of training and testing data for the Artificial Neural Network. Full article

Gradient-based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have been shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method, though less commonly used in training neural networks, is known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks, and to briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem. Full article

Parkinson’s disease (PD) is characterized by abnormal brain oscillations that can change rapidly. Tracking neural alternations with high temporal resolution electrophysiological monitoring methods such as EEG can lead to valuable information about alterations observed in PD. Concomitantly, there have been advances in the high-accuracy performance of deep neural networks (DNNs) using few-patient data. In this study, we propose a method to transform resting-state EEG data into a deep latent space to classify PD subjects from healthy cases. We first used a general orthogonalized directed coherence (gOPDC) method to compute directional connectivity (DC) between all pairwise EEG channels in four frequency bands (Theta, Alpha, Beta, and Gamma) and then converted the DC maps into 2D images. We then used the VGG-16 architecture (trained on the ImageNet dataset) as our pre-trained model, enlisted weights of convolutional layers as initial weights, and fine-tuned all layer weights with our data. After training, the classification achieved 99.62% accuracy, 100% precision, 99.17% recall, 0.9958 F1 score, and 0.9958 AUC averaged for 10 random repetitions of training/evaluating on the proposed deep transfer learning (DTL) network. Using the latent features learned by the network and employing LASSO regression, we found that latent features (as opposed to the raw DC values) were significantly correlated with five clinical indices routinely measured: left and right finger tapping, left and right tremor, and body bradykinesia. Our results demonstrate the power of transfer learning and latent space derivation for the development of oscillatory biomarkers in PD. Full article

We present an algorithm for simultaneous retrieval of aerosol and marine parameters in coastal waters. The algorithm is based on a radiative transfer forward model for a coupled atmosphere-ocean system, which is used to train a radial basis function neural network (RBF-NN) to obtain a fast and accurate method to compute radiances at the top of the atmosphere (TOA) for given aerosol and marine input parameters. The inverse modelling algorithm employs multidimensional unconstrained non-linear optimization to retrieve three marine parameters (concentrations of chlorophyll and mineral particles, as well as absorption by coloured dissolved organic matter (CDOM)), and two aerosol parameters (aerosol fine-mode fraction and aerosol volume fraction). We validated the retrieval algorithm using synthetic data and found it, for both low and high sun, to predict each of the five parameters accurately, both with and without white noise added to the top of the atmosphere (TOA) radiances. When varying the solar zenith angle (SZA) and retraining the RBF-NN without noise added to the TOA radiance, we found the algorithm to predict the CDOM absorption, chlorophyll concentration, mineral concentration, aerosol fine-mode fraction, and aerosol volume fraction with correlation coefficients greater than 0.72, 0.73, 0.93, 0.67, and 0.87, respectively, for 45${}^{\circ }\le$ SZA ≤ 75${}^{\circ }$. By adding white Gaussian noise to the TOA radiances with varying values of the signal-to-noise-ratio (SNR), we found the retrieval algorithm to predict CDOM absorption, chlorophyll concentration, mineral concentration, aerosol fine-mode fraction, and aerosol volume fraction well with correlation coefficients greater than 0.77, 0.75, 0.91, 0.81, and 0.86, respectively, for high sun and SNR ≥ 95. Full article

This paper presents an algorithm for the fully automatic mesh generation for the finite element analysis of ships and offshore structures. The quality requirements on the mesh generator are imposed by the acceptance criteria of the classification societies as well as the need to avoid shear locking when using low degree shell elements. The meshing algorithm will be generating quadrilateral dominated meshes (consisting of quads and triangles) and the mesh quality requirements mandate that quadrilaterals with internal angles close to 90° are to be preferred. The geometry is described by a dictionary containing points, rods, surfaces, and openings. The first part of the proposed method consists of an algorithm to automatically clean the geometry. The corrected geometry is then meshed by the frontal Delaunay mesh generator as implemented in the gmsh package. We present a heuristic method to precondition the cross field of the fronatal quadrilateral mesher. In addition, the influence of the order in which the plates are meshed will be explored as a preconditioning step. Full article

Kriging-based modeling has been widely used in computationally intensive simulations. However, the Kriging modeling of high-dimensional problems not only takes more time, but also leads to the failure of model construction. To this end, a Kriging modeling method based on multidimensional scaling (KMDS) is presented to avoid the “dimensional disaster”. Under the condition of keeping the distance between the sample points before and after the dimensionality reduction unchanged, the KMDS method, which mainly calculates each element in the inner product matrix due to the mapping relationship between the distance matrix and the inner product matrix, completes the conversion of design data from high dimensional to low dimensional. For three benchmark functions with different dimensions and the aviation field problem of aircraft longitudinal flight control, the proposed method is compared with other dimensionality reduction methods. The KMDS method has better modeling efficiency while meeting certain accuracy requirements. Full article

The autonomous and adaptable identification of anomalies in industrial contexts, particularly in the physical processes of Cyber-Physical Production Systems (CPPS), requires using critical technologies to identify failures correctly. Most of the existing solutions in the anomaly detection research area do not consider such systems’ dynamics. Due to the complexity and multidimensionality of CPPS, a scalable, adaptable, and rapid anomaly detection system is needed, considering the new design specifications of Industry 4.0 solutions. Immune-based models, such as the Dendritic Cell Algorithm (DCA), may provide a rich source of inspiration for detecting anomalies, since the anomaly detection problem in CPPS greatly resembles the functionality of the biological dendritic cells in defending the human body from hazardous pathogens. This paper tackles DCA limitations that may compromise its usage in anomaly detection applications, such as the manual characterization of safe and danger signals, data analysis not suitable for online classification, and the lack of an object-oriented implementation of the algorithm. The proposed approach, the Cursory Dendritic Cell Algorithm (CDCA), is a novel variation of the DCA, developed to be flexible and monitor physical industrial processes continually while detecting anomalies in an online fashion. This work’s contribution is threefold. First, it provides a comprehensive review of Artificial Immune Systems (AIS), focusing on AIS applied to the anomaly detection problem. Then, a new object-oriented architecture for the DCA implementation is described, enabling the modularity and abstraction of the algorithm stages into different classes (modules). Finally, the CDCA for the anomaly detection problem is proposed. The CDCA was successfully validated in two industrial-oriented dataset benchmarks for physical anomaly and network intrusion detection, the Skoltech Anomaly Benchmark (SKAB) and M2M using OPC UA. When compared to other algorithms, the proposed approach exhibits promising classification results. It was placed fourth on the SKAB scoreboard and presented a competitive performance with the incremental Dendritic Cell Algorithm (iDCA). Full article