Search Results (27)

Seismic waveform feature extraction is a critical task in seismic exploration, as it directly impacts reservoir prediction and geological interpretation. However, large-scale seismic data and nonlinear relationships between seismic signals and reservoir properties are challenging for traditional machine learning methods. To address these limitations, this paper proposes a novel framework combining Convolutional Neural Network (CNN) and Uniform Manifold Approximation and Projection (UMAP) for seismic waveform feature extraction and analysis. The UMAP-CNN framework leverages the strengths of manifold learning and deep learning, enabling multi-scale feature extraction and dimensionality reduction while preserving both local and global data structures. The evaluation experiments, which considered runtime, receiver operating characteristic (ROC) curves, embedding distribution maps, and other quantitative assessments, illustrated that the UMAP-CNN outperformed t-distributed stochastic neighbor embedding (t-SNE), locally linear embedding (LLE) and isometric feature mapping (Isomap). A case study in the Ordos Basin further demonstrated that UMAP-CNN offers a high degree of accuracy in predicting coal seam thickness. Furthermore, our framework exhibited superior computational efficiency and robustness in handling large-scale datasets. Full article

Malicious software presents significant challenges in cybersecurity, leveraging rapidly evolving technologies to bypass traditional defense mechanisms. This research introduces a novel image-based malware classification framework that uses hybrid-model Convolutional Neural Networks to process RGB images generated from assembly code. We present MalevisAsm, an enriched dataset that merges MaleVis malware samples with benign files, and propose a hybrid deep learning model that combines EfficientNetB0 and DenseNet121 for robust feature extraction. The approach transforms Portable Executable files into assembly code, maps opcode transitions into three-channel images, and uses a fine-tuned CNN to classify malware families. Additionally, we implemented Uniform Manifold Approximation and Projection a contemporary nonlinear dimensionality reduction technique, to enhance the identification of previously unseen malware samples via binary classification. Our experiments achieve a top-tier accuracy of 98.45%, surpassing existing benchmarks on the MaleVis dataset. This research contributes to the field by integrating static binary analysis with advanced computer vision techniques, offering a scalable and effective solution for malware detection. Full article

Large Language models have shown a remarkable ability to “converse” with humans in a natural language across myriad topics. Despite the proliferation of these models, a deep understanding of how they work under the hood remains elusive. The core of these Generative AI models is composed of layers of neural networks that employ the Transformer architecture. This architecture learns from large amounts of training data and creates new content in response to user input. In this study, we analyze the internals of the Transformer using Information Theory. To quantify the amount of information passing through a layer, we view it as an information transmission channel and compute the capacity of the channel. The highlight of our study is that, using Information-Theoretical tools, we develop techniques to visualize on an Information plane how the Transformer encodes the relationship between words in sentences while these words are projected into a high-dimensional vector space. We use Information Geometry to analyze the high-dimensional vectors in the Transformer layer and infer relationships between words based on the length of the geodesic connecting these vector distributions on a Riemannian manifold. Our tools reveal more information about these relationships than attention scores. In this study, we also show how Information-Theoretic analysis can help in troubleshooting learning problems in the Transformer layers. Full article

The development of neural networks has made the introduction of multimodal systems inevitable. Computer vision methods are still not widely used in biological research, despite their importance. It is time to recognize the significance of advances in feature extraction and real-time analysis of information from cells. Teacherless learning for the image clustering task is of great interest. In particular, the clustering of single cells is of great interest. This study will evaluate the feasibility of using latent representation and clustering of single cells in various applications in the fields of medicine and biotechnology. Of particular interest are embeddings, which relate to the morphological characterization of cells. Studies of C2C12 cells will reveal more about aspects of muscle differentiation by using neural networks. This work focuses on analyzing the applicability of the latent space to extract morphological features. Like many researchers in this field, we note that obtaining high-quality latent representations for phase-contrast or bright-field images opens new frontiers for creating large visual-language models. Graph structures are the main approaches to non-Euclidean manifolds. Graph-based segmentation has a long history, e.g., the normalized cuts algorithm treated segmentation as a graph partitioning problem—but only recently have such ideas merged with deep learning in an unsupervised manner. Recently, a number of works have shown the advantages of hyperbolic embeddings in vision tasks, including clustering and classification based on the Poincaré ball model. One area worth highlighting is unsupervised segmentation, which we believe is undervalued, particularly in the context of non-Euclidean spaces. In this approach, we aim to mark the beginning of our future work on integrating visual information and biological aspects of individual cells to multimodal space in comparative studies in vitro. Full article

Microcrack detection using deep neural networks (DNNs) through an automated pipeline using wave fields interacting with the damaged areas is highly sought after. However, these high-dimensional spatio–temporal crack data are limited. Moreover, these datasets have large dimensions in the temporal domain. The dataset presents a substantial class imbalance, with crack pixels constituting an average of only 5% of the total pixels per sample. This extreme class imbalance poses a challenge for deep learning models with different microscale cracks, as the network can be biased toward predicting the majority class, generally leading to poor detection accuracy for the under-represented class. This study proposes an asymmetric encoder–decoder network with an adaptive feature reuse block for microcrack detection. The impact of various activation and loss functions are examined through feature space visualisation using the manifold discovery and analysis (MDA) algorithm. The optimized architecture and training methodology achieved an accuracy of 87.74%. Full article

In this paper, we focus on h-manifolds related to impulsive reaction–diffusion Cohen–Grossberg neural networks with time-varying delays. By constructing a new Lyapunov-type function and a comparison principle, sufficient conditions that guarantee the global practical exponential stability of specific states are established. The states of interest are determined by the so-called h-manifolds, i.e., manifolds defined by a specific function h, which is essential for various applied problems in imposing constraints on their dynamics. The established criteria are less restrictive for the variable domain and diffusion coefficients. The effect of some uncertain parameters on the stability behavior is also considered and a robust practical stability analysis is proposed. In addition, the obtained h-manifolds’ practical stability results are applied to a bidirectional associative memory (BAM) neural network model with impulsive perturbations and time-varying delays. Appropriate examples are discussed. Full article

Background: Alzheimer’s disease is a progressive neurological condition marked by a decline in cognitive abilities. Early diagnosis is crucial but challenging due to overlapping symptoms among impairment stages, necessitating non-invasive, reliable diagnostic tools. Methods: We applied information geometry and manifold learning to analyze grayscale MRI scans classified into No Impairment, Very Mild, Mild, and Moderate Impairment. Preprocessed images were reduced via Principal Component Analysis (retaining 95% variance) and converted into statistical manifolds using estimated mean vectors and covariance matrices. Geodesic distances, computed with the Fisher Information metric, quantified class differences. Graph Neural Networks, including Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and GraphSAGE, were utilized to categorize impairment levels using graph-based representations of the MRI data. Results: Significant differences in covariance structures were observed, with increased variability and stronger feature correlations at higher impairment levels. Geodesic distances between No Impairment and Mild Impairment (58.68, $p<0.001$) and between Mild and Moderate Impairment (58.28, $p<0.001$) are statistically significant. GCN and GraphSAGE achieve perfect classification accuracy (precision, recall, F1-Score: 1.0), correctly identifying all instances across classes. GAT attains an overall accuracy of 59.61%, with variable performance across classes. Conclusions: Integrating information geometry, manifold learning, and GNNs effectively differentiates AD impairment stages from MRI data. The strong performance of GCN and GraphSAGE indicates their potential to assist clinicians in the early identification and tracking of Alzheimer’s disease progression. Full article

The Nonnegative Matrix Factorization (NMF) algorithm and its variants have gained widespread popularity across various domains, including neural networks, text clustering, image processing, and signal analysis. In the context of hyperspectral unmixing (HU), an important task involving the accurate extraction of endmembers from mixed spectra, researchers have been actively exploring different regularization techniques within the traditional NMF framework. These techniques aim to improve the precision and reliability of the endmember extraction process in HU. In this study, we propose a novel HU algorithm called KMBNMF, which introduces an average kurtosis regularization term based on endmember spectra to enhance endmember extraction, additionally, it integrates a manifold regularization term into the average kurtosis-constrained NMF by constructing a symmetric weight matrix. This combination of these two regularization techniques not only optimizes the extraction process of independent endmembers but also improves the part-based representation capability of hyperspectral data. Experimental results obtained from simulated and real-world hyperspectral datasets demonstrate the competitive performance of the proposed KMBNMF algorithm when compared to state-of-the-art algorithms. Full article

This paper presents an innovative approach leveraging Neuronal Manifold Analysis of EEG data to identify specific time intervals for feature extraction, effectively capturing both class-specific and subject-specific characteristics. Different pipelines were constructed and employed to extract distinctive features within these intervals, specifically for motor imagery (MI) tasks. The methodology was validated using the Graz Competition IV datasets 2A (four-class) and 2B (two-class) motor imagery classification, demonstrating an improvement in classification accuracy that surpasses state-of-the-art algorithms designed for MI tasks. A multi-dimensional feature space, constructed using NMA, was built to detect intervals that capture these critical characteristics, which led to significantly enhanced classification accuracy, especially for individuals with initially poor classification performance. These findings highlight the robustness of this method and its potential to improve classification performance in EEG-based MI-BCI systems. Full article

The gadolinium vanadate doped with samarium (GdVO₄:Sm³⁺) nanopowder was prepared by the solution combustion synthesis (SCS) method. After synthesis, in order to achieve full crystallinity, the material was annealed in air atmosphere at 900 °C. Phase identification in the post-annealed powder samples was performed by X-ray diffraction, and morphology was investigated by high-resolution scanning electron microscope (SEM) and transmission electron microscope (TEM). Photoluminescence characterization of emission spectrum and time resolved analysis was performed using tunable laser optical parametric oscillator excitation and streak camera. In addition to samarium emission bands, a weak broad luminescence emission band of host VO₄³⁻ was also observed by the detection system. In our earlier work, we analyzed the possibility of using the host luminescence for two-color temperature sensing, improving the method by introducing the temporal dependence in line intensity ratio measurements. Here, we showed that further improvements are possible by using the machine learning approach. To facilitate the initial data assessment, we incorporated Principal Component Analysis (PCA), t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) clustering of GdVO4:Sm³⁺ spectra at various temperatures. Good predictions of temperature were obtained using deep neural networks. Performance of the deep learning network was enhanced by data augmentation technique. Full article

Self-similarities at different time scales embedded within a self-organizing neural manifold are well recognized. In this study, we hypothesize that the Hurst fractal dimension (HFD) of the scalp electroencephalographic (EEG) signal reveals statistical differences between chronic pain and opioid use. We test this hypothesis by using EEG resting state signals acquired from a total of 23 human subjects: 14 with chronic pain, 9 with chronic pain taking opioid medications, 5 with chronic pain and not taking opioid medications, and 9 healthy controls. Using the multifractal analysis algorithm, the HFD for full spectrum EEG and EEG frequency band time series was computed for all groups. Our results indicate the HFD varies spatially and temporally across all groups and is of lower magnitude in patients not taking opioids as compared to those taking opioids and healthy controls. A global decrease in HFD was observed with changes in gamma and beta power in the chronic pain group compared to controls and when paired to subject handedness and sex. Our results show the loss of complexity representative of brain wide dysfunction and reduced neural processing can be used as an EEG biomarker for chronic pain and subsequent opioid use. Full article

As the field of deep learning experiences a meteoric rise, the urgency to decipher the complex geometric properties of feature spaces, which underlie the effectiveness of diverse learning algorithms and optimization techniques, has become paramount. In this scholarly review, a comprehensive, holistic outlook on the geometry of feature spaces in deep learning models is provided in order to thoroughly probe the interconnections between feature spaces and a multitude of influential factors such as activation functions, normalization methods, and model architectures. The exploration commences with an all-encompassing examination of deep learning models, followed by a rigorous dissection of feature space geometry, delving into manifold structures, curvature, wide neural networks and Gaussian processes, critical points and loss landscapes, singular value spectra, and adversarial robustness, among other notable topics. Moreover, transfer learning and disentangled representations in feature space are illuminated, accentuating the progress and challenges in these areas. In conclusion, the challenges and future research directions in the domain of feature space geometry are outlined, emphasizing the significance of comprehending overparameterized models, unsupervised and semi-supervised learning, interpretable feature space geometry, topological analysis, and multimodal and multi-task learning. Embracing a holistic perspective, this review aspires to serve as an exhaustive guide for researchers and practitioners alike, clarifying the intricacies of the geometry of feature spaces in deep learning models and mapping the trajectory for future advancements in this enigmatic and enthralling domain. Full article

The causes of ventricular fibrillation (VF) are not yet elucidated, and it has been proposed that different mechanisms might exist. Moreover, conventional analysis methods do not seem to provide time or frequency domain features that allow for recognition of different VF patterns in electrode-recorded biopotentials. The present work aims to determine whether low-dimensional latent spaces could exhibit discriminative features for different mechanisms or conditions during VF episodes. For this purpose, manifold learning using autoencoder neural networks was analyzed based on surface ECG recordings. The recordings covered the onset of the VF episode as well as the next 6 min, and comprised an experimental database based on an animal model with five situations, including control, drug intervention (amiodarone, diltiazem, and flecainide), and autonomic nervous system blockade. The results show that latent spaces from unsupervised and supervised learning schemes yielded moderate though quite noticeable separability among the different types of VF according to their type or intervention. In particular, unsupervised schemes reached a multi-class classification accuracy of 66%, while supervised schemes improved the separability of the generated latent spaces, providing a classification accuracy of up to 74%. Thus, we conclude that manifold learning schemes can provide a valuable tool for studying different types of VF while working in low-dimensional latent spaces, as the machine-learning generated features exhibit separability among different VF types. This study confirms that latent variables are better VF descriptors than conventional time or domain features, making this technique useful in current VF research on elucidation of the underlying VF mechanisms. Full article

Saliency detection as an active research direction in image understanding and analysis has been studied extensively. In this paper, to improve the accuracy of saliency detection, we propose an efficient unsupervised salient object detection method. The first step of our method is that we extract local low-level features of each superpixel after segmenting the image into different scale parts, which helps to locate the approximate locations of salient objects. Then, we use convolutional neural networks to extract high-level, semantically rich features as complementary features of each superpixel, and low-level features, as well as high-level features of each superpixel, are incorporated into a new feature vector to measure the distance between different superpixels. The last step is that we use a manifold space-ranking method to calculate the saliency of each superpixel. Extensive experiments over four challenging datasets indicate that the proposed method surpasses state-of-the-art methods and is closer to the ground truth. Full article

Fractional calculus has gained a lot of attention in the last couple of years. Researchers have discovered that processes in various fields follow fractional dynamics rather than ordinary integer-ordered dynamics, meaning that the corresponding differential equations feature non-integer valued derivatives. There are several arguments for why this is the case, one of which is that fractional derivatives inherit spatiotemporal memory and/or the ability to express complex naturally occurring phenomena. Another popular topic nowadays is machine learning, i.e., learning behavior and patterns from historical data. In our ever-changing world with ever-increasing amounts of data, machine learning is a powerful tool for data analysis, problem-solving, modeling, and prediction. It has provided many further insights and discoveries in various scientific disciplines. As these two modern-day topics hold a lot of potential for combined approaches in terms of describing complex dynamics, this article review combines approaches from fractional derivatives and machine learning from the past, puts them into context, and thus provides a list of possible combined approaches and the corresponding techniques. Note, however, that this article does not deal with neural networks, as there is already extensive literature on neural networks and fractional calculus. We sorted past combined approaches from the literature into three categories, i.e., preprocessing, machine learning and fractional dynamics, and optimization. The contributions of fractional derivatives to machine learning are manifold as they provide powerful preprocessing and feature augmentation techniques, can improve physically informed machine learning, and are capable of improving hyperparameter optimization. Thus, this article serves to motivate researchers dealing with data-based problems, to be specific machine learning practitioners, to adopt new tools, and enhance their existing approaches. Full article