Search Results (42)

Accurately estimating the effort and duration required for software development is one of the most important challenges in the field of software engineering. In a context where software projects are becoming increasingly complex, project managers face real difficulties in meeting established deadlines and staying within budget constraints. The purpose of this research study is to identify which type of artificial neural network is most suitable for estimating the effort and duration of software development, given the relatively small size of existing datasets. In the process of software effort and duration prediction, four datasets were used: China, Desharnais, Kemerer and Maxwell. Additionally, different types of artificial neural networks were used: Multilayer Perceptron, Fractal Neural Network, Deep Fully Connected Neural Network, Extreme Learning Machine, and Hybrid Neural Network. Another goal of this research is to analyze the impact of a new and innovative hybrid architecture, which combines Fractal Neural Network with Random Forests in the estimation process. Five metrics were used to compare the accuracy of artificial neural networks: mean absolute error, median absolute error, root mean square error, coefficient of determination, and mean squared logarithmic error. Python 3.11 programming language was used in combination with TensorFlow, Keras, and Scikit-learn libraries to implement artificial neural networks. Full article

We classified common dog activities, such as sitting, standing, and lying down, which are crucial for monitoring the well-being of pets. To create a new model, we used convolutional neural networks (CNNs) on a Raspberry Pi platform and the InceptionV3 model, optimized on a dataset of Siberian Husky photos. The accuracy was 88% on a test set of 50 samples. In the developed model, TensorFlow Keras was used, while the OpenCV library was also used for system interaction with the Raspberry Pi and its Camera module. The model was effective for the image classification of dog behaviors in various environmental circumstances. The model substantially contributes to the development of pet welfare monitoring systems and improves the care for beloved animal companions. Full article

Identifying blueberry characteristics such as the wax bloom is an important task that not only helps in phenotyping (for novel variety development) but also in classifying berries better suited for commercialization. Deep learning techniques for image analysis have long demonstrated their capability for solving image classification problems. However, they usually rely on large architectures that could be difficult to implement in the field due to high computational needs. This paper presents a small (only 1502 parameters) Bayesian–CNN ensemble architecture that can be implemented in any small electronic device and is able to classify wax bloom content in images. The Bayesian model was implemented using Keras image libraries and consists of only two convolutional layers (eight and four filters, respectively) and a dense layer. It includes a statistical module with two metrics that combines the results of the Bayesian ensemble to detect potential misclassifications. The first metric is based on the Euclidean distance (${\mathcal{L}}_2$) between Gaussian mixture models while the second metric is based on a quantile analysis of the binary class predictions. Both metrics attempt to establish whether the model was able to find a good prediction or not. Three experiments were performed: first, the Bayesian–CNN ensemble model was compared with state-of-the-art small architectures. In experiment 2, the metrics for detecting potential misclassifications were evaluated and compared with similar techniques derived from the literature. Experiment 3 reports results while using cross validation and compares performance considering the trade-off between accuracy and the number of samples considered as potentially misclassified (not classified). Both metrics show a competitive performance compared to the state of the art and are able to improve the accuracy of a Bayesian–CNN ensemble model from $96.98\%$ to $98.72\pm 0.54\%$ and $98.38\pm 0.34\%$ for the ${\mathcal{L}}_2$ and $r^2$ metrics, respectively. Full article

The continuous assessment of cover crop growth throughout the season is a crucial baseline observation for making informed crop management decisions and sustainable farming operation. Precision agriculture techniques involving applications of sensors and unmanned aerial vehicles provide precise and prompt spectral and structural data, which allows for effective evaluation of cover crop biomass. Vegetation indices are widely used to quantify crop growth and biomass metrics. The objective of this study was to evaluate the accuracy of biomass estimation using a machine learning approach leveraging spectral and canopy height data acquired from unmanned aerial vehicles (UAVs), comparing different neural network architectures, optimizers, and activation functions. Field trials were carried out at two sites in Louisiana involving winter cover crops. The canopy height was estimated by subtracting the digital surface model taken at the time of peak growth of the cover crop from the data captured during a bare ground condition. When evaluated against the validation dataset, the neural network model facilitated with a Keras TensorFlow library with Adam optimizers and a sigmoid activation function performed the best, predicting cover crop biomass with an average of 96 g m⁻² root mean squared error (RMSE). Other statistical metrics including the Pearson correlation and R² also showed satisfactory conditions with this combination of hyperparameters. The observed cover crop biomass ranged from 290 to 1217 g m⁻². The present study findings highlight the merit of comprehensive analysis of cover crop traits using UAV remote sensing and machine learning involving realistic underpinning biophysical mechanisms, as our approach captured both horizontal (vegetation indices) and vertical (canopy height) aspects of plant growth. Full article

Background/Objectives: This study aimed to develop a machine learning (ML) algorithm that can predict unplanned reoperations and surgical/medical complications after vestibular schwannoma (VS) surgery. Methods: All pre- and peri-operative variables available in the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database (n = 110), except those directly related to our outcome variables, were used as input variables. A deep neural network model consisting of seven layers was developed using the Keras open-source library, with a 70:30 breakdown for training and testing. The feature importance of input variables was measured to elucidate their relative permutation effect in the ML model. Results: Of the 1783 patients with VS undergoing surgery, unplanned reoperation, surgical complications, and medical complications were seen in 8.5%, 5.2%, and 6.2% of patients, respectively. The deep neural network model had area under the curve of receiver operating characteristics (ROC-AUC) of 0.6315 (reoperation), 0.7939 (medical complications), and 0.719 (surgical complications). Accuracy, specificity, and negative predictive values of the model for all outcome variables ranged from 82.1 to 96.6%, while positive predictive values and sensitivity ranged from 16.7 to 51.5%. Variables such as the length of stay post-operation until discharge, days from operation to discharge, and the total hospital length of stay had the highest permutation importance. Conclusions: We developed an effective ML algorithm predicting unplanned reoperation and surgical/medical complications post-VS surgery. This may offer physicians guidance into potential post-surgical outcomes to allow for personalized medical care plans for VS patients. Full article

Background/Objectives: High-grade serous carcinomas (HGSCs) are highly heterogeneous tumors, both among patients and within a single tumor. Differences in molecular mechanisms significantly describe this heterogeneity. Four molecular subtypes have been previously described by the Cancer Genome Atlas Consortium: differentiated, immunoreactive, mesenchymal, and proliferative. These subtypes may have varying degrees of progression, relapse-free survival, and overall survival, as well as response to therapy. The precise determination of these subtypes is certainly necessary both for diagnosis and future development of targeted therapies within personalized medicine. Methods: In this study, we analyzed gene expression data based on bulk RNA-seq, scRNA-seq, and spatial transcriptomic data from six cohorts (totaling 535 samples, including 60 single-cell samples). Differential expression analysis was performed using the edgeR package. The KEGG database and GSVA package were used for pathways enrichment analysis. As a predictive model, a deep neural network was created using the keras and tensorflow libraries. Results: We identified 357 differentially expressed genes among the four subtypes: 96 differentiated, 33 immunoreactive, 91 mesenchymal, and 137 proliferative. Based on these, we created OVsignGenes, a neural network model resistant to the effects of platform (test dataset AUC = 0.969). We then ran data from five more cohorts through our model, including scRNA-seq and spatial transcriptomics. Conclusions: Because the differentiated subtype is located at the intersection of the other three subtypes based on PCA and does not have a unique profile of differentially expressed genes or enriched pathways, it can be considered an initiating subtype of tumor that will develop into one of the three other subtypes. Full article

Arabic raw audio datasets were initially gathered to produce a corresponding signal spectrum, which was further used to extract the Mel-Frequency Cepstral Coefficients (MFCCs). The pronunciation dictionary, language model, and acoustic model were further derived from the MFCCs’ features. These output data were processed into Baidu’s Deep Speech model (ASR system) to attain the text corpus. Baidu’s Deep Speech model was implemented to precisely identify the global optimal value rapidly while preserving a low word and character discrepancy rate by attaining an excellent performance in isolated and end-to-end speech recognition. The desired outcome in this work is to forecast the next word and character in a sequential and systematic order that applies under natural language processing (NLP). This work combines the trained Arabic language model ARABERT with the potential of Long Short-Term Memory (LSTM) networks to predict the next word and character in an Arabic text. We used the pre-trained ARABERT embedding to improve the model’s capacity and, to capture semantic relationships within the language, we educated LSTM + CNN and Markov models on Arabic text data to assess the efficacy of this model. Python libraries such as TensorFlow, Pickle, Keras, and NumPy were used to effectively design our development model. We extensively assessed the model’s performance using new Arabic text, focusing on evaluation metrics like accuracy, word error rate, character error rate, BLEU score, and perplexity. The results show how well the combined LSTM + ARABERT and Markov models have outperformed the baseline models in envisaging the next word or character in the Arabic text. The accuracy rates of 64.9% for LSTM, 74.6% for ARABERT + LSTM, and 78% for Markov chain models were achieved in predicting the next word, and the accuracy rates of 72% for LSTM, 72.22% for LSTM + CNN, and 73% for ARABERET + LSTM models were achieved for the next-character prediction. This work unveils a novelty in Arabic natural language processing tasks, estimating a potential future expansion in deriving a precise next-word and next-character forecasting, which can be an efficient utility for text generation and machine translation applications. Full article

Theano, TensorFlow, Keras, Torch, PyTorch, and other software frameworks have remarkably stimulated the popularity of deep learning (DL). Apart from all the good they achieve, the danger of such frameworks is that they unintentionally spur a black-box attitude. Some practitioners play around with building blocks offered by frameworks and rely on them, having a superficial understanding of the internal mechanics. This paper constitutes a concise tutorial that elucidates the flows of signals and gradients in deep neural networks, enabling readers to successfully implement a deep network from scratch. By “from scratch”, we mean with access to a programming language and numerical libraries but without any components that hide DL computations underneath. To achieve this goal, the following five topics need to be well understood: (1) automatic differentiation, (2) the initialization of weights, (3) learning algorithms, (4) regularization, and (5) the organization of computations. We cover all of these topics in the paper. From a tutorial perspective, the key contributions include the following: (a) proposition of R and S operators for tensors—rashape and stack, respectively—that facilitate algebraic notation of computations involved in convolutional, pooling, and flattening layers; (b) a Python project named hmdl (“home-made deep learning”); and (c) consistent notation across all mathematical contexts involved. The hmdl project serves as a practical example of implementation and a reference. It was built using NumPy and Numba modules with JIT and CUDA amenities applied. In the experimental section, we compare hmdl implementation to Keras (backed with TensorFlow). Finally, we point out the consistency of the two in terms of convergence and accuracy, and we observe the superiority of the latter in terms of efficiency. Full article

Borehole hydraulic fracturing in coal seams can prevent dynamic coal mine disasters and promote the sustainability of the mining industry, and microseismic signal recognition is a prerequisite and foundation for microseismic monitoring technology that evaluates the effectiveness of hydraulic fracturing. This study constructed ultra-lightweight CNN models specifically designed to identify microseismic waveforms induced by borehole hydraulic fracturing in coal seams, namely Ul-Inception28, Ul-ResNet12, Ul-MobileNet17, and Ul-TripleConv8. The three best-performing models were selected to create both a probability averaging ensemble CNN model and a voting ensemble CNN model. Additionally, an automatic threshold adjustment strategy for CNN identification was introduced. The relationships between feature map entropy, training data volume, and model performance were also analyzed. The results indicated that our in-house models surpassed the performance of the InceptionV3, ResNet50, and MobileNetV3 models from the TensorFlow Keras library. Notably, the voting ensemble CNN model achieved an improvement of at least 0.0452 in the F1 score compared to individual models. The automatic threshold adjustment strategy enhanced the identification threshold’s precision to 26 decimal places. However, a continuous zero-entropy value in the feature maps of various channels was found to detract from the model’s generalization performance. Moreover, the expanded training dataset, derived from thousands of waveforms, proved more compatible with CNN models comprising hundreds of thousands of parameters. The findings of this research significantly contribute to the prevention of dynamic coal mine disasters, potentially reducing casualties, economic losses, and promoting the sustainable progress of the coal mining industry. Full article

Artificial neural networks (ANNs) can be used for accurate heat load forecasting in district heating systems (DHSs). This paper presents an application of a shallow ANN with two hidden layers in the case of a local DHS. The developed model was used to write a simple application in Python 3.10 that can be used in the operation of a district heating plant to carry out a preliminary analysis of heat demand, taking into account the ambient temperature on a given day. The model was trained using the real data from the period 2019–2022. The training was sufficient for the number of 150 epochs. The prediction effectiveness indicator was proposed. In the considered case, the effectiveness of the trained network was 85% and was better in comparison to five different regression models. The developed tool was based on an open-source programming environment and proved its ability to predict heating load. Full article

Investigating the optimal laser processing parameters for industrial purposes can be time-consuming. Moreover, an exact analytic model for this purpose has not yet been developed due to the complex mechanisms of laser processing. The main goal of this study was the development of a backpropagation neural network (BPNN) with a grey wolf optimization (GWO) algorithm for the quick and accurate prediction of multi-input laser etching parameters (energy, scanning velocity, and number of exposures) and multioutput surface characteristics (depth and width), as well as to assist engineers by reducing the time and energy require for the optimization process. The Keras application programming interface (API) Python library was used to develop a GWO-BPNN model for predictions of laser etching parameters. The experimental data were obtained by adopting a 30 W laser source. The GWO-BPNN model was trained and validated on experimental data including the laser processing parameters and the etching characterization results. The R² score, mean absolute error (MAE), and mean squared error (MSE) were examined to evaluate the prediction precision of the model. The results showed that the GWO-BPNN model exhibited excellent accuracy in predicting all properties, with an R² value higher than 0.90. Full article

Background: Recent advancements in deep learning have significantly impacted ophthalmology, especially in glaucoma, a leading cause of irreversible blindness worldwide. In this study, we developed a reliable predictive model for glaucoma detection using deep learning models based on clinical data, social and behavior risk factor, and demographic data from 1652 participants, split evenly between 826 control subjects and 826 glaucoma patients. Methods: We extracted structural data from control and glaucoma patients’ electronic health records (EHR). Three distinct machine learning classifiers, the Random Forest and Gradient Boosting algorithms, as well as the Sequential model from the Keras library of TensorFlow, were employed to conduct predictive analyses across our dataset. Key performance metrics such as accuracy, F1 score, precision, recall, and the area under the receiver operating characteristics curve (AUC) were computed to both train and optimize these models. Results: The Random Forest model achieved an accuracy of 67.5%, with a ROC AUC of 0.67, outperforming the Gradient Boosting and Sequential models, which registered accuracies of 66.3% and 64.5%, respectively. Our results highlighted key predictive factors such as intraocular pressure, family history, and body mass index, substantiating their roles in glaucoma risk assessment. Conclusions: This study demonstrates the potential of utilizing readily available clinical, lifestyle, and demographic data from EHRs for glaucoma detection through deep learning models. While our model, using EHR data alone, has a lower accuracy compared to those incorporating imaging data, it still offers a promising avenue for early glaucoma risk assessment in primary care settings. The observed disparities in model performance and feature significance show the importance of tailoring detection strategies to individual patient characteristics, potentially leading to more effective and personalized glaucoma screening and intervention. Full article

The most significant threat that networks established in IoT may encounter is cyber attacks. The most commonly encountered attacks among these threats are DDoS attacks. After attacks, the communication traffic of the network can be disrupted, and the energy of sensor nodes can quickly deplete. Therefore, the detection of occurring attacks is of great importance. Considering numerous sensor nodes in the established network, analyzing the network traffic data through traditional methods can become impossible. Analyzing this network traffic in a big data environment is necessary. This study aims to analyze the obtained network traffic dataset in a big data environment and detect attacks in the network using a deep learning algorithm. This study is conducted using PySpark with Apache Spark in the Google Colaboratory (Colab) environment. Keras and Scikit-Learn libraries are utilized in the study. ‘CICIoT2023’ and ‘TON_IoT’ datasets are used for training and testing the model. The features in the datasets are reduced using the correlation method, ensuring the inclusion of significant features in the tests. A hybrid deep learning algorithm is designed using one-dimensional CNN and LSTM. The developed method was compared with ten machine learning and deep learning algorithms. The model’s performance was evaluated using accuracy, precision, recall, and F1 parameters. Following the study, an accuracy rate of 99.995% for binary classification and 99.96% for multiclassification is achieved in the ‘CICIoT2023’ dataset. In the ‘TON_IoT’ dataset, a binary classification success rate of 98.75% is reached. Full article

Today, specific convolution neural network (CNN) models assigned to specific tasks are often used. In this article, the authors explored three models: MobileNet, EfficientNetB0, and InceptionV3 combined. The authors were interested in investigating how quickly an artificial intelligence model can be taught with limited computer resources. Three types of training bases were investigated, starting with a simple base verifying five colours, then recognizing two different orthogonal elements, followed by more complex images from different families. This research aimed to demonstrate the capabilities of the models based on training base parameters such as the number of images and epoch types. Architectures proposed by the authors in these cases were chosen based on simulation studies conducted on a virtual machine with limited hardware parameters. The proposals present the advantages and disadvantages of the different models based on the TensorFlow and Keras libraries in the Jupiter environment based on the Python programming language. An artificial intelligence model with a combination of MobileNet, proposed by Siemens, and Efficient and Inception, selected by the authors, allows for further work to be conducted on image classification, but with limited computer resources for industrial implementation on a programmable logical controller (PLC). The study showed a 90% success rate, with a learning time of 180 s. Full article

Building deep learning models proposed by third parties can become a simple task when specialized libraries are used. However, much mystery still surrounds the design of new models or the modification of existing ones. These tasks require in-depth knowledge of the different components or building blocks and their dimensions. This information is limited and broken up in different literature. In this article, we collect and explain the building blocks used to design deep learning models in depth, starting from the artificial neuron to the concepts involved in building deep neural networks. Furthermore, the implementation of each building block is exemplified using the Keras library. Full article