Search Results (95)

This paper addresses the problem of decision-making support for the modernization of distributed automated control systems (ACS) in power engineering by proposing an integral quality criterion that combines similarity-driven Markov process modeling with geometric programming. The methodology transforms the transition rate matrix of a continuous-time Markov chain (CTMC) into a matrix polynomial, enabling the derivation of normalized similarity indices and the development of a criterion-based model to quantify relative variations in system quality without requiring global optimization. The proposed approach yields a generalized criterion model that facilitates the ranking of modernization alternatives and the evaluation of the sensitivity of optimal decisions to parameter variations. The practical implementation is demonstrated through updated state transition graphs, quality functions, and UML-based architectures of diagnostic-ready evaluation modules. The scientific contribution of this work lies in the integration of similarity-based Markov modeling with the mathematical framework of geometric programming into a unified criterion model for the quantitative assessment of functional readiness under multistate conditions and probabilistic failures. The methodology enables the comparison of modernization scenarios using a unified integral indicator, assessment of sensitivity to structural and parametric changes, and seamless integration of quality evaluation into SCADA/Smart Grid environments as part of real-time diagnostics. The accuracy of the assessment depends on the adequacy of transition rate identification and the validity of the Markovian assumption. Future extensions include the real-time estimation of transition rates from event streams, generalization to semi-Markov processes, and multicriteria optimization considering cost, risk, and readiness. Full article

Blockchain oracles, as data intermediaries between on-chain and off-chain environments, have opened up a wide range of application scenarios for blockchain technology. The dependability of a blockchain oracle system will affect the dependability of blockchain systems. However, the dynamic and heterogeneous nature of blockchain oracle systems poses challenges to assessing their dependability. Furthermore, how to comprehensively analyze the dependability of blockchain oracle systems from multiple dimensions of transient availability, steady-state availability, and reliability is also a challenge. In order to solve these challenges, this paper proposes three models based on a semi-Markov process (SMP): (1) the SMP model for steady-state availability analysis; (2) the hierarchical model for transient analysis; and (3) the SMP model with absorption states for reliability analysis. Then, we derive the formulas for calculating the dependability metrics, which can be used to evaluate the dependability of blockchain oracle systems composed of any number of oracle nodes. Finally, based on the comparative experiments to verify the approximate accuracy of the proposed model and formulas, we analyze the impact of system parameters and the number of oracle nodes on the dependability metrics. The experimental results reveal that the key factor affecting availability is the failure time and recovery time of the threshold oracle, while the key factor affecting MTTF is the failure time of the threshold oracle. Full article

Maritime ferries increasingly operate under non-stationary hydro–meteorological conditions that complicate cost planning. This study investigates how short-term weather variability affects expenditures for a ferry on the Gdynia–Karlskrona route. We combine a state-based operational framework (18 discrete states) with a subsystem-level cost model covering navigation, propulsion/steering, loading/unloading, stability control, and mooring/anchoring. Direct and indirect costs are linked to subsystem activity and state duration, while weather is incorporated through hazard categories that scale hourly costs. Expert-elicited rates and observed monthly state durations provide the basis for baseline estimates and hazard scenario simulations. Results reveal a disproportionate cost structure: two open-sea states constitute over 97% of the baseline monthly cost (19,490.19 PLN). Weather hazards further amplify costs, with moderate (1st-degree) and severe (2nd-degree) scenarios producing increases of ~8% and ~20%, respectively, compared to normal conditions. By embedding weather as an endogenous factor in a probabilistic cost model based on a semi-Markov process, the approach enhances predictive fidelity and supports decision-making for climate-resilient planning. These findings suggest that adaptive routing, speed management, and targeted maintenance of the propulsion and steering subsystems during open-sea navigation offer the highest potential for cost resilience. The study provides operators and policymakers with a transparent framework for climate-resilient planning and investment in semi-enclosed maritime corridors. Full article

Multi-robot ensembles comprising several manipulators are commonly used in industrial settings to process non-deterministic flows of items loaded by an upstream source onto a shared transportation system. After the execution of a given task, the robots regularly deposit the items on a common output flow, which conveys the semi-finished material towards the downstream portion of the plant for further processing. The productivity and reliability of the entire process, which is affected by the plant layout, by the quality of the adopted scheduling and task assignment algorithms, and by the proper balancing of the input and output flows, may be degraded by random disturbances and transient conditions of the input flow. In this paper, a highly accurate event-based simulator of this kind of system is used in conjunction with a rollout algorithm to optimize the performance of the plant in all operating scenarios. The proposed method relies on a simulation of the plant that comprehensively considers the dynamic performance of the manipulators, their actual motion planning algorithms, the adopted scheduling and task assignment methods, and the regulation of the material flows. The simulation environment is built upon computationally efficient maps able to predict the execution time of the tasks assigned to the robots, considering all the determining factors, and on a representation of the manipulators themselves as finite state automata. The proposed formalization of the line balancing problem as a Markov Decision Process and the resulting rollout optimization method are shown to substantially improve the performance of the plant, even in challenging situations, and to be well suited to real-time implementation even on commodity hardware. Full article

Global supply chains often face uncertainties in production lead times, fluctuating exchange rates, and varying tariff regulations, all of which can significantly impact total profit. To address these challenges, this study formulates a multi-country supply chain problem as a Semi-Markov Decision Process (SMDP), integrating both currency variability and tariff levels. Using a Q-learning-based method (SMART), we explore three scenarios: (1) wide currency gaps under a uniform tariff, (2) narrowed currency gaps encouraging more local sourcing, and (3) distinct tariff structures that highlight how varying duties can reshape global fulfillment decisions. Beyond these baselines we analyze uncertainty-extended variants and targeted sensitivities (quantity discounts, tariff escalation, and the joint influence of inventory holding costs and tariff costs). Simulation results, accompanied by policy heatmaps and performance metrics, illustrate how small or large shifts in exchange rates and tariffs can alter sourcing strategies, transportation modes, and inventory management. A Deep Q-Network (DQN) is also applied to validate the Q-learning policy, demonstrating alignment with a more advanced neural model for moderate-scale problems. These findings underscore the adaptability of reinforcement learning in guiding practitioners and policymakers, especially under rapidly changing trade environments where exchange rate volatility and incremental tariff changes demand robust, data-driven decision-making. Full article

In the present study, we model the migration process and the changes in the market environment. The migration process is being modeled as an $\mathcal{F}$-inhomogeneous semi-Markov process with fuzzy states. The evolution of the migration process takes place within a stochastic market environment with fuzzy states, the transitions of which are being modeled as an $\mathcal{F}$-inhomogeneous semi-Markov process. We prove a recursive relation from which we could find the survival probabilities of the bonds or debts as functions of the basic parameters of the two $\mathcal{F}$-inhomogeneous semi-Markov processes. The asymptotic behavior of the survival probabilities is being found under certain easily met conditions in closed analytic form. Finally, we provide maximum likelihood estimators for the basic parameters of the proposed models. Full article

This paper is devoted to the study of stochastic optimal control of averaged stochastic differential delay equations (SDDEs) with semi-Markov switchings and their applications in economics. By using the Dynkin formula and solution of the Dirichlet–Poisson problem, the Hamilton–Jacobi–Bellman (HJB) equation and the inverse HJB equation are derived. Applications are given to a new Ramsey stochastic models in economics, namely the averaged Ramsey diffusion model with semi-Markov switchings. A numerical example is presented as well. Full article

To enhance road safety and optimize intelligent driving systems, this study introduces the concept of “driving style tendency” to characterize short-term driver behavior, particularly lane-changing patterns. A multidimensional framework is established to analyze driving roles and behaviors, utilizing a Hidden Semi-Markov Model and Hierarchical Dirichlet Process for the unsupervised segmentation of driving trajectory data into behavioral primitives. By systematically analyzing driver behaviors in leading and following scenarios, characteristic thresholds are derived through distribution fitting, enabling the development of a non-parametric Bayesian-based scoring method for driving style tendency. The K-means clustering algorithm is employed to transform primitive segments into quantifiable semantic information, facilitating the interpretation of driver behavior preferences. This research contributes to improved collision risk prediction in complex traffic environments, supports the design of personalized driving assistance systems, and provides valuable insights for autonomous driving technology development. Full article

The conventional diagnostic techniques for ethylene cracker furnace tube coking rely on manual expertise, offline analysis and on-site inspection. However, these methods have inherent limitations, including prolonged inspection times, low accuracy and poor real-time performance. This makes it challenging to meet the requirements of chemical production. The necessity for high efficiency, high reliability and high safety, coupled with the inherent complexity of the production process, results in data that is characterized by multimodal, nonlinear, non-Gaussian and strong noise. This renders the traditional data processing and analysis methods ineffective. In order to address these issues, this paper puts forth a novel soft measurement approach, namely the ‘Mixed Student’s t-distribution regression soft measurement model based on Variational Inference (VI) and Markov Chain Monte Carlo (MCMC)’. The initial variational distribution is selected during the initialization step of VI. Subsequently, VI is employed to iteratively refine the distribution in order to more closely approximate the true posterior distribution. Subsequently, the outcomes of VI are employed to initiate the MCMC, which facilitates the placement of the iterative starting point of the MCMC in a region that more closely approximates the true posterior distribution. This approach allows the convergence process of MCMC to be accelerated, thereby enabling a more rapid approach to the true posterior distribution. The model integrates the efficiency of VI with the accuracy of the MCMC, thereby enhancing the precision of the posterior distribution approximation while preserving computational efficiency. The experimental results demonstrate that the model exhibits enhanced accuracy and robustness in the diagnosis of ethylene cracker tube coking compared to the conventional Partial Least Squares Regression (PLSR), Gaussian Process Regression (GPR), Gaussian Mixture Regression (GMR), Bayesian Student’s T-Distribution Mixture Regression (STMR) and Semi-supervised Bayesian T-Distribution Mixture Regression (SsSMM). This method provides a scientific basis for optimizing and maintaining the ethylene cracker, enhancing its production efficiency and reliability, and effectively addressing the multimodal, non-Gaussian distribution and uncertainty of the coking data of the ethylene cracker furnace tube. Full article

Complex dynamics and nonlinear systems play a critical role in industrial processes, where complex interactions, high uncertainty, and external disturbances can significantly impact efficiency, stability, and safety. In sectors such as mining, manufacturing, and energy networks, even small perturbations can lead to unexpected system behaviors, operational inefficiencies, or cascading failures. Understanding and controlling these dynamics is essential for developing robust, adaptive, and resilient industrial systems. This study conducts a systematic literature review covering 2015–2025 in Scopus and Web of Science, initially retrieving 2628 (Scopus) and 343 (WoS) articles. After automated filtering (Python) and applying inclusion/exclusion criteria, a refined dataset of 2900 references was obtained, from which 89 highly relevant studies were selected. The literature was categorized into six key areas: (i) heat transfer with magnetized fluids, (ii) nonlinear control, (iii) big-data-driven optimization, (iv) energy transition via SOEC, (v) fault detection in control valves, and (vi) stochastic modeling with semi-Markov switching. Findings highlight the convergence of robust control, machine learning, IoT, and Industry 4.0 methodologies in tackling industrial challenges. Cybersecurity and sustainability also emerge as critical factors in developing resilient models, alongside barriers such as limited data availability, platform heterogeneity, and interoperability gaps. Future research should integrate multiscale analysis, deterministic chaos, and deep learning to enhance the adaptability, security, and efficiency of industrial operations in high-complexity environments. Full article

We develop a deep reinforcement learning (RL) framework for an optimal market-making (MM) trading problem, specifically focusing on price processes with semi-Markov and Hawkes Jump-Diffusion dynamics. We begin by discussing the basics of RL and the deep RL framework used; we deployed the state-of-the-art Soft Actor–Critic (SAC) algorithm for the deep learning part. The SAC algorithm is an off-policy entropy maximization algorithm more suitable for tackling complex, high-dimensional problems with continuous state and action spaces, like those in optimal market-making (MM). We introduce the optimal MM problem considered, where we detail all the deterministic and stochastic processes that go into setting up an environment to simulate this strategy. Here, we also provide an in-depth overview of the jump-diffusion pricing dynamics used and our method for dealing with adverse selection within the limit order book, and we highlight the working parts of our optimization problem. Next, we discuss the training and testing results, where we provide visuals of how important deterministic and stochastic processes such as the bid/ask prices, trade executions, inventory, and the reward function evolved. Our study includes an analysis of simulated and real data. We include a discussion on the limitations of these results, which are important points for most diffusion style models in this setting. Full article

Federated learning has attracted widespread attention due to its strong capabilities of privacy protection, making it a powerful supporting technology for addressing data silos in the future. However, federated learning still lags significantly behind traditional centralized learning in terms of learning efficiency and system security. In this paper, we first construct a hierarchical federated learning architecture integrated with blockchain based on the cooperation of the cloud, edge, and terminal, which has the ability to enhance the security of federated learning while reducing the introduction costs of blockchain. Under this architecture, we propose a semi-asynchronous aggregation scheme at the edge layer and introduce a hierarchical aggregation scheme that combines it with synchronous aggregation at the cloud end to improve system efficiency. Furthermore, we present a multi-objective node selection scheme that considers various influencing factors such as security and efficiency. We formulate the node selection problem as a Markov Decision Process (MDP) and propose a solution based on deep reinforcement learning to address it more efficiently. The experimental results show that the proposed scheme can effectively improve system efficiency and enhance system security. In addition, the proposed DQN-based node selection algorithm can efficiently realize the selection of the optimal policy. Full article

Caching the contents of unmanned aerial vehicles (UAVs) could significantly improve the content fetching performance of request users (RUs). In this paper, we study UAV trajectory design, content fetching, power allocation, and content placement problems in multi-UAV-aided networks, where multiple UAVs can transmit contents to the assigned RUs. To minimize the energy consumption of the system, we develop a constrained optimization problem that simultaneously designs UAV trajectory, power allocation, content fetching, and content placement. Since the original minimization problem is a mixed-integer nonlinear programming (MINLP) problem that is difficult to solve, the optimization problem was first transformed into a semi-Markov decision process (SMDP). Next, we developed a new technique to solve the joint optimization problem: option-based hierarchical deep reinforcement learning (OHDRL). We define UAV trajectory planning and power allocation as the low-level action space and content placement and content fetching as the high-level option space. Stochastic optimization can be handled using this strategy, where the agent makes high-level option selections, and the action is carried out at a low level based on the chosen option’s policy. When comparing the proposed approach to the current technique, the numerical results show that it can produce more consistent learning performance and reduced energy consumption. Full article

In the current mobile edge computing (MEC) system, the user dynamics, diversity of applications, and heterogeneity of services have made cooperative service caching and task offloading decision increasingly important. Service caching and task offloading have a naturally hierarchical structure, and thus, hierarchical reinforcement learning (HRL) can be used to effectively alleviate the dimensionality curse in it. However, traditional HRL algorithms are designed for short-term missions with sparse rewards, while existing HRL algorithms proposed for MEC lack delicate a coupling structure and perform poorly. This article introduces a novel HRL-based algorithm, named hierarchical service caching and task offloading (HSCTO), to solve the problem of the cooperative optimization of service caching and task offloading in MEC. The upper layer of HSCTO makes decisions on service caching while the lower layer is in charge of task offloading strategies. The upper-layer module learns policies by directly utilizing the rewards of the lower-layer agent, and the tightly coupled design guarantees algorithm performance. Furthermore, we adopt a fixed multiple time step method in the upper layer, which eliminates the dependence on the semi-Markov decision processes (SMDPs) theory and reduces the cost of frequent service replacement. We conducted numerical evaluations and the experimental results show that HSCTO improves the overall performance by 20%, and reduces the average energy consumption by 13% compared with competitive baselines. Full article

We propose a semi-empirical method—based on a filtered Markov process—to convert 10 min rain rate time series into 1 min time series, i.e., quasi-instantaneous rainfall—the latter to be used as input to the synthetic storm technique, which is a very reliable tool for calculating rain attenuation time series in satellite communication systems or for estimating runoff, erosion, pollutant transport, and other applications in hydrology. To develop the method, we used a very large data bank of 1 min rain rate time series collected in several sites with different climatic conditions. The experimental and simulated 1 min rain rate time series agree very well. Afterward, we used them to simulate rain attenuation time series at 20.7 GHz, in 35.5° slant paths to geostationary satellites. The two simulated annual rain attenuation probability distributions show very small differences. We conclude that the rain rate conversion method is very reliable. Full article