Search Results (18)

The rapid expansion of the tourism industry underscores the necessity for sustainable hotel recommendation systems that guide user choices while safeguarding the long-term viability of the tourism ecosystem. However, existing methods often struggle to reconcile individual user preferences with sustainable consumption objectives, frequently encountering the “information cocoon” effect and lacking interpretability in their decision-making processes. To address these issues, this study proposes a multi-objective, context-aware hotel recommendation framework that integrates text mining, sequential behavior modeling, and reinforcement learning. The framework begins by employing unsupervised learning to extract multidimensional hotel features from online reviews, with an explicit emphasis on comprehensive sustainability metrics. It subsequently applies a dynamic state representation approach that merges long-term and short-term interests with real-time contextual information to accurately reflect evolving consumer needs. Furthermore, a dynamic feature weighting module is incorporated to enhance interpretability and enable context-adaptive evaluation of both commercial and sustainable attributes. The recommendation process is structured as a Markov Decision Process, leveraging a composite reward function comprising diversity penalties and sustainability incentives. Empirical analysis using real-world data validates the framework, demonstrating its contribution to sustainable tourism and achieving recommendation accuracy that surpasses existing benchmark models. Full article

Distributed Energy Resource (DER)-integrated power grids are vulnerable to cascading effects under concurrent cyber–physical attacks, where even minor disruptions in system states accumulate and amplify over time, leading to significant system failures. Traditional static risk assessment methods are insufficient for modeling these time-varying, dynamic scenarios, particularly in the context of concurrent attacks. This paper presents a dynamic risk assessment framework leveraging time-synchronized co-simulation, which integrates power system and communication network simulations within a unified time framework. Cyber-attack actions in the communication layer are mapped to corresponding physical disturbances in the distribution network, including voltage, frequency, and power variations. Using the resulting system state evolution trajectories, a Markov Decision Process (MDP)-based state transition tree captures the progression of system risk under concurrent attacks. This framework accounts for cumulative risk across different attack paths and identifies critical nodes and high-risk propagation paths within the network. By incorporating a concurrent event detector into the MDP model, the method quantifies evolving risk dynamics, overcoming the limitations of traditional static methods. Case studies on the IEEE 13-node test feeder and IEEE 14-bus system demonstrate that concurrent attacks result in a security risk metric 2.3 times higher than single-point attacks, validating the effectiveness of the proposed approach in identifying vulnerable nodes whose compromise could lead to cascading failures, supporting the risk-aware prioritization of defensive resources. Full article

In the field of ocean observation, we often face the challenge of the contradiction between the vast ocean environment and limited ocean sensor observations, resulting in significant sparsity in the acquired ocean sensor data. This sparse ocean sensor data typically exhibits characteristics such as discrete spatial distribution, discontinuous observation time, and vertical stratification with water depth variations. Current methods primarily employ rule-based quality control, time series modeling, or traditional graph neural networks for processing. This paper addresses the characteristics of sparse ocean sensor data, building upon these methods by further utilizing topological correlation and hierarchical feature modeling on a topological basis. It proposes a depth-aware heterogeneous spatiotemporal graph neural network (DAHSGNN) to achieve efficient anomaly detection and data correction for this type of data. DAHSGNN integrates discrete observation data along the depth axis using a local graph construction method. It employs hierarchical feature engineering to characterize the vertical stratification of the ocean. A Gaussian Hidden Markov Model is used to segment the water layers, and intra- and inter-layer trend features are extracted using a water layer probability-guided Transformer encoder. Then, a bidirectional long short-term memory deep sequence encoder captures the local dynamic context, thereby achieving fine-grained modeling of the ocean’s vertical stratification features. Finally, a heterogeneous graph autoencoder is used to reconstruct the site-level data distribution. Experiments were conducted using multiple environmental variables from the International Seabed Authority (ISA) DeepData database. Results show that DAHSGNN exhibits good cross-variable generalization ability, achieves higher reconstruction accuracy than baseline methods, and significantly improves anomaly detection performance. Full article

The accurate and reliable evaluation of large language models (LLMs) in medical domains is critical for real-world clinical deployment, automated medical reasoning, and patient safety. However, the evaluation process is highly sensitive to prompt design, and prevalent reliance on fixed or randomly sampled prompt policies often fails to dynamically adapt to clinical context, question complexity, or evolving safety requirements. This article presents a novel reinforcement learning-based framework for multi-prompt selection, which dynamically optimizes prompt policy per input for medical LLM evaluation across the Medical Knowledge Question-Answering dataset (MKQA), the Medical Multiple-Choice Question dataset (MCQ), and the Doctor-Patient Dialogue dataset. We formulate prompt selection as a Markov Decision Process (MDP) and employ a deep Q-Network (DQN) agent to maximize a reward signal incorporating textual accuracy, domain terminology coverage, safety, and dialogue relevance. Experiments on three medical LLM benchmarks demonstrate consistent improvements in composite reward (e.g., a 6.66% increase in MKQA vs. Random Baseline, and a 2.41% increase in Dialogue vs. Fixed Baseline) when compared to baselines. This was accompanied by robust enhancements in Safety (e.g., achieving 1.0000 in MKQA, a 5.26% increase vs. Fixed Baseline; and a 5.03% increase in Dialogue vs. Fixed Baseline) and substantial gains in Medical Terminology Coverage (e.g., a 74.61% increase in MKQA vs. Fixed Baseline, and a 9.13% increase in MCQ vs. Fixed Baseline) when compared to baselines. While varying across tasks, an improvement in accuracy was observed in the MKQA task, and the framework effectively optimizes the multi-objective reward function, even when minor trade-offs in other metrics like Accuracy and Contextual Relevance were observed in some contexts. Our framework enables robust, context-aware, and adaptive evaluation, laying a foundation for safer and more reliable LLM application in healthcare. Full article

Geochemical data from mine tailings are layered, compositional, and noisy, complicating automated domaining. This study introduces a continuity-aware workflow the Geostatistical k-means Recurrent Neural Network (GkRNN) that links compositional preprocessing and geostatistical continuity to sequence learning, allowing depth order and lateral context to influence final domain labels. The workflow begins with a centered log-ratio (CLR) transform, followed by construction of a spectral embedding derived from kernelized direct and cross variograms. Clustering is carried out in this embedded space, and depth sequences are regularized with a hidden Markov model (HMM) model and a long short-term memory (LSTM) network. When applied to a multivariate set of tailing drillholes, stratigraphically coherent zones were obtained, depthwise proportions were stabilized, and vertical as well as lateral semivariograms remained consistent with laminated material. Compared with k-means and Gaussian Mixture baselines, over-segmentation was reduced and the intended layered architecture was recovered in most drillholes. The result is a reproducible domaining workflow that enables clearer grade estimation and more transparent risk evaluation. Full article

In the context of growing data availability and increasing complexity of demand patterns in retail fuel distribution, selecting effective forecasting models for large collections of time series is becoming a key operational challenge. This study investigates the effectiveness of a hybrid forecasting approach combining ARIMA models with dynamically updated Markov Chains. Unlike many existing studies that focus on isolated or small-scale experiments, this research evaluates the hybrid model across a full set of approximately 150 time series collected from multiple petrol stations, without pre-clustering or manual selection. A comprehensive set of statistical and structural features is extracted from each time series to analyze their relation to forecast performance. The results show that the hybrid ARIMA–Markov approach significantly outperforms both individual statistical models and commonly applied machine learning methods in many cases, particularly for non-stationary or regime-shifting series. In 100% of cases, the hybrid model reduced the error compared to both baseline models—the median RMSE improvement over ARIMA was 13.03%, and 15.64% over the Markov model, with statistical significance confirmed by the Wilcoxon signed-rank test. The analysis also highlights specific time series features—such as entropy, regime shift frequency, and autocorrelation structure—as strong indicators of whether hybrid modeling yields performance gains. Feature-conditioning analyses (e.g., lag-1 autocorrelation, volatility, entropy) explain when hybridization helps, enabling a feature-aware workflow that selectively deploys model components and narrows parameter searches. The greatest benefits of applying the hybrid model were observed for time series characterized by high variability, moderate entropy of differences, and a well-defined temporal dependency structure—the correlation values between these features and the improvement in hybrid performance relative to ARIMA and Markov models reached 0.55–0.58, ensuring adequate statistical significance. Such approaches are particularly valuable in enterprise environments dealing with thousands of time series, where automated model configuration becomes essential. The findings position interpretable, adaptive hybrids as a practical default for short-horizon demand forecasting in fuel supply chains and, more broadly, in energy-use applications characterized by heterogeneous profiles and evolving regimes. Full article

We introduce a novel, privacy-preserving AI authoring copilot designed for educational content creation, which uniquely combines a Markov Decision Process (MDP) as a reinforcement learning orchestrator with a locally deployed Qwen3-1.7B-ONNX large language model to iteratively refine text for clarity, unity, and engagement—all running on a modest CPU-only system (Intel i7, 16 GB RAM). Unlike cloud-dependent models, our agent treats writing as a sequential decision problem, selecting refinement actions (e.g., simplification, elaboration) based on real-time LLM and sentiment feedback, ensuring pedagogically sound outputs without internet dependency. Evaluated across five diverse topics, our MDP-orchestrated agent achieved an overall average quality score of 4.23 (on a 0–5 scale), statistically equivalent to leading cloud-based LLMs like ChatGPT and DeepSeek. This performance was validated through blind evaluations by four independent LLMs and human raters, supported by statistical consistency analysis. Our work demonstrates that lightweight local LLMs, when guided by principled MDP policies, can deliver high-quality, context-aware educational content, bridging the gap between powerful AI generation and ethical, on-device deployment. This advancement empowers educators, researchers, and curriculum designers with a trustworthy, accessible tool for intelligent content augmentation aligning with the Quality Education Sustainable Development Goal through innovations in educational technology, inclusive education, equity in education, and lifelong learning. Full article

Autonomous vehicle (AV) navigation in dynamic urban environments faces challenges such as unpredictable traffic conditions, varying road user behaviors, and complex road networks. This study proposes a novel reinforcement learning-based framework that enhances AV decision making through spatial-temporal context awareness. The framework integrates Proximal Policy Optimization (PPO) and Graph Neural Networks (GNNs) to effectively model urban features like intersections, traffic density, and pedestrian zones. A key innovation is the urban context-aware reward mechanism (UCARM), which dynamically adapts the reward structure based on traffic rules, congestion levels, and safety considerations. Additionally, the framework incorporates a Dynamic Risk Assessment Module (DRAM), which uses Bayesian inference combined with Markov Decision Processes (MDPs) to proactively evaluate collision risks and guide safer navigation. The framework’s performance was validated across three datasets—Argoverse, nuScenes, and CARLA. Results demonstrate significant improvements: An average travel time of 420 ± 20 s, a collision rate of 3.1%, and energy consumption of 11,833 ± 550 J in Argoverse; 410 ± 20 s, 2.5%, and 11,933 ± 450 J in nuScenes; and 450 ± 25 s, 3.6%, and 13,000 ± 600 J in CARLA. The proposed method achieved an average navigation success rate of 92.5%, consistently outperforming baseline models in safety, efficiency, and adaptability. These findings indicate the framework’s robustness and practical applicability for scalable AV deployment in real-world urban traffic conditions. Full article

Efficient and intelligent spectrum access is crucial for meeting the diverse Quality of Service (QoS) demands of Vehicular Internet of Things (V-IoT) systems in next-generation cellular networks. This work proposes a novel reinforcement learning (RL)-based priority-aware spectrum management (RL-PASM) framework, a centralized self-learning priority-aware spectrum management framework operating through Roadside Units (RSUs). RL-PASM dynamically allocates spectrum resources across three traffic classes: high-priority (HP), low-priority (LP), and best-effort (BE), utilizing reinforcement learning (RL). This work compares four RL algorithms: Q-Learning, Double Q-Learning, Deep Q-Network (DQN), and Actor-Critic (AC) methods. The environment is modeled as a discrete-time Markov Decision Process (MDP), and a context-sensitive reward function guides fairness-preserving decisions for access, preemption, coexistence, and hand-off. Extensive simulations conducted under realistic vehicular load conditions evaluate the performance across key metrics, including throughput, delay, energy efficiency, fairness, blocking, and interruption probability. Unlike prior approaches, RL-PASM introduces a unified multi-objective reward formulation and centralized RSU-based control to support adaptive priority-aware access for dynamic vehicular environments. Simulation results confirm that RL-PASM balances throughput, latency, fairness, and energy efficiency, demonstrating its suitability for scalable and resource-constrained deployments. The results also demonstrate that DQN achieves the highest average throughput, followed by vanilla QL. DQL and AC maintain fairness at high levels and low average interruption probability. QL demonstrates the lowest average delay and the highest energy efficiency, making it a suitable candidate for edge-constrained vehicular deployments. Selecting the appropriate RL method, RL-PASM offers a robust and adaptable solution for scalable, intelligent, and priority-aware spectrum access in vehicular communication infrastructures. Full article

We propose a novel framework for adaptive stochastic dynamics analysis of tourist behavior by integrating context-aware Markov models with finite mixture models (FMMs). Conventional Markov models often fail to capture abrupt changes induced by external shocks, such as event announcements or weather disruptions, leading to inaccurate predictions. The proposed method addresses this limitation by introducing virtual sensors that dynamically detect contextual anomalies and trigger regime switches in real-time. These sensors process streaming data to identify shocks, which are then used to reweight the probabilities of pre-learned behavioral regimes represented by FMMs. The system employs expectation maximization to train distinct Markov sub-models for each regime, enabling seamless transitions between them when contextual thresholds are exceeded. Furthermore, the framework leverages edge computing and probabilistic programming for efficient, low-latency implementation. The key contribution lies in the explicit modeling of contextual shocks and the dynamic adaptation of stochastic processes, which significantly improves robustness in volatile tourism scenarios. Experimental results demonstrate that the proposed approach outperforms traditional Markov models in accuracy and adaptability, particularly under rapidly changing conditions. Quantitative results show a 13.6% improvement in transition accuracy (0.742 vs. 0.653) compared to conventional context-aware Markov models, with an 89.2% true positive rate in shock detection and a median response latency of 47 min for regime switching. This work advances the state-of-the-art in tourist behavior analysis by providing a scalable, real-time solution for capturing complex, context-dependent dynamics. The integration of virtual sensors and FMMs offers a generalizable paradigm for stochastic modeling in other domains where external shocks play a critical role. Full article

The rapid development of mobile sensing technologies (e.g., smart devices embedded with various powerful sensors) has encouraged the proliferation of the Internet of Things (IoT). Although data reliability and accuracy are crucial in many sensor applications (e.g., air-quality monitoring), it is often difficult to ensure these properties. Mobile IoT’s people-centric architecture allows for more inaccurate and corrupted data. In this manuscript, we are addressing the problem of how to predict data more accurately in the presence of malicious participants who inject false data to manipulate the system. Our goal is to recover those missing or imprecise data values from the correlated data streams. To do so, we propose a Reputation-Based Context-Aware Data-Fusion (RCoD) mechanism that is resilient against on–off and data-corruption attacks. Furthermore, the Contextual Hidden Markov Model-based data prediction facilitates more accurate real-time data prediction. We tested the scenarios where most participants were malicious, injecting false data at varied rates. Our method accurately identified the honest participants based on their reported data and context. We empirically evaluate the performance using Beijing’s air-quality dataset. We compared the performance of our RCoD method against four state-of-the-art methods, and the results justify its superiority. Full article

This paper presents a context-aware adaptive assembly assistance system meant to support factory workers by embedding predictive capabilities. The research is focused on the predictor which suggests the next assembly step. Hidden Markov models are analyzed for this purpose. Several prediction methods have been previously evaluated and the prediction by partial matching, which was the most efficient, is considered in this work as a component of a hybrid model together with an optimally configured hidden Markov model. The experimental results show that the hidden Markov model is a viable choice to predict the next assembly step, whereas the hybrid predictor is even better, outperforming in some cases all the other models. Nevertheless, an assembly assistance system meant to support factory workers needs to embed multiple models to exhibit valuable predictive capabilities. Full article

Timely status updates are critical in remote control systems such as autonomous driving and the industrial Internet of Things, where timeliness requirements are usually context dependent. Accordingly, the Urgency of Information (UoI) has been proposed beyond the well-known Age of Information (AoI) by further including context-aware weights which indicate whether the monitored process is in an emergency. However, the optimal updating and scheduling strategies in terms of UoI remain open. In this paper, we propose a UoI-optimal updating policy for timely status information with resource constraint. We first formulate the problem in a constrained Markov decision process and prove that the UoI-optimal policy has a threshold structure. When the context-aware weights are known, we propose a numerical method based on linear programming. When the weights are unknown, we further design a reinforcement learning (RL)-based scheduling policy. The simulation reveals that the threshold of the UoI-optimal policy increases as the resource constraint tightens. In addition, the UoI-optimal policy outperforms the AoI-optimal policy in terms of average squared estimation error, and the proposed RL-based updating policy achieves a near-optimal performance without the advanced knowledge of the system model. Full article

The age of information (AoI) has been widely used to quantify the information freshness in real-time status update systems. As the AoI is independent of the inherent property of the source data and the context, we introduce a mutual information-based value of information (VoI) framework for hidden Markov models. In this paper, we investigate the VoI and its relationship to the AoI for a noisy Ornstein–Uhlenbeck (OU) process. We explore the effects of correlation and noise on their relationship, and find logarithmic, exponential and linear dependencies between the two in three different regimes. This gives the formal justification for the selection of non-linear AoI functions previously reported in other works. Moreover, we study the statistical properties of the VoI in the example of a queue model, deriving its distribution functions and moments. The lower and upper bounds of the average VoI are also analysed, which can be used for the design and optimisation of freshness-aware networks. Numerical results are presented and further show that, compared with the traditional linear age and some basic non-linear age functions, the proposed VoI framework is more general and suitable for various contexts. Full article

The co-evolution between information and epidemic in multilayer networks has attracted wide attention. However, previous studies usually assume that two networks with the same individuals are coupled into a multiplex network, ignoring the context that the individuals of each layer in the multilayer network are often different, especially in group structures with rich collective phenomena. In this paper, based on the scenario of group-based multilayer networks, we investigate the coupled UAU-SIS (Unaware-Aware-Unaware-Susceptible-Infected-Susceptible) model via microscopic Markov chain approach (MMCA). Importantly, the evolution of such transmission process with respective to various impact factors, especially for the group features, is captured by simulations. We further obtain the theoretical threshold for the onset of epidemic outbreaks and analyze its characteristics through numerical simulations. It is concluded that the growth of the group size of information (physical) layer effectively suppresses (enhances) epidemic spreading. Moreover, taking the context of epidemic immunization into account, we find that the propagation capacity and robustness of this type of network are greater than the conventional multiplex network. Full article