Search Results (18)

Urban healthcare systems are fundamentally constrained by the mismatch between static resource configurations and dynamically evolving patient demand. Under the tiered healthcare system, traditional static planning methods struggle to capture the complexity and randomness of patient flows. While recent reinforcement learning (RL) approaches enable adaptive decision-making, they suffer from dimensionality explosion and unstable convergence due to massive action spaces and delayed spatiotemporal credit assignment in city-scale environments. To address this gap, we propose Twin–Game: a digital twin-driven hierarchical reinforcement learning (HRL) framework that formulates adaptive healthcare resource optimization as a “Twin Game” between a simulation-based game environment (Strategic Sandbox) and a hierarchical decision policy. First, we construct the “first twin”—an offline digital twin that serves as the Strategic Sandbox parameterized with Wuhan’s observed facility, population, and transportation data, while patient arrivals and disease profiles are generated synthetically under documented assumptions because individual-level clinical flow data are not publicly available. This environment integrates a dynamic gravity model with a two-way referral mechanism to represent the nonlinear coupling between hospital attractiveness, crowding levels, and patient choice behaviors. Second, we build the “second twin”—an Option-based HRL policy. The Manager (Macro-level Strategic Layer) uses a Deep Q-Network (DQN) for discrete spatial attention allocation; the Worker (Micro-level Execution Layer) uses Proximal Policy Optimization (PPO) for continuous, fine-grained controls such as bed expansion ratios and personnel scheduling. The two twins interact in a closed-loop game, performing strategy search and game evolution under complex constraints to optimize allocation. Experimental results from the Wuhan case indicate that the Twin–Game framework outperforms static baselines and single-layer RL in reducing average travel times, enhancing resource utilization, and improving tiered diagnosis and treatment within the simulation setting. The results should be interpreted as simulation-based decision-support evidence rather than direct clinical validation. This study provides a data-driven, game-theoretic decision support tool for building resilient urban healthcare systems. Full article

Cement stabilized macadam (CSM) bases are prone to cracking and void damage under long-term traffic loading and environmental actions, which accelerates structural deterioration. Although grouting is an effective method for treating such concealed defects, laboratory-based evaluation of repair effectiveness remains limited. In this study, field-cored CSM specimens were recombined in a cylindrical mold to simulate four void conditions (1/4, 2/4, 3/4, and 4/4), and repaired using an inorganic cementitious composite grouting material based on ultra-fine cement and high-belite sulphoaluminate cement (HBSAC), and modified with ethylene-vinyl acetate (EVA) latex, wollastonite (WO) whiskers, and polyvinyl alcohol (PVA) fibers. The repair effectiveness was evaluated through ultrasonic testing, capacitance measurement, uniaxial compression with acoustic emission (AE) monitoring, and computed tomography (CT). The results show that the longitudinal wave velocity of all repaired groups increases continuously with curing time, with a maximum increase of 21.98% at 28 days. The normalized capacitance response exhibits clear time- and layer-dependent variation, with the 4/4 group showing the most pronounced spatial heterogeneity. In the uniaxial compression tests, the peak load increases from 181 kN in the control group to 201–286 kN in the repaired groups, while the tensile-related AE event proportion increases from 77.35% in the 1/4 group to 89.38% in the 4/4 group. CT analysis shows that the proportion of micropores smaller than 1 mm³ increases from 66.3% to 82.7%, whereas the proportion of pores larger than 100 mm³ decreases from 46.5% to 21.6% after repair. These results demonstrate that the composite grouting material provides effective filling, structural reconstruction, and mechanical enhancement for void-damaged CSM, and that the proposed multi-source characterization framework is suitable for evaluating grouting repair performance. Full article

Extended reality (XR) has increasingly been applied to nursing practicum education; however, most systems rely on controller-based interfaces that limit precise capture of continuous fine motor performance and objective assessment. This study developed and validated a sensor-integrated, controller-free XR nursing practicum system (Smart Nursing v1.0) grounded in continuous precision sensing. Based on internationally standardized intravenous injection protocols, the system integrated optical hand tracking and speech recognition to quantify hand kinematics, spatial accuracy, procedural sequencing, and verbal compliance. A three-phase validation framework was implemented. Internal technical verification confirmed stable real-time performance (≥60 FPS) and consistent action recognition. In a user-based study involving 63 undergraduate nursing students, XR-based automated scores demonstrated high agreement with expert instructor ratings (ICC = 0.932, 95% CI = 0.91–0.96, p < 0.001). XR baseline scores significantly predicted post-training performance (β = 0.632, p < 0.001) and showed significant incremental validity beyond instructor pre-training scores (ΔR² = 0.186, p < 0.001). Independent verification confirmed high recognition accuracy (100%) and system stability. These findings indicate that precision sensing enables XR environments to function as reliable performance measurement systems, supporting standardized non-face-to-face nursing practicum education. Full article

In skeletal muscles fibers, cellular respiration, excitation–contraction (EC) coupling (the mechanism that translates action potentials in Ca²⁺ release), and store-operated Ca²⁺ entry (SOCE, a mechanism that allows recovery of external Ca²⁺ during fatigue) take place in organelles specifically dedicated to each function: (a) aerobic ATP production in mitochondria; (b) EC coupling in intracellular junctions formed by association between transverse tubules (TTs) and sarcoplasmic reticulum (SR) named triads; (c) SOCE in Ca²⁺ entry units (CEUs), SR-TT junctions that are in continuity with membranes of triads, but that contain a different molecular machinery (see Graphical Abstract). In the past 20 years, we have studied skeletal muscle fibers by collecting biopsies from humans and isolating muscles from animal models (mouse, rat, rabbit) under different conditions of muscle inactivity (sedentary aging, denervation, immobilization by casting) and after exercise, either after voluntary training in humans (running, biking, etc.) or in mice kept in wheel cages or after running protocols on a treadmill. In all these studies, we have assessed the ultrastructure of the mitochondrial network and of the sarcotubular system (i.e., SR plus TTs) by electron microscopy (EM) and then collected functional data correlating (i) the changes occurring with aging and inactivity with a loss-of-function, and (ii) the structural improvement/rescue after exercise with a gain-of-function. The picture that emerged from this long journey points to the importance of the internal architecture of muscle fibers for their capability to function properly. Indeed, we discovered how the intracellular organization of the mitochondrial network and of the membrane systems involved in controlling intracellular calcium concentration (i[Ca²⁺]) is finely controlled and remodeled by inactivity and exercise. In this manuscript, we give an integrated picture of changes caused by inactivity and exercise and how they may affect muscle function. Full article

Hydrocyclones are widely used for solid–liquid separation, but their performance is highly sensitive to the geometric design. Previous studies often focused on individual structural parameters; however, the combined effects of the vortex finder diameter and aspect ratios on the internal flow field and particle separation behavior remain insufficiently clarified. This study conducted three-dimensional numerical simulations using the realizable k-ε turbulence model, combined with the discrete phase model. The particle size distribution behaves according to the Rosin–Rammler function. Seven different geometries were evaluated under identical operating conditions to systematically investigate how the diameter and aspect ratios influence the internal vortex structures and separation behavior. A decrease in the diameter ratio enhances the dominance of the outward centrifugal forces, which increases the downward discharge of coarse particles but also results in greater liquid entrainment through the underflow. Conversely, larger diameter ratios strengthen the secondary vortex and promote upward flow. However, this also leads to decreased recovery of fine particles due to weakened centrifugal action. Adjusting the aspect ratio effectively mitigates these tradeoffs. Increasing the cone length enhances the residence time, stabilizes the upward vortex, and improves the separation of fine particles. Although the overall separation performance shows diminishing returns beyond a certain aspect-ratio threshold, the recovery of fine particles continues to improve. The results reveal that a balance between centrifugal and drag forces is essential, which is achieved through coordinated control of the vortex finder diameter and cone geometry. This balance is critical for maintaining stable flow fields and high efficiency in fine-particle removal. The findings provide practical design guidance for hydrocyclones, particularly in applications that require enhanced recovery of fine particles and stable multiphase flow behavior. Full article

In many real-world control tasks, agents operate under partial observability, where access to complete state information is limited or corrupted by noise. This poses significant challenges for reinforcement learning algorithms, as methods relying on full states or long observation histories can be computationally expensive and less robust. Four-Phase Rapid Motor Adaptation (FRMA) is a reinforcement learning framework designed to address these challenges in high-frequency control tasks under partial observability. FRMA proceeds through four sequential stages: (i) full-state pretraining to establish a strong initial policy, (ii) auxiliary hidden-state prediction for LSTM memory initialization, (iii) aligned latent representation learning to bridge partial observations with full-state dynamics, and (iv) latent-state policy fine-tuning for robust deployment. Notably, FRMA leverages full-state information ($s_t$) only during training to supervise latent representation learning, while at deployment it requires only short sequences of recent observations and actions. This allows agents to infer compact and informative latent states, achieving performance comparable to policies with full-state access. Extensive experiments on continuous control benchmarks show that FRMA attains near-optimal performance even with minimal observation–action histories, reducing reliance on long-term memory and computational resources. Moreover, FRMA demonstrates strong robustness to observation noise, maintaining high control accuracy under substantial sensory corruption. These results indicate that FRMA provides an effective and generalizable solution for partially observable control tasks, enabling efficient and reliable agent operation when full state information is unavailable or noisy. Full article

Brushless DC (BLDC) motors are commonly used in electric vehicles (EVs) because of their efficiency, small size and great torque-speed performance. These motors have a few benefits such as low maintenance, increased reliability and power density. Nevertheless, BLDC motors are highly nonlinear and their dynamics are very complicated, in particular, under changing load and supply conditions. The above features require the design of strong and adaptable control methods that can ensure performance over a broad spectrum of disturbances and uncertainties. In order to overcome these issues, this paper uses a Fractional-Order Proportional-Integral-Derivative (FOPID) controller that offers better control precision, better frequency response, and an extra degree of freedom in tuning by using non-integer order terms. Although it has the benefits, there are three primary drawbacks: (i) it is not real-time adaptable, (ii) it is hard to choose appropriate initial gain values, and (iii) it is sensitive to big disturbances and parameter changes. A new control framework is suggested to address these problems. First, a Reinforcement Learning (RL) approach based on Deep Deterministic Policy Gradient (DDPG) is presented to optimize the FOPID gains online so that the controller can adjust itself continuously to the variations in the system. Second, Snake Optimization (SO) algorithm is used in fine-tuning of the FOPID parameters at the initial stages to guarantee stable convergence. Lastly, cascade control structure is adopted, where FOPID controllers are used in the inner (current) and outer (speed) loops. This construction adds robustness to the system as a whole and minimizes the effect of disturbances on the performance. In addition, the cascade design also allows more coordinated and smooth control actions thus reducing stress on the power electronic switches, which reduces switching losses and the overall efficiency of the drive system. The suggested RL-enhanced cascade FOPID controller is verified by Hardware-in-the-Loop (HIL) testing, which shows better performance in the aspects of speed regulation, robustness, and adaptability to realistic conditions of operation in EV applications. Full article

Phosphate ores, which are regarded as critical mineral resources, play an important role in various industrial fields. Apatite is the main source of phosphate mineral resources and must be concentrated before it is processed into industrial products. Flotation is the most commonly employed method for apatite concentration. However, as the proportion of fine apatite increases, the challenge of separating it from gangue minerals intensifies, due to the resemblance in surface characteristics between apatite and gangue. Interfacial regulation during flotation is fundamental to the process, including the regulation of the mineral/water interface wettability by flotation reagents (collectors and modifiers), the control of interactions between mineral particles, and the regulation of interactions between mineral particles and bubbles. This article introduces the surface characteristics of apatite and its main gangue minerals. It discusses innovative work on flotation reagents (primarily collectors and depressants) and their action mechanisms on mineral surfaces. It reviews the current development of theories on the regulation of interactions between interparticles and between particles and bubbles. Finally, the study outlook the future research on interfacial regulation in apatite flotation. This study is intended to offer references for the continued advancement of apatite flotation. Full article

Fine particulate matter (PM_2.5) and Ozone (O₃) pollution have emerged as the primary environmental challenges in China in recent years. Following the implementation of the Air Pollution Prevention and Control Action Plan, a substantial decline in PM_2.5 concentrations was observed, while O₃ concentrations exhibited an increasing trend across the country. Here, we investigated the long-term trend of O₃ from 2015 to 2022 in Xinxiang City, a typical city within the Central Plains urban agglomeration. Our findings indicate that the hourly average O₃ increased by 3.41 μg m⁻³ yr⁻¹, with the trend characterized by two distinct phases (Phase I, 2015–2018; Phase II, 2019–2022). Interestingly, the increasing rate of O₃ concentration in Phase I (7.89 μg m⁻³) was notably higher than that in Phase II (2.89 μg m⁻³). The Random Forest (RF) model was employed to identify the key factors influencing O₃ concentrations during the two phases. The significant dropping of PM_2.5 in Phase I could be responsible for the O₃ increase. In Phase II, the reductions in nitrogen dioxide (NO₂) and unfavorable meteorological conditions were the major drivers of the continued increase in O₃. The Observation-Based Model (OBM) was developed to further explore the role of PM_2.5 in O₃ formation. Our results suggest that PM_2.5 can influence O₃ concentrations and the chemical sensitivity regime through heterogeneous reactions and changes in photolysis rates. In addition, the relatively high concentration of PM_2.5 in Xinxiang City in recent years underscores its significant role in O₃ formation. Future efforts should focus on the joint control of PM_2.5 and O₃ to improve air quality in the Central Plains urban agglomeration. Full article

With the continuous progress of UAV technology and the rapid development of mobile edge computing (MEC), the UAV-assisted MEC system has shown great application potential in special fields such as disaster rescue and emergency response. However, traditional deep reinforcement learning (DRL) decision-making methods suffer from limitations such as difficulty in balancing multiple objectives and training convergence when making mixed action space decisions for UAV path planning and task offloading. This article innovatively proposes a hybrid decision framework based on the improved Dynamic Adaptive Genetic Optimization Algorithm (DAGOA) and soft actor–critic with hierarchical action decomposition, an uncertainty-quantified critic ensemble, and adaptive entropy temperature, where DAGOA performs an effective search and optimization in discrete action space, while SAC can perform fine control and adjustment in continuous action space. By combining the above algorithms, the joint optimization of drone path planning and task offloading can be achieved, improving the overall performance of the system. The experimental results show that the framework offers significant advantages in improving system performance, reducing energy consumption, and enhancing task completion efficiency. When the system adopts a hybrid decision framework, the reward score increases by a maximum of 153.53% compared to pure deep reinforcement learning algorithms for decision-making. Moreover, it can achieve an average improvement of 61.09% on the basis of various reinforcement learning algorithms such as proposed SAC, proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), and twin delayed deep deterministic policy gradient (TD3). Full article

Multi-AUV cooperative hunting requires autonomous underwater vehicles (AUVs) to strategize the encirclement of evaders while navigating around obstacles and other AUVs. Despite the promise of multi-agent reinforcement learning (MARL) in continuous control problems, its low sample efficiency poses a challenge in unknown environments and complex control scenarios. To overcome these limitations, we present a Knowledge-Guided Reinforcement Learning (KG-RL) approach, which integrates an Artificial Potential Field (APF) to enhance sample efficiency and operational safety. Our methodology is bifurcated into pre-training and fine-tuning phases. During the pre-training phase, an APF is employed to generate a concise set of demonstration trajectories that provide agents with foundational knowledge. Subsequently, the fine-tuning phase leverages real-time APF knowledge to direct the learning process, encouraging agents to balance following demonstrated actions with seeking out more optimal solutions. We assess the efficacy of our method through extensive simulations across diverse tasks, demonstrating its ability to expedite the learning process and yield more strategic decision-making. Our approach achieves superior results compared to traditional MARL benchmarks, particularly in learning efficiency, decision quality, and overall performance. Full article

Over the last decades, bioprospecting of tropical corals has revealed numerous bioactive compounds with potential for biotechnological applications. However, this search involves sampling in natural reefs, and this is currently hampered by multiple ethical and technological constraints. Living coral displays, research laboratories, and biobanks currently offer an opportunity to continue to unravel coral chemodiversity, acting as “Noah’s Arks” that may continue to support the bioprospecting of molecules of interest. This issue is even more relevant if one considers that tropical coral reefs currently face unprecedent threats and irreversible losses that may impair the biodiscovery of molecules with potential for new products, processes, and services. Living coral displays provide controlled environments for studying corals and producing both known and new metabolites under varied conditions, and they are not prone to common bottlenecks associated with bioprospecting in natural coral reefs, such as loss of the source and replicability. Research laboratories may focus on a particular coral species or bioactive compound using corals that were cultured ex situ, although they may differ from wild conspecifics in metabolite production both in quantitative and qualitative terms. Biobanks collect and preserve coral specimens, tissues, cells, and/or information (e.g., genes, associated microorganisms), which offers a plethora of data to support the study of bioactive compounds’ mode of action without having to cope with issues related to access, standardization, and regulatory compliance. Bioprospecting in these settings faces several challenges and opportunities. On one hand, it is difficult to ensure the complexity of highly biodiverse ecosystems that shape the production and chemodiversity of corals. On the other hand, it is possible to maximize biomass production and fine tune the synthesis of metabolites of interest under highly controlled environments. Collaborative efforts are needed to overcome barriers and foster opportunities to fully harness the chemodiversity of tropical corals before in-depth knowledge of this pool of metabolites is irreversibly lost due to tropical coral reefs’ degradation. Full article

Vision and Language Navigation (VLN) is a task in which an agent needs to understand natural language instructions to reach the target location in a real-scene environment. To improve the model ability of long-horizon planning, emerging research focuses on extending the models with different types of memory structures, mainly including topological maps or a hidden state vector. However, the fixed-length hidden state vector is often insufficient to capture long-term temporal context. In comparison, topological maps have been shown to be beneficial for many robotic navigation tasks. Therefore, we focus on building a feasible and effective topological map representation and using it to improve the navigation performance and the generalization across seen and unseen environments. This paper presents a S elf-organizing Memory based on Adaptive Resonance Theory (SMART) module for incremental topological mapping and a framework for utilizing the SMART module to guide navigation. Based on fusion adaptive resonance theory networks, the SMART module can extract salient scenes from historical observations and build a topological map of the environmental layout. It provides a compact spatial representation and supports the discovery of novel shortcuts through inferences while being explainable in terms of cognitive science. Furthermore, given a language instruction and on top of the topological map, we propose a vision–language alignment framework for navigational decision-making. Notably, the framework utilizes three off-the-shelf pre-trained models to perform landmark extraction, node–landmark matching, and low-level controlling, without any fine-tuning on human-annotated datasets. We validate our approach using the Habitat simulator on VLN-CE tasks, which provides a photo-realistic environment for the embodied agent in continuous action space. The experimental results demonstrate that our approach achieves comparable performance to the supervised baseline. Full article

Accurate recognition of disabled persons’ behavioral intentions is the key to reconstructing hand function. Their intentions can be understood to some extent by electromyography (EMG), electroencephalogram (EEG), and arm movements, but they are not reliable enough to be generally accepted. In this paper, characteristics of foot contact force signals are investigated, and a method of expressing grasping intentions based on hallux (big toe) touch sense is proposed. First, force signals acquisition methods and devices are investigated and designed. By analyzing characteristics of signals in different areas of the foot, the hallux is selected. The peak number and other characteristic parameters are used to characterize signals, which can significantly express grasping intentions. Second, considering complex and fine tasks of the assistive hand, a posture control method is proposed. Based on this, many human-in-the-loop experiments are conducted using human–computer interaction methods. The results showed that people with hand disabilities could accurately express their grasping intentions through their toes, and could accurately grasp objects of different sizes, shapes, and hardness using their feet. The accuracy of the action completion for single-handed and double-handed disabled individuals was 99% and 98%, respectively. This proves that the method of using toe tactile sensation for assisting disabled individuals in hand control can help them complete daily fine motor activities. The method is easily acceptable in terms of reliability, unobtrusiveness, and aesthetics. Full article

Continuous action spaces impose a serious challenge for reinforcement learning agents. While several off-policy reinforcement learning algorithms provide a universal solution to continuous control problems, the real challenge lies in the fact that different actuators feature different response functions due to wear and tear (in mechanical systems) and fatigue (in biomechanical systems). In this paper, we propose enhancing the actor-critic reinforcement learning agents by parameterising the final layer in the actor network. This layer produces the actions to accommodate the behaviour discrepancy of different actuators under different load conditions during interaction with the environment. To achieve this, the actor is trained to learn the tuning parameter controlling the activation layer (e.g., Tanh and Sigmoid). The learned parameters are then used to create tailored activation functions for each actuator. We ran experiments on three OpenAI Gym environments, i.e., Pendulum-v0, LunarLanderContinuous-v2, and BipedalWalker-v2. Results showed an average of 23.15% and 33.80% increase in total episode reward of the LunarLanderContinuous-v2 and BipedalWalker-v2 environments, respectively. There was no apparent improvement in Pendulum-v0 environment but the proposed method produces a more stable actuation signal compared to the state-of-the-art method. The proposed method allows the reinforcement learning actor to produce more robust actions that accommodate the discrepancy in the actuators’ response functions. This is particularly useful for real life scenarios where actuators exhibit different response functions depending on the load and the interaction with the environment. This also simplifies the transfer learning problem by fine-tuning the parameterised activation layers instead of retraining the entire policy every time an actuator is replaced. Finally, the proposed method would allow better accommodation to biological actuators (e.g., muscles) in biomechanical systems. Full article