Search Results (2)

The traditional Model-Based Reinforcement Learning (MBRL) algorithm has high computational cost, poor convergence, and poor performance in robot spatial cognition and navigation tasks, and it cannot fully explain the ability of animals to quickly adapt to environmental changes and learn a variety of complex tasks. Studies have shown that vicarious trial and error (VTE) and the hippocampus forward prediction mechanism in rats and other mammals can be used as key components of action selection in MBRL to support “goal-oriented” behavior. Therefore, we propose an improved Dyna-Q algorithm inspired by the forward prediction mechanism of the hippocampus to solve the above problems and tackle the exploration–exploitation dilemma of Reinforcement Learning (RL). This algorithm alternately presents the potential path in the future for mobile robots and dynamically adjusts the sweep length according to the decision certainty, so as to determine action selection. We test the performance of the algorithm in a two-dimensional maze environment with static and dynamic obstacles, respectively. Compared with classic RL algorithms like State-Action-Reward-State-Action (SARSA) and Dyna-Q, the algorithm can speed up spatial cognition and improve the global search ability of path planning. In addition, our method reflects key features of how the brain organizes MBRL to effectively solve difficult tasks such as navigation, and it provides a new idea for spatial cognitive tasks from a biological perspective. Full article

Central nervous system drug discovery and development is hindered by the impermeable nature of the blood–brain barrier. Pharmacokinetic modeling can provide a novel approach to estimate CNS drug exposure; however, existing models do not predict temporal drug concentrations in distinct brain regions. A rat CNS physiologically based pharmacokinetic (PBPK) model was developed, incorporating brain compartments for the frontal cortex (FC), hippocampus (HC), “rest-of-brain” (ROB), and cerebrospinal fluid (CSF). Model predictions of FC and HC C_max, t_max and AUC were within 2-fold of that reported for carbamazepine and phenytoin. The inclusion of a 30% coefficient of variation on regional brain tissue volumes, to assess the uncertainty of regional brain compartments volumes on predicted concentrations, resulted in a minimal level of sensitivity of model predictions. This model was subsequently extended to predict human brain morphine concentrations, and predicted a ROB C_max of 21.7 ± 6.41 ng/mL when compared to “better” (10.1 ng/mL) or “worse” (29.8 ng/mL) brain tissue regions with a FC C_max of 62.12 ± 17.32 ng/mL and a HC C_max of 182.2 ± 51.2 ng/mL. These results indicate that this simplified regional brain PBPK model is useful for forward prediction approaches in humans for estimating regional brain drug concentrations. Full article