Search Results (22)

Event cameras provide microsecond-level temporal resolution, low latency, and high dynamic range, enabling robust perception under fast motion and challenging lighting conditions. Nevertheless, event streams are susceptible to background activity, thermal noise, and hot pixels. Their sparse and irregular patterns can corrupt event structures and degrade downstream tasks. We propose MSF, a multi-level spatiotemporal filtering framework that couples motion-compensated aggregation with neighborhood-level verification. In each temporal window, MSF estimates a constant 2D optical flow by maximizing a robust, density-normalized contrast objective on the image of warped events (IWE). We further incorporate polarity–gradient decorrelation to suppress mixed-polarity noise and an explicit peak-suppression regularizer to avoid hot-pixel-induced degeneracy. The motion parameters are optimized via coarse grid initialization followed by gradient-ascent refinement. Based on the estimated motion, MSF performs hierarchical event selection: central events are extracted from high-confidence aggregated regions, local events are recovered through joint spatial–temporal–directional–polarity consistency, and weak border events are identified using a density-normalized probabilistic support model that rewards support from reliable structures while penalizing self-clustering. Experiments on four public benchmarks (DVSNOISE20, DVSMOTION20, DVSCLEAN, and E-MLB) show that MSF consistently improves the Event Structural Ratio (ESR) and outperforms representative baselines across diverse motion regimes and severe low-light noise. Full article

This study examines risky decision-making in a free-ranging population of Balinese long-tailed macaques (Macaca fascicularis fascicularis), addressing gaps in research that have largely focused on captive primates and have rarely considered individual differences by age and sex. Thirty-three macaques of different age–sex classes were tested using a choice task contrasting a guaranteed small reward with a probabilistic larger reward. At the group level, macaques showed no preference for safe or risky options. However, substantial individual variation emerged: some individuals were risk-prone, others risk-averse, and many indifferent. Notably, age and sex interacted in shaping risk preferences. Among males, adults and juveniles were more risk-prone than younger adults, whereas among females, adults were more risk-prone than juveniles. Juveniles also displayed outcome-dependent flexibility, choosing the risky option more often after a previous successful risky choice, consistent with a win–stay strategy. Like in rodents, this pattern may reflect adaptive learning during developmental transitions. Importantly, the observed behavioral differences were not due to misunderstanding of the task, as macaques reliably chose the larger option when outcomes were visible. This pronounced individual variability in primate risk preferences underscore the importance of considering demographic factors when characterizing species-typical risk preferences. Full article

Emergency supplies allocation is a critical task in post-disaster response, as ineffective or delayed decisions can directly lead to increased human suffering and loss of life. In practice, emergency managers must make rapid allocation decisions over multiple periods under incomplete information and highly unpredictable demand, making robust and adaptive decision support essential. However, existing allocation approaches face several challenges: (1) Those traditional approaches rely heavily on predefined uncertainty sets or probabilistic models, and are inherently static, making them unsuitable for multi-period, dynamically allocation problems; and (2) while reinforcement learning (RL) technique is inherently suitable for dynamic decision-making, most existing RL-base approaches assume fixed demand, making them unable to cope with the non-stationary demand patterns seen in real disasters. To address these challenges, we first establish a multi-period and multi-objective emergency supplies allocation problem with demand uncertainty and then formulate it as a two-player zero-sum Markov game (TZMG). Demand uncertainty is modeled through an adversary rather than predefined uncertainty sets. We then propose RESA, a novel RL framework that uses adversarial training to learn robust allocation policies. In addition, RESA introduces a combinatorial action representation and reward clipping methods to handle high-dimensional allocations and nonlinear objectives. Building on RESA, we develop RESA_PPO by employing proximal policy optimization as its policy optimizer. Experiment results with realistic post-disaster data show that RESA_PPO achieves near-optimal performance, with an average gap of only 3.7% in terms of the objective value of the formulated problem, from the theoretical optimum derived by exact solvers. Moreover, RESA_PPO outperforms all baseline methods, including heuristic and standard RL methods, by at least 5.25% on average. Full article

This study aimed to explore psychosocial factors in a sample of Peruvian workers, examine their convergence with the PROPSIT model, and identify the emergence of new or idiosyncratic psychosocial dimensions. At the same time, the quality and efficiency of the categorization process were evaluated. n = 48 workers were contacted by a non-probabilistic sampling method and asked to fill out a form with open-ended questions that explored negative stressors and positive engaging factors. Some strategies were used to assess the quality and efficiency of the categorization process. The results showed that the quality, speed, and reliability of the categorization procedure were satisfactory, and several categories were aligned with the PROPSIT model and other literature, both in their negative aspects (workload and rhythm, working hours, shifts, etc.) and positive aspects (rewarding tasks, atmosphere of unity, etc.). The emerging new categories were confined to aspects of teamwork and conflict climate, as well as topics such as order, cleanliness, and recreation. These findings underline the need to adapt existing models and instruments to capture idiosyncratic aspects of the Peruvian work environment. In conclusion, this study validated an efficient mixed approach for categorizing psychosocial work factors in Peru, revealing both PROPSIT-aligned and novel context-specific categories, and highlighting the need for culturally adapted tools and broader validation. Full article

With the rapid expansion of digital knowledge platforms and intelligent information systems, organizations and communities are producing a vast number of unstructured knowledge data, including annotated corpora, technical diagrams, collaborative whiteboard content, and domain-specific multimedia archives. However, knowledge sharing across institutions is hindered by privacy risks, high communication overhead, and fragmented ownership of data. Federated learning promises to overcome these barriers by enabling collaborative model training without exchanging raw knowledge artifacts, but its success depends on motivating data holders to undertake the additional computational and communication costs. Most existing incentive schemes, which are based on non-cooperative game formulations, neglect unstructured interactions and communication efficiency, thereby limiting their applicability in knowledge-driven scenarios. To address these challenges, we introduce SC-NBTI, a smart contract and Nash bargaining-based incentive framework for federated learning in knowledge collaboration environments. We cast the reward allocation problem as a cooperative game, devise a heuristic algorithm to approximate the NP-hard Nash bargaining solution, and integrate a probabilistic gradient sparsification method to trim communication costs while safeguarding privacy. Experiments on the FMNIST image classification task show that SC-NBTI requires fewer training rounds while achieving 5.89% higher accuracy than the DRL-Incentive baseline. Full article

The warehouse picking process is one of the most critical components of logistics operations. Human–robot collaboration (HRC) is seen as an important trend in warehouse picking, as it combines the strengths of both humans and robots in the picking process. However, in current human–robot collaboration frameworks, there is a lack of effective communication between humans and robots, which results in inefficient task execution during the picking process. To address this, this paper considers trust as a communication bridge between humans and robots and proposes the Stackelberg trust-based human–robot collaboration framework for warehouse picking, aiming to achieve efficient and effective human–robot collaborative picking. In this framework, HRC with trust for warehouse picking is defined as the Partially Observable Stochastic Game (POSG) model. We model human fatigue with the logistic function and incorporate its impact on the efficiency reward function of the POSG. Based on the POSG model, belief space is used to assess human trust, and human strategies are formed. An iterative Stackelberg trust strategy generation (ISTSG) algorithm is designed to achieve the optimal long-term collaboration benefits between humans and robots, which is solved by the Bellman equation. The generated human–robot decision profile is formalized as a Partially Observable Markov Decision Process (POMDP), and the properties of human–robot collaboration are specified as PCTL (probabilistic computation tree logic) with rewards, such as efficiency, accuracy, trust, and human fatigue. The probabilistic model checker PRISM is exploited to verify and analyze the corresponding properties of the POMDP. We take the popular human–robot collaboration robot TORU as a case study. The experimental results show that our framework improves the efficiency of human–robot collaboration for warehouse picking and reduces worker fatigue while ensuring the required accuracy of human–robot collaboration. Full article

This paper studies value function transfer within reinforcement learning frameworks, focusing on tasks continuously assigned to an agent through a probabilistic distribution. Specifically, we focus on environments characterized by sparse rewards with a terminal goal. Initially, we propose and theoretically demonstrate that the distribution of the computed value function from such environments, whether in cases where the goals or the dynamics are changing across tasks, can be reformulated as the distribution of the number of steps to the goal generated by their optimal policies, which we name the expected optimal path length. To test our propositions, we hypothesize that the distribution of the expected optimal path lengths resulting from the task distribution is normal. This claim leads us to propose that if the distribution is normal, then the distribution of the value function follows a log-normal pattern. Leveraging this insight, we introduce “LogQInit” as a novel value function transfer method, based on the properties of log-normality. Finally, we run experiments on a scenario of goals and dynamics distributions, validate our proposition by providing an a dequate analysis of the results, and demonstrate that LogQInit outperforms existing methods of value function initialization, policy transfer, and reward shaping. Full article

The intelligent warehouse is a modern logistics management system that uses technologies like the Internet of Things, robots, and artificial intelligence to realize automated management and optimize warehousing operations. The multi-robot system (MRS) is an important carrier for implementing an intelligent warehouse, which completes various tasks in the warehouse through cooperation and coordination between robots. As an extension of reinforcement learning and a kind of swarm intelligence, MARL (multi-agent reinforcement learning) can effectively create the multi-robot systems in intelligent warehouses. However, MARL-based multi-robot systems in intelligent warehouses face serious safety issues, such as collisions, conflicts, and congestion. To deal with these issues, this paper proposes a safe MARL method based on runtime verification, i.e., an optimized safety policy-generation framework, for multi-robot systems in intelligent warehouses. The framework consists of three stages. In the first stage, a runtime model SCMG (safety-constrained Markov Game) is defined for the multi-robot system at runtime in the intelligent warehouse. In the second stage, rPATL (probabilistic alternating-time temporal logic with rewards) is used to express safety properties, and SCMG is cyclically verified and refined through runtime verification (RV) to ensure safety. This stage guarantees the safety of robots’ behaviors before training. In the third stage, the verified SCMG guides SCPO (safety-constrained policy optimization) to obtain an optimized safety policy for robots. Finally, a multi-robot warehouse (RWARE) scenario is used for experimental evaluation. The results show that the policy obtained by our framework is safer than existing frameworks and includes a certain degree of optimization. Full article

In unstructured environments, robots need to deal with a wide variety of objects with diverse shapes, and often, the instances of these objects are unknown. Traditional methods rely on training with large-scale labeled data, but in environments with continuous and high-dimensional state spaces, the data become sparse, leading to weak generalization ability of the trained models when transferred to real-world applications. To address this challenge, we present an innovative maximum entropy Deep Q-Network (ME-DQN), which leverages an attention mechanism. The framework solves complex and sparse reward tasks through probabilistic reasoning while eliminating the trouble of adjusting hyper-parameters. This approach aims to merge the robust feature extraction capabilities of Fully Convolutional Networks (FCNs) with the efficient feature selection of the attention mechanism across diverse task scenarios. By integrating an advantage function with the reasoning and decision-making of deep reinforcement learning, ME-DQN propels the frontier of robotic grasping and expands the boundaries of intelligent perception and grasping decision-making in unstructured environments. Our simulations demonstrate a remarkable grasping success rate of 91.6%, while maintaining excellent generalization performance in the real world. Full article

A central question in behavioural neuroscience is how different rewards modulate learning. While the role of monetary rewards is well-studied in decision-making research, the influence of abstract rewards like music remains poorly understood. This study investigated the dissociable effects of these two reward types on decision making. Forty participants completed two decision-making tasks, each characterised by probabilistic associations between stimuli and rewards, with probabilities changing over time to reflect environmental volatility. In each task, choices were reinforced either by monetary outcomes (win/lose) or by the endings of musical melodies (consonant/dissonant). We applied the Hierarchical Gaussian Filter, a validated hierarchical Bayesian framework, to model learning under these two conditions. Bayesian statistics provided evidence for similar learning patterns across both reward types, suggesting individuals’ similar adaptability. However, within the musical task, individual preferences for consonance over dissonance explained some aspects of learning. Specifically, correlation analyses indicated that participants more tolerant of dissonance behaved more stochastically in their belief-to-response mappings and were less likely to choose the response associated with the current prediction for a consonant ending, driven by higher volatility estimates. By contrast, participants averse to dissonance showed increased tonic volatility, leading to larger updates in reward tendency beliefs. Full article

Research in reinforcement learning indicates that animals respond differently to positive and negative reward prediction errors, which can be calculated by assuming learning rate bias. Many studies have shown that humans and other animals have learning rate bias during learning, but it is unclear whether and how the bias changes throughout the entire learning process. Here, we recorded the behavior data and the local field potentials (LFPs) in the striatum of five pigeons performing a probabilistic learning task. Reinforcement learning models with and without learning rate biases were used to dynamically fit the pigeons’ choice behavior and estimate the option values. Furthemore, the correlation between the striatal LFPs power and the model-estimated option values was explored. We found that the pigeons’ learning rate bias shifted from negative to positive during the learning process, and the striatal Gamma (31 to 80 Hz) power correlated with the option values modulated by dynamic learning rate bias. In conclusion, our results support the hypothesis that pigeons employ a dynamic learning strategy in the learning process from both behavioral and neural aspects, providing valuable insights into reinforcement learning mechanisms of non-human animals. Full article

Stress modulates feedback-based learning, a process that has been implicated in declining mental function in aging and mental disorders. While acute physical and psychosocial stressors have been used interchangeably in studies on feedback-based learning, the two types of stressors involve distinct physiological and psychological processes. Whether the two types of stressors differentially influence feedback processing remains unclear. The present study compared the effects of physical and psychosocial stressors on feedback-based learning. Ninety-six subjects (M_age = 19.11 years; 50 female) completed either a cold pressor task (CPT) or mental arithmetic task (MAT), as the physical or psychosocial stressor, while electrocardiography and blood pressure were measured to assess cardiovascular stress reactivity (CVR). Self-ratings on the emotional valence of the stressors were also obtained. A probabilistic learning task was given prior to and after the stressors. Accuracy in selecting positive (Go accuracy) and avoiding negative stimuli (No-go accuracy) were recorded as learning outcomes. Repeated measures ANOVA and multiple regressions were used to compare the effects of two stressors and examine the effects of CVR and valence on the learning outcomes. The results showed that although the effects of CPT and MAT on feedback processing were not different, CVR and valence influenced Go and No-go accuracy, respectively. The results suggest that stress-modulated feedback-based learning involves multiple pathways and underscore the link between CVR and reward sensitivity. The findings have clinical implications and may contribute to a better understanding of human behavioral systems. Full article

The probabilistic reward task (PRT) has identified reward learning impairments in those with major depressive disorder (MDD), as well as anhedonia-specific reward learning impairments. However, attempts to validate the anhedonia-specific impairments have produced inconsistent findings. Thus, we seek to determine whether the Reward Behavior Disengagement (RBD), our proposed economic augmentation of PRT, differs between MDD participants and controls, and whether there is a level at which RBD is high enough for depressed participants to be considered objectively disengaged. Data were gathered as part of the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, a double-blind, placebo-controlled clinical trial of antidepressant response. Participants included 195 individuals with moderate to severe MDD (Quick Inventory of Depressive Symptomatology (QIDS–SR) score ≥ 15), not in treatment for depression, and with complete PRT data. Healthy controls (n = 40) had no history of psychiatric illness, a QIDS–SR score < 8, and complete PRT data. Participants with MDD were treated with sertraline or placebo for 8 weeks (stage I of the EMBARC trial). RBD was applied to PRT data using discriminant analysis, and classified MDD participants as reward task engaged (n = 137) or reward task disengaged (n = 58), relative to controls. Reward task engaged/disengaged groups were compared on sociodemographic features, reward–behavior, and sertraline/placebo response (Hamilton Depression Rating Scale scores). Reward task disengaged MDD participants responded only to sertraline, whereas those who were reward task engaged responded to sertraline and placebo (F(1293) = 4.33, p = 0.038). Reward task engaged/disengaged groups did not differ otherwise. RBD was predictive of reward impairment in depressed patients and may have clinical utility in identifying patients who will benefit from antidepressants. Full article

The task of tracking the pose of an object with a known geometry from point cloud measurements arises in robot perception. It calls for a solution that is both accurate and robust, and can be computed at a rate that aligns with the needs of a control system that might make decisions based on it. The Iterative Closest Point (ICP) algorithm is widely used for this purpose, but it is susceptible to failure in practical scenarios. We present a robust and efficient solution for pose-from-point cloud estimation called the Pose Lookup Method (PLuM). PLuM is a probabilistic reward-based objective function that is resilient to measurement uncertainty and clutter. Efficiency is achieved through the use of lookup tables, which substitute complex geometric operations such as raycasting used in earlier solutions. Our results show millimetre accuracy and fast pose estimation in benchmark tests using triangulated geometry models, outperforming state-of-the-art ICP-based methods. These results are extended to field robotics applications, resulting in real-time haul truck pose estimation. By utilising point clouds from a LiDAR fixed to a rope shovel, the PLuM algorithm tracks a haul truck effectively throughout the excavation load cycle at a rate of 20 Hz, matching the sensor frame rate. PLuM is straightforward to implement and provides dependable and timely solutions in demanding environments. Full article

Anhedonia, a central depression symptom, is associated with impairments in reward processing. However, it is not well understood which sub-components of reward processing (anticipation, motivation, consummation, and learning) are impaired in association with anhedonia in depression. In particular, it is unclear how learning about different rewards and the effort needed to obtain them might be associated with anhedonia and depression symptoms. Therefore, we examined learning in young people (N = 132, mean age 20, range 17–25 yrs.) with a range of depression and anhedonia symptoms using a probabilistic instrumental learning task. The task required participants to learn which options to choose to maximize their reward outcomes across three conditions (chocolate taste, puppy images, or money) and to minimize the physical effort required to obtain the rewards. Additionally, we collected questionnaire measures of anticipatory and consummatory anhedonia, as well as subjective reports of “liking”, “wanting” and “willingness to exert effort” for the rewards used in the task. We found that as anticipatory anhedonia increased, subjective liking and wanting of rewards decreased. Moreover, higher anticipatory anhedonia was significantly associated with lower reward learning accuracy, and participants demonstrated significantly higher reward learning than effort learning accuracy. To our knowledge, this is the first study observing an association of anhedonia with reward liking, wanting, and learning when reward and effort learning are measured simultaneously. Our findings suggest an impaired ability to learn from rewarding outcomes could contribute to anhedonia in young people. Future longitudinal research is needed to confirm this and reveal the specific aspects of reward learning that predict anhedonia. These aspects could then be targeted by novel anhedonia interventions. Full article