Search Results (36)

This study presents community-specific modeling approaches for simulating power injection from photovoltaic and wind energy systems in a German metropolitan region. Developed within the EMN_SIM project and based on openly accessible datasets, the methods are broadly transferable across Germany. For PV, a cluster-based model groups systems by geographic and technical characteristics, using real weather data to reduce computational effort. Validation against measured specific yields shows strong agreement, confirming energetic accuracy. The wind model operates on a per-turbine basis, integrating technical specifications, land use, and high-resolution wind data. Energetic validation indicates good consistency with Bavarian reference values, while power-based comparisons with selected turbines show reasonable correlation, subject to expected limitations in wind data resolution. The resulting high-resolution generation profiles reveal spatial and temporal patterns valuable for grid planning and targeted policy design. While further validation with additional measurement data could enhance model precision, the current results already offer a robust foundation for urban energy system analyses and future grid integration studies. Full article

In recent years, with the rapid development of science and technology and the substantial improvement of computing power, various deep learning research topics have been promoted. However, existing autonomous driving technologies still face significant challenges in achieving robust lane-keeping and navigation performance, especially when transferring learned models from simulation to real-world environments due to environmental complexity and domain gaps. Many fields such as computer vision, natural language processing, and medical imaging have also accelerated their development due to the emergence of this wave, and the field of self-driving cars is no exception. The trend of self-driving cars is unstoppable. Many technology companies and automobile manufacturers have invested a lot of resources in the research and development of self-driving technology. With the emergence of different levels of self-driving cars, most car manufacturers have already reached the L2 level of self-driving classification standards and are moving towards L3 and L4 levels. This study applies deep reinforcement learning (DRL) to train autonomous vehicles with lane-keeping and navigation capabilities. Through simulation training and Sim2Real strategies, including domain randomization and CycleGAN, the trained models are evaluated in real-world environments to validate performance. The results demonstrate the feasibility of DRL-based autonomous driving and highlight the challenges in transferring models from simulation to reality. Full article

This paper proposes a transition control strategy for a rotary double-inverted pendulum (RDIP) system using a sim-to-real reinforcement learning (RL) controller, built upon mathematical modeling and parameter estimation. High-resolution sensor data are used to estimate key physical parameters, ensuring model fidelity for simulation. The resulting mathematical model serves as the training environment in which the RL agent learns to perform transitions between various initial conditions and target equilibrium configurations. The training process adopts the Truncated Quantile Critics (TQC) algorithm, with a reward function specifically designed to reflect the nonlinear characteristics of the system. The learned policy is directly deployed on physical hardware without additional tuning or calibration, and the TQC-based controller successfully achieves all four equilibrium transitions. Furthermore, the controller exhibits robust recovery properties under external disturbances, demonstrating its effectiveness as a reliable sim-to-real control approach for high-dimensional nonlinear systems. Full article

Path planning in multi-agent UAV swarms is a crucial issue that involves avoiding collisions in dynamic, obstacle-filled environments while consuming the least amount of time and energy possible. This work comprehensively evaluates reinforcement learning (RL) algorithms for multi-agent UAV path planning in 2D and 3D simulated environments. First, we develop a 2D simulation setup using Python in which UAVs (quadcopters), represented as points in space, navigate toward their respective targets while avoiding static obstacles and inter-agent collisions. In the second phase, we transition this comparison to a physics-based 3D simulation, incorporating realistic UAV (fixed wing) dynamics and checkpoint-based navigation. We compared five algorithms, namely, Proximal Policy Optimization (PPO), Soft Actor–Critic (SAC), Deep Deterministic Policy Gradient (DDPG), Trust Region Policy Optimization (TRPO), and Multi–Agent DDPG (MADDPG), in various scenarios. Our findings reveal significant performance differences between the algorithms across multiple dimensions. DDPG consistently demonstrated superior reward optimization and collision avoidance performance, while PPO and MADDPG excelled in the execution time required to reach the goal. Furthermore, our findings reveal how algorithms perform while transitioning from a simplistic 2D setup to a realistic 3D physics-based environment, which is essential for performing sim-to-real transfer. This work provides valuable insights into the suitability of several reinforcement learning (RL) algorithms for developing autonomous systems and UAV swarm navigation. Full article

Deep reinforcement learning (DRL), a vital branch of artificial intelligence, has shown great promise in mobile robot navigation within dynamic environments. However, existing studies mainly focus on simplified dynamic scenarios or the modeling of static environments, which results in trained models lacking sufficient generalization and adaptability when faced with real-world dynamic environments, particularly in handling complex task variations, dynamic obstacle interference, and multimodal data fusion. Addressing these gaps is essential for enhancing its real-time performance and versatility. Through a comparative analysis of classical DRL algorithms, this study highlights their advantages and limitations in handling real-time navigation tasks under dynamic environmental conditions. In particular, the paper systematically examines value-based, policy-based, and hybrid-based DRL methods, discussing their applicability to different navigation challenges. Additionally, by reviewing recent studies from 2021 to 2024, it identifies key trends in DRL-based navigation, revealing a strong focus on indoor environments while outdoor navigation and multi-robot collaboration remain underexplored. The analysis also highlights challenges in real-world deployment, particularly in sim-to-real transfer and sensor fusion. Based on these findings, this paper outlines future directions to enhance real-time adaptability, multimodal perception, and collaborative learning frameworks, providing theoretical and technical insights for advancing DRL in dynamic environments. Full article

Marine mammal monitoring, a growing field of research, is critical to cetacean conservation. Traditional ‘tagging’ attaches sensors such as GPS to such animals, though these are intrusive and susceptible to infection and, ultimately, death. A less intrusive approach exploits UUV commanded by a human operator above ground. The development of AI for autonomous underwater vehicle navigation models training environments in simulation, providing visual and physical fidelity suitable for sim-to-real transfer. Previous solutions, including UVMS and L2D, provide only satisfactory results, due to poor environment generalisation while sensors including sonar create environmental disturbances. Though rich in features, image data suffer from high dimensionality, providing a state space too great for many machine learning tasks. Underwater environments, susceptible to image noise, further complicate this issue. We propose SWiMM_2.0, coupling a Unity simulation modelling of a BLUEROV UUV with a DRL backend. A pre-processing step exploits a state-of-the-art CMVAE, reducing dimensionality while minimising data loss. Sim-to-real generalisation is validated by prior research. Custom behaviour metrics, unbiased to the naked eye and unprecedented in current ROV simulators, link our objectives ensuring successful ROV behaviour while tracking targets. Our experiments show that SAC maximises the former, achieving near-perfect behaviour while exploiting image data alone. Full article

Typically, developing new digital agriculture technologies requires substantial on-site resources and data. However, the crop’s growth cycle provides only limited time windows for experiments and equipment validation. This study presents a photorealistic digital twin of a commercial-scale strawberry farm, coupled with a simulated ground vehicle, to address these constraints by generating high-fidelity synthetic RGB and LiDAR data. These data enable the rapid development and evaluation of a deep learning-based machine vision pipeline for fruit detection and sizing without continuously relying on real-field access. Traditional simulators often lack visual realism, leading many studies to mix real images or adopt domain adaptation methods to address the reality gap. In contrast, this work relies solely on photorealistic simulation outputs for training, eliminating the need for real images or specialized adaptation approaches. After training exclusively on images captured in the virtual environment, the model was tested on a commercial-scale strawberry farm using a physical ground vehicle. Two separate trials with field images resulted in F1-scores of 0.92 and 0.81 for detection and a sizing error of 1.4 mm (R² = 0.92) when comparing image-derived diameters against caliper measurements. These findings indicate that a digital twin-driven sim2real transfer can offer substantial time and cost savings by refining crucial tasks such as stereo sensor calibration and machine learning model development before extensive real-field deployments. In addition, the study examined geometric accuracy and visual fidelity through systematic comparisons of LiDAR and RGB sensor outputs from the virtual and real farms. Results demonstrated close alignment in both topography and textural details, validating the digital twin’s ability to replicate intricate field characteristics, including raised bed geometry and strawberry plant distribution. The techniques developed and validated in this strawberry project have broad applicability across agricultural commodities, particularly for fruit and vegetable production systems. This study demonstrates that integrating digital twins with simulation tools can significantly reduce the need for resource-intensive field data collection while accelerating the development and refinement of agricultural robotics algorithms and hardware. Full article

The parallel dual-slider telescopic leg bipedal robot (L04) is characterized by its simple structure and low leg rotational inertia, which contribute to its walking efficiency. However, end-to-end methods often overlook the robot’s physical structure, leading to difficulties in maintaining the parallel alignment of the dual sliders, which in turn compromises walking stability. One potential solution to this issue involves utilizing imitation learning to replicate human motion data. However, the dual telescopic leg structure of the L04 robot makes it difficult to perform motion retargeting of human motion data. To enable L04 walking, we design a method that integrates prior feedforward with reinforcement learning (PFRL), specifically tailored for the parallel dual-slider structure. We utilize prior knowledge as a feedforward action to compensate for system nonlinearities; meanwhile, the feedback action generated by the policy network adaptively regulates dynamic balance and, combined with the feedforward action, jointly controls the robot’s walking. PFRL enforces constraints within the motion space to mitigate the chaotic behavior of the parallel dual sliders. Experimental results show that our method successfully achieves sim2real transfer on a real bipedal robot without the need for domain randomization techniques or intricate reward functions. L04 achieves omnidirectional walking with minimal energy consumption and exhibits robustness against external disturbances. Full article

This study presents a sim2real reinforcement learning-based controller for transition control in a double-inverted pendulum system, addressing the limitations of traditional control methods that rely on precomputed trajectories and lack adaptability to strong external disturbances. By introducing the novel concept of ‘transition control’, this research expands the scope of inverted pendulum studies to tackle the challenging task of navigating between multiple equilibrium points. To overcome the reality gap—a persistent challenge in sim2real transfer—a hardware-centered approach was employed, aligning the physical system’s mechanical design with high-fidelity dynamic equations derived from the Euler–Lagrange equation. This design eliminates the need for software-based corrections, ensuring consistent and robust system performance across simulated and real-world environments. Experimental validation demonstrates the controller’s ability to reliably execute all 12 transition scenarios within the double-inverted pendulum system. Additionally, it exhibits recovery characteristics, enabling the system to stabilize and return to equilibrium point even under severe disturbances. Full article

Precision agriculture seeks to optimize crop yields while minimizing resource use. A key challenge is achieving uniform pesticide spraying to prevent crop damage and environmental contamination. Water-sensitive paper (WSP) is a common tool used for assessing spray quality, as it visually registers droplet impacts through color change. This work introduces a smartphone-based solution for capturing WSP images within vegetation, offering a tool for farmers to assess spray quality in real-world conditions. To achieve this, two approaches were explored: classical computer vision techniques and machine learning (ML) models (YOLOv8, Mask-RCNN, and Cellpose). Addressing the challenges of limited real-world data and the complexity of manual annotation, a programmatically generated synthetic dataset was employed to enable sim-to-real transfer learning. For the task of WSP segmentation within vegetation, YOLOv8 achieved an average Intersection over Union of 97.76%. In the droplet detection task, which involves identifying individual droplets on WSP, Cellpose achieved the highest precision of 96.18%, in the presence of overlapping droplets. While classical computer vision techniques provided a reliable baseline, they struggled with complex cases. Additionally, ML models, particularly Cellpose, demonstrated accurate droplet detection even without fine-tuning. Full article

In the realm of autonomous driving, motion control generally involves two critical aspects—path following and stability control—and an inevitable mutual interference exists between them under extreme conditions. To tackle this challenge, this study proposes a collaborative approach of path tracking and stability control. When designing the path tracking control module, the effects of vertical tire load transfer and road surface adhesion coefficients on tire force calculations were taken into account to mitigate vehicle dynamics model mismatch. Leveraging the receding horizon optimization characteristic of nonlinear model predictive control (NMPC), a cascaded discretization approach was utilized to realize a balance between precision and real-time performance in numerical solutions. Then, a stability controller, which employs rear wheel steering, was designed to prevent excessive increases in the vehicle’s sideslip angle, thereby ensuring the vehicle’s lateral stability. The effectiveness of the proposed strategy is validated through CarSim 8.0/Simulink cosimulation. The outcomes demonstrate that the stability controller significantly enhances vehicle stability under high-speed and low-adhesion conditions. On the premise of stability, the proposed path tracking controller has exhibited significant enhancements in real-time performance, without compromising the accuracy of path tracking. Full article

Robotic systems are crucial in modern manufacturing. Complex assembly tasks require the collaboration of multiple robots. Their orchestration is challenging due to tight tolerances and precision requirements. In this work, we set up two Franka Panda robots performing a peg-in-hole insertion task of 1 mm clearance. We structure the control system hierarchically, planning the robots’ feedback-based trajectories with a central policy trained with reinforcement learning. These trajectories are executed by a low-level impedance controller on each robot. To enhance training convergence, we use reverse curriculum learning, novel for such a two-armed control task, iteratively structured with a minimum requirements and fine-tuning phase. We incorporate domain randomization, varying initial joint configurations of the task for generalization of the applicability. After training, we test the system in a simulation, discovering the impact of curriculum parameters on the emerging process time and its variance. Finally, we transfer the trained model to the real-world, resulting in a small decrease in task duration. Comparing our approach to classical path planning and control shows a decrease in process time, but higher robustness towards calibration errors. Full article

This study presents a cost-effective IoT-based Supervisory Control and Data Acquisition system for the real-time monitoring and control of photovoltaic systems in a rural Pakistani community. The system utilizes the Blynk platform with Arduino Nano, GSM SIM800L, and ESP-32 microcontrollers. The key components include a ZMPT101B voltage sensor, ACS712 current sensors, and a Maximum Power Point Tracking module for optimizing power output. The system operates over both Global System for Mobile Communications and Wi-Fi networks, employing universal asynchronous receiver–transmitter serial communication and using the transmission control protocol/Internet protocol and hypertext transfer protocol for data exchange. Testing showed that the system consumes only 3.462 W of power, making it highly efficient. With an implementation cost of CAD 35.52, it offers an affordable solution for rural areas. The system achieved an average data transmission latency of less than 2 s over Wi-Fi and less than 5 s over GSM, ensuring timely data updates and control. The Blynk 2.0 app provides data retention capabilities, allowing users to access historical data for performance analysis and optimization. This open-source SCADA system demonstrates significant potential for improving efficiency and user engagement in renewable energy management, offering a scalable solution for global applications. Full article

Target recognition and fine-grained ship classification in remote sensing face challenges of high inter-class similarity and sample scarcity. A transfer fusion-based ship image harmonization algorithm is proposed to overcome these challenges. This algorithm designs a feature transfer fusion strategy based on the combination of a region-aware instantiation and attention mechanism. Adversarial learning is implemented through an image harmony generator and discriminator module to generate realistic remote sensing ship harmony images. Furthermore, the domain encoder and domain discriminator modules are responsible for extracting feature representations of the foreground and background, and further align the ship foreground with remote sensing ocean background features through feature discrimination. Compared with other advanced image conversion techniques, our algorithm delivers more realistic visuals, improving classification accuracy for six ship types by 3% and twelve types by 2.94%, outperforming Sim2RealNet. Finally, a mixed dataset containing data augmentation and harmonizing samples and real data was proposed for the fine-grained classification task of remote sensing ships. Evaluation experiments were conducted on eight typical fine-grained classification algorithms, and the accuracy of the fine-grained classification for all categories of ships was analyzed. The experimental results show that the mixed dataset proposed in this paper effectively alleviates the long-tail problem in real datasets, and the proposed remote sensing ship data augmentation framework performs better than state-of-the-art data augmentation methods in fine-grained ship classification tasks. Full article

This paper explores the application of Deep Reinforcement Learning (DRL) and Sim2Real strategies to enhance the autonomy of beach-cleaning robots. Experiments demonstrate that DRL agents, initially refined in simulations, effectively transfer their navigation skills to real-world scenarios, achieving precise and efficient operation in complex natural environments. This method provides a scalable and effective solution for beach conservation, establishing a significant precedent for the use of autonomous robots in environmental management. The key advancements include the ability of robots to adhere to predefined routes and dynamically avoid obstacles. Additionally, a newly developed platform validates the Sim2Real strategy, proving its capability to bridge the gap between simulated training and practical application, thus offering a robust methodology for addressing real-life environmental challenges. Full article