Search Results (39)

As the “visual perception hub” of unmanned aerial vehicles (UAVs), electro-optical (EO) pods play an increasingly critical role in tasks such as intelligence gathering, situational awareness, target tracking, and localization. With the expanding scope and depth of UAV applications, higher demands are placed on the precision and adaptability of installation error calibration techniques for EO pods. Current mainstream calibration methods typically require specialized procedures under constrained conditions, while few approaches integrate existing UAV system capabilities and mission requirements, which leads to cumbersome, time-consuming processes and suboptimal alignment between calibration outcomes and task objectives. This paper proposes an online calibration method for UAV EO pod installation errors based on target tracking, which can rapidly compute the optimal closed-form solution for installation errors by leveraging UAV tracking missions. First, an observation equation for pod installation errors is established using tracking results. Second, multi-temporal observations are combined to model the calibration problem as an optimal rotation matrix estimation task, and then the optimal closed-form solution for installation errors is derived. Concurrently, a statistics-based approximate calibration method is introduced specifically for tracking missions. Furthermore, an online calibration system compatible with diverse UAV platforms is designed, along with different rapid calibration schemes for emergency response scenarios, fully incorporating existing system capabilities and mission needs. Finally, a fixed-wing UAV experimental platform is developed, with calibration tests conducted under various flight regimes. Experimental results validate the feasibility and robustness of the proposed methodology. Full article

Maintaining high-precision line-of-sight pointing in airborne electro-optical gimbals remains a significant challenge due to the simultaneous presence of heterogeneous disturbances and strict mechanical structural constraints within complex dynamic conditions. Traditional anti-disturbance methods often struggle to provide fine-grained compensation for multi-source uncertainties where low-frequency lumped disturbances (e.g., friction and unbalanced torques) and high-frequency harmonic vibrations (e.g., engine-induced vibrations and aerodynamic gusts) are intricately coupled. To address these challenges, this paper proposes a refined disturbance separation-based composite control scheme. First, a deep-coupled aircraft–gimbal dynamics model is constructed to reveal the spectral separation characteristics of multi-source disturbances under the “moving base” effect. Second, a Refined Disturbance Observer architecture is developed by coupling an Extended State Observer with a Harmonic Disturbance Observer, enabling the decoupled separation and precise estimation of heterogeneous disturbances based on their spectral characteristics. Furthermore, a finite-time composite controller incorporating a Barrier Lyapunov Function is designed to guarantee that the system output strictly adheres to inherent mechanical structural boundaries. Numerical simulations confirm high-precision tracking and strict constraint satisfaction of the scheme. Full article

This paper provides an overview of the three-year effort to design and implement a prototypical sense-and-avoid (SAA) system based on a multisensory architecture leveraging data fusion between optical and radar sensors. The work was carried out within the context of the Italian research project named TERSA (electrical and radar technologies for remotely piloted aircraft systems) undertaken by the University of Pisa in collaboration with its industrial partners, aimed at the design and development of a series of innovative technologies for remotely piloted aircraft systems of small scale (MTOW < 25 Kgf). The system leverages advanced computer vision algorithms and an extended Kalman filter to enhance obstacle detection and tracking capabilities. The “Sense” module processes environmental data through a radar and an electro-optical sensor, while the “Avoid” module utilizes efficient geometric algorithms for collision prediction and evasive maneuver computation. A novel hardware-in-the-loop (HIL) simulation environment was developed and used for validation, enabling the evaluation of closed-loop real-time interaction between the “Sense” and “Avoid” subsystems. Extensive numerical simulations and a flight test campaign demonstrate the system’s effectiveness in real-time detection and the avoidance of non-cooperative obstacles, ensuring compliance with UAV aero mechanical and safety constraints in terms of minimum separation requirements. The novelty of this research lies in (1) the design of an innovative and efficient visual processing pipeline tailored for SWaP-constrained mini-UAVs, (2) the formulation an EKF-based data fusion strategy integrating optical data with a custom-built Doppler radar, and (3) the development of a unique HIL simulation environment with realistic scenery generation for comprehensive system evaluation. The findings underscore the potential for deploying such advanced SAA systems in tactical UAV operations, significantly contributing to the safety of flight in non-segregated airspaces Full article

In order to contribute to the operation of unmanned aerial vehicles (UAVs) according to visual flight rules (VFR), this article proposes a monocular approach for cloud detection using an electro-optical sensor. Cloud avoidance is motivated by several factors, including improving visibility for collision prevention and reducing the risks of icing and turbulence. The described workflow is based on parallelized detection, tracking and triangulation of features with prior segmentation of clouds in the image. As output, the system generates a cloud occupancy grid of the aircraft’s vicinity, which can be used for cloud avoidance calculations afterwards. The proposed methodology was tested in simulation and flight experiments. With the aim of developing cloud segmentation methods, datasets were created, one of which was made publicly available and features 5488 labeled, augmented cloud images from a real flight experiment. The trained segmentation models based on the YOLOv8 framework are able to separate clouds from the background even under challenging environmental conditions. For a performance analysis of the subsequent cloud position estimation stage, calculated and actual cloud positions are compared and feature evaluation metrics are applied. The investigations demonstrate the functionality of the approach, even if challenges become apparent under real flight conditions. Full article

Given the recent proliferation of Unmanned Aerial Systems (UASs) and the consequent importance of counter-UASs, this project aims to perform the detection and tracking of small non-cooperative UASs using Electro-optical (EO) and Infrared (IR) sensors. Two data integration techniques, at the decision and pixel levels, are compared with the use of each sensor independently to evaluate the system robustness in different operational conditions. The data are submitted to a YOLOv7 detector merged with a ByteTrack tracker. For training and validation, additional efforts are made towards creating datasets of spatially and temporally aligned EO and IR annotated Unmanned Aerial Vehicle (UAV) frames and videos. These consist of the acquisition of real data captured from a workstation on the ground, followed by image calibration, image alignment, the application of bias-removal techniques, and data augmentation methods to artificially create images. The performance of the detector across datasets shows an average precision of 88.4%, recall of 85.4%, and mAP@0.5 of 88.5%. Tests conducted on the decision-level fusion architecture demonstrate notable gains in recall and precision, although at the expense of lower frame rates. Precision, recall, and frame rate are not improved by the pixel-level fusion design. Full article

We propose a new free-space optical (FSO) communication system for moving platform tracking, which can achieve high precision aiming and tracking. Our prototype system consists of three parts. As a coarse sighting structure, the electro-optical pod module is used for target searching and coarse sighting in the initial stage. As a precise aiming structure, the precise targeting loads module located inside the electro-optical pod module uses miniaturized tubular folding optical path technology for high-precision alignment and tracking. The bottom module of the system is used for communication. In the tracking process, the control unit uses spot offset collected by CCD to perform decoupling calculation and then compensates the offset by swinging tracking and aiming structure. We did track experiments on a mobile platform. The experiment successfully tracked a moving target at 100 m distance, and the tracking error was less than 1 mrad. The proposed system can provide stable communication links between the mobile platforms. Full article

This study presents a novel approach to address the autonomous stable tracking issue in electro-optical theodolite operating in closed-loop mode. The proposed methodology includes a multi-sensor adaptive weighted fusion algorithm and a fusion tracking algorithm based on a three-state transition model. A refined recursive formula for error covariance estimation is developed by integrating attenuation factors and least squares extrapolation. This formula is employed to formulate a multi-sensor weighted fusion algorithm that utilizes error covariance estimation. By assigning weighted coefficients to calculate the residual of the newly introduced error term and defining the sensor’s unique states based on these coefficients, a fusion tracking algorithm grounded on the three-state transition model is introduced. In cases of interference or sensor failure, the algorithm either computes the weighted fusion value of the multi-sensor measurement or triggers autonomous sensor switching to ensure the autonomous and stable measurement of the theodolite. Experimental results indicate that when a specific sensor is affected by interference or the off-target amount cannot be extracted, the algorithm can swiftly switch to an alternative sensor. This capability facilitates the precise and consistent generation of data, thereby ensuring the stable operation of the tracking system. Furthermore, the algorithm demonstrates robustness across various measurement scenarios. Full article

In this paper, we propose a novel equivalent composite control architecture for electro-optical equipment. The improved tracking performance and loss of robustness caused by this structure have a clear relationship with $a_2$, the time coefficient of the compensation circuit. The compensation circuit can make the speed quality factor and the acceleration quality factor of the system infinite, and the jerk quality factor can be expanded to $1/a_2$ times the original acceleration quality factor, but it will cause a main zero point of the servo system to be far away from the virtual axis and the main poles to be close to the virtual axis. As the time coefficient of the compensation loop controller decreases, the tracking performance of the system increases, but the robustness decreases, the dynamic response deteriorates, the water bed effect becomes more obvious, and the system is more susceptible to noise and disturbances. Compared to the existing method, our method focuses on system performance and robustness. Experimental results show that our method can achieve target tracking with a peak accuracy of 64$″$ and 22$″$ (RMS), which is superior to the tracking performance without equivalent composite control. Full article

In satellite and airborne electro-optical tracking systems, there are numerous processing devices and complex data flows. To ensure the coordinated operation of the system, the multiple devices within the target system must operate under unified time control for data acquisition, computation, and output. This study introduces a multi-channel data simulator based on a time unification system. The complete simulation system includes the host computer, simulator, target system, and time reference generator. The simulator has programmable input and output interfaces for multi-channel protocols and has storage and real-time working modes. In the storage mode, the simulated data are pre-transmitted to the simulator’s storage and sent to the target system according to the time reference generator. The simulator simultaneously stores the target system results. In the real-time mode, the host computer generates simulated data based on the target system’s results and outputs the data through the simulator in real time. The main contribution of the simulator is that it achieves system-level closed-loop simulation and completes the functional and performance verification of the target system. Through experimental verification, it is found that the simulator can achieve 4.2 Gbps of simulated data transmission and 1.6 Gbps of data reception and storage, with a closed-loop delay of 39.9 µs. Full article

This paper investigates an anticipatory activation anti-windup approach based on Linear Active Disturbance Rejection Control (LADRC) to address the influences of accelerated saturation on the actuators in a Miniaturized Inertial Stabilized Platform (MISP) with extreme external disturbance. The proposed method aims to eliminate the high-frequency vibrations on the Line of Sight (LOS) of electro-optical devices during actuator saturation. To achieve this, the Linear Extended State Observer (LESO) is modified by adding saturation feedback to the total disturbance observed state variable, which is operated as an anticipatory activation anti-windup compensator. The stability of the proposed controller is discussed, and the gains are optimized by the Linear Matrix Inequality (LMI) constraints though quadratic programming and an H-infinite performance indicator. Additionally, as the multiple activated scheme for anti-windup, the effectiveness of immediate activation in dealing with accelerated saturation is compared and analyzed. These comparisons and verification are implemented through simulations, where the external disturbance is introduced using recorded attitude data from USV sailing. Finally, experiments are conducted on an MISP for a visual tracking system, demonstrating that the anticipatory activation mothed effectively suppresses high-frequency vibrations on the LOS during instances of accelerated saturation. Full article

A multi-channel phase-compensated active disturbance rejection control (MPADRC) incorporating an improved backstepping strategy is proposed in this paper to handle the phase lag in the extended state observer (ESO) and the residual uncertainty in the system. Firstly, a multi-channel phase-compensated ESO (MPESO) is constructed by adding phase-advanced networks to all output channels of the ESO, which allows disturbances and system states to be compensated and feedback in a more timely manner, respectively. Then, to estimate and offset the residual uncertainty in the system, an improved backstepping control method is employed and a Lyapunov function is designed to verify the convergence of the error between the estimated and actual values of the residual uncertainty. After that, the improved backstepping control is combined with MPADRC, and comparisons with the conventional linear active disturbance rejection control (LADRC) are conducted for a range of cases. Finally, on an inertial stabilization platform in the electro-optical tracking system (ETS), simulation and experimental results verified the effectiveness of the proposed method. Full article

A modified Extended State Kalman Filter (ESKF)-based Model Predictive Control (MPC) algorithm is introduced to tailor the enhanced disturbance suppression in electro-optical tracking systems. Traditional control techniques, although robust, often struggle in scenarios with concurrent internal, external disturbances, and sensor noise. The proposed algorithm effectively overcomes these limitations by precisely estimating system states and actively mitigating disturbances, thus significantly boosting noise and perturbation control resilience. The primary contributions of this study include the integration of ESKF for accurate system state and disturbance estimation in noisy environments, the embedding of an ESKF estimation-compensation loop to simulate an improved disturbance-free system, and a simplified modeling approach for the controlled device. This designed structure minimizes the reliance on extensive system identification, easing the predictive control model-based constraints. Moreover, the approach incorporates total disturbance estimation into the optimization problem, safeguarding against actuator damage and ensuring high tracking accuracy. Through rigorous simulations and experiments, the ESKF-based MPC has demonstrated enhanced model error tolerance and superior disturbance suppression capabilities. Comparative analyses under varying model parameters and external disturbances highlight its exceptional trajectory tracking performance, even in the presence of model uncertainties and external noise. Full article

Electro-optical detection systems face numerous challenges due to the complexity and difficulty of targeting controls for “low, slow and tiny” moving targets. In this paper, we present an optimal model of an advanced n-step adaptive Kalman filter and gyroscope short-term integration weighting fusion (nKF-Gyro) method with targeting control. A method is put forward to improve the model by adding a spherical coordinate system to design an adaptive Kalman filter to estimate target movements. The targeting error formation is analyzed in detail to reveal the relationship between tracking controller feedback and line-of-sight position correction. Based on the establishment of a targeting control coordinate system for tracking moving targets, a dual closed-loop composite optimization control model is proposed. The outer loop is used for estimating the motion parameters and predicting the future encounter point, while the inner loop is used for compensating the targeting error of various elements in the firing trajectory. Finally, the modeling method is substituted into the disturbance simulation verification, which can monitor and compensate for the targeting error of moving targets in real time. The results show that in the optimal model incorporating the nKF-Gyro method with targeting control, the error suppression was increased by up to 36.8% compared to that of traditional KF method and was 25% better than that of the traditional nKF method. Full article

To address the problems of a low tracking accuracy and slow error convergence in high-order single-input, single-output electro-optical tracking systems, a backstepping control method based on a Softsign linear–nonlinear tracking differentiator is proposed. First, a linear–nonlinear tracking differentiator is designed in conjunction with the Softsign excitation function, using its output as an approximate replacement for the conventional differentiation process. Then, this is combined with backstepping control to eliminate the “explosion of complexity” problem in conventional backstepping procedures due to repeated derivation of virtual control quantities. This reduces the workload of parameter tuning, takes into account the rapidity and stability of signal convergence, and improves the trajectory tracking performance. This method can ensure the boundedness of the system signal. The effectiveness and superiority of this control method are verified through simulations and experiments. Full article

This paper addresses the challenge of reduced tracking accuracy in maritime electro-optical tracking equipment when dealing with high-mobility targets like speedboats and aircraft due to off-target error delays. We propose an innovative technique that leverages inertial navigation data to enhance target prediction and tracking control. Our approach involves the real-time integration of high-frequency inertial navigation-derived attitude information into the tracking system. By combining off-target error information with angular measurements from the tracking mechanism, we project the vector of the tracked target into multiple coordinate systems, including the imaging coordinate system, carrier coordinate system, and geographic coordinate system. Subsequently, we model and predict the target’s motion trajectory in the relatively slow-changing geographic coordinate system. This transformation process increases the update frequency and real-time performance of the tracking control position loop command angle. Unlike traditional control methods that heavily rely on the model of the controlled object, our approach significantly improves tracking accuracy and engineering applicability. It offers a technology-based optimization of tracking and control performance through an interdisciplinary theoretical fusion, deeply integrating inertial navigation technology with tracking control technology. Experimental results with maritime electro-optical tracking equipment demonstrate that our proposed control technique increases tracking accuracy for high-speed targets by approximately threefold compared to traditional methods. Under the same experimental conditions, the off-target error statistics are reduced from 1.8 mrad to 633 μrad. Full article