Benefiting from the development of unmanned aerial vehicles (UAVs), the types and number of datasets available for image synthesis have greatly increased. Based on such abundant datasets, many types of virtual scenes can be created and visualized using image synthesis technology before they are implemented in the real world, which can then be used in different applications. To achieve a convenient and fast image synthesis model, there are some common issues such as the blurred semantic information in the normalized layer and the local spatial information of the feature map used only in the generation of images. To solve such problems, an improved image synthesis model, SYGAN, is proposed in this paper, which imports a spatial adaptive normalization module (SPADE) and a sparse attention mechanism YLG on the basis of generative adversarial network (GAN). In the proposed model SYGAN, the utilization of the normalization module SPADE can improve the imaging quality by adjusting the normalization layer with spatially adaptively learned transformations, while the sparsified attention mechanism YLG improves the receptive field of the model and has less computational complexity which saves training time. The experimental results show that the Fréchet Inception Distance (FID) of SYGAN for natural scenes and street scenes are 22.1, 31.2; the Mean Intersection over Union (MIoU) for them are 56.6, 51.4; and the Pixel Accuracy (PA) for them are 86.1, 81.3, respectively. Compared with other models such as CRN, SIMS, pix2pixHD and GauGAN, the proposed image synthesis model SYGAN has better performance and improves computational efficiency. Full article

In this article, we introduce a cooperative obstacle-aware surveillance system for virtual emotion intelligence which is supported by low energy configuration with the minimal wasted communication cost in self-sustainable network with 6G components. We make a formal definition of the main research problem whose goal is to minimize the wasted communication range of system members on condition that the required detection accuracy with the given number of obstacles is satisfied when the requested number of obstacle-aware surveillance low energy barriers are built in self-sustainable network. To solve the problem, we have originally designed and implemented two different approaches, and then thoroughly evaluated them through extensive simulations. Then, their performances based on numerical outcomes are demonstrated with detailed discussions. Full article

Various port cities and authorities have established emission control areas (ECAs) to constrain ships’ fuel usage in a specified offshore geographical range. However, these ECA policies involve high costs and have low monitoring and regulation enforcement efficiencies. In this study, a meeting model was used to investigate the drone-scheduling problem by considering the simultaneous movements of drones and ships. Set-covering integer linear programs were developed to formulate the assignments of drones to ships, and a model and solution algorithm were devised to determine the moving times and meeting positions for particular drones and ships. The proposed models and algorithms were employed and verified in experiments. The flying times for the datasets with three drone base stations were shorter than those with two. More drones resulted in shorter flying distances. The use of the meeting model enabled the acquirement of shorter flying times and distances than when it was not used. The datasets with more ships had longer flying times and distances, with almost linear relationships. The sensitivity of the effect of varying 5% of the ships’ speeds on the flying time metrics was less than 1%, affecting the flying distance by about 4–5%. Accelerating the drones was more effective towards optimizing the drones’ flying distances than times. Numerical studies showed that the consideration of simultaneous movements in the model allowed for a reduction in the drones’ flying distances and increased efficiency. Based on the modeling and experimental studies, managerial implications and possible extensions are discussed. Full article

In human-computer gaming scenarios, the autonomous decision-making problem of an unmanned combat air vehicle (UCAV) is a complex sequential decision-making problem involving multiple decision-makers. In this paper, an autonomous maneuver decision-making method for UCAV that considers the partially observable states of Human (the adversary) is proposed, building on a game-theoretic approach. The maneuver decision-making process within the current time horizon is modeled as a game of Human and UCAV, which significantly reduces the computational complexity of the entire decision-making process. In each established game decision-making model, an improved maneuver library that contains all possible maneuvers (called the continuous maneuver library) is designed, and each of these maneuvers corresponds to a mixed strategy of the established game. In addition, the unobservable states of Human are predicted via the Nash equilibrium strategy of the previous decision-making stage. Finally, the effectiveness of the proposed method is verified by some adversarial experiments. Full article

In this paper, sensor fault diagnosis and fault tolerant control strategy are investigated for quadcopters under sensor faults and disturbances. We propose the fault diagnosis estimation system and the fault-tolerant control (FTC) method. The fault diagnosis system provides time-varying sensor fault estimation under an unknown bound of disturbances. Moreover, the fault-tolerant control eliminates disturbance that is estimated through the associated disturbance observer. Overall, the proposed FTC guarantees the finite-time tracking convergence using nonsingular fast terminal sliding mode algorithm. Stability of the closed-loop system is validated through the Lyapunov theory. Finally, conventional nonsingular fast terminal sliding mode and adaptive neural network sliding mode control are compared with the proposed method through simulations under sensor faults and disturbances with different scenarios. Full article

Structural features of the overstorey in managed and unmanaged forests can significantly influence plant community composition. Native Acacia species are common in temperate eucalypt forests in southeastern Australia. In these forests, intense disturbances, such as logging and wildfire, lead to high densities of regenerating trees, shrubs, and herbs. The tree layer is dominated by Acacia and Eucalyptus, that compete intensely for resources in the first decades after stand establishment. The relative abundance and size of Acacia and Eucalyptus varies widely due to stochastic factors such as dispersal, microsite variability, and weather and climatic conditions. This variability may influence the structure and composition of the herbaceous and shrub species. In the temperate forests of southeastern Australia, understorey plant diversity is assumed to be influenced by Acacia species density, rather than Eucalyptus density. To quantify the influence of Acacia and Eucalyptus density on plant community composition, we used remote sensing and machine learning methods to map canopy composition and then compare it to understorey composition. We combined unoccupied aerial vehicle (UAV or drone) imagery, supervised image classifications, and ground survey data of plant composition from post-logging regrowth forests in the Central Highlands of southeastern Australia. We found that aggregation and patch metrics of Eucalyptus and Acacia were strongly associated with understorey plant beta diversity. Increasing aggregation of Acacia and the number of Acacia patches had a significant negative effect on plant beta diversity, while the number of Eucalyptus patches had a positive influence. Our research demonstrates how accessible UAV remote sensing can be used to quantify variability in plant biodiversity in regrowth forests. This can help forest managers map patterns of plant diversity at the stand-scale and beyond to guide management activities across forested landscapes. Full article

In this paper, we propose an adaptive control approach to deal with the problems of input saturation, external disturbances, and uncertainty in the unmanned autonomous helicopter system. The dynamics of the system take into account the presence of input saturation, uncertainty, and external disturbances. Auxiliary systems are built to handle the input saturation. The neural networks are applied to approximate the uncertain terms. The control scheme combining integral backstepping and sliding mode control is developed in position and attitude subsystems, respectively. In the closed-loop system, the boundedness of the signals is proved by means of the Lyapunov theory. The simulation demonstrates that the approach has good robustness and tracking performance. Full article

In recent years, visual tracking has been employed in all walks of life. The Siamese trackers formulate the tracking problem as a template-matching process, and most of them can meet the real-time requirements, making them more suitable for UAV tracking. Because existing trackers can only use the first frame of a video sequence as a reference, the appearance of the tracked target will change when an occlusion, fast motion, or similar target appears, resulting in tracking drift. It is difficult to recover the tracking process once the drift phenomenon occurs. Therefore, we propose a motion-aware Siamese framework to assist Siamese trackers in detecting tracking drift over time. The base tracker first outputs the original tracking results, after which the drift detection module determines whether or not tracking drift occurs. Finally, the corresponding tracking recovery strategies are implemented. More stable and reliable tracking results can be obtained using the Kalman filter’s short-term prediction ability and more effective tracking recovery strategies to avoid tracking drift. We use the Siamese region proposal network (SiamRPN), a typical representative of an anchor-based algorithm, and Siamese classification and regression (SiamCAR), a typical representative of an anchor-free algorithm, as the base trackers to test the effectiveness of the proposed method. Experiments were carried out on three public datasets: UAV123, UAV20L, and UAVDT. The modified trackers (MaSiamRPN and MaSiamCAR) both outperformed the base tracker. Full article

Mapping forest canopy height at large regional scales is of great importance for the global carbon cycle. Polarized interferometric synthetic aperture radar is an efficient and irreplaceable remote sensing tool. Developing an efficient and accurate method for forest canopy height estimation is an important issue that needs to be addressed urgently. In this paper, we propose a novel four-stage forest height inversion method based on a Fourier–Legendre polynomial (FLP) with reference to the RVoG three-stage method, using the multi-baseline UAVSAR data from the AfriSAR project as the data source. The third-order FLP is used as the vertical structure function, and a small amount of ground phase and LiDAR canopy height is used as the input to solve and fix the FLP coefficients to replace the exponential function in the RVoG three-stage method. The performance of this method was tested in different forest types (mangrove and inland tropical forests). The results show that: (1) in mangroves with homogeneous forest structure, the accuracy based on the four-stage FLP method is better than that of the RVoG three-stage method. For the four-stage FLP method, R² is 0.82, RMSE is 6.42 m and BIAS is 0.92 m, while the R² of the RVoG three-stage method is 0.77, RMSE is 7.33 m, and bias is −3.49 m. In inland tropical forests with complex forest structure, the inversion accuracy based on the four-stage FLP method is lower than that of the RVoG three-stage method. The R² is 0.50, RMSE is 11.54 m, and BIAS is 6.53 m for the four-stage FLP method; the R² of the RVoG three-stage method is 0.72, RMSE is 8.68 m, and BIAS is 1.67 m. (2) Compared to the RVoG three-stage method, the efficiency of the four-stage FLP method is improved by about tenfold, with the reduction of model parameters. The inversion time of the FLP method in a mangrove forest is 3 min, and that of the RVoG three-stage method is 33 min. In an inland tropical forest, the inversion time of the FLP method is 2.25 min, and that of the RVoG three-stage method is 21 min. With the application of large regional scale data in the future, the method proposed in this study is more efficient when conditions allow. Full article

Full-duplex unmanned aerial vehicle (UAV) communication systems are characterized by mobility, so the self-interference (SI) channel characteristics change over time constantly. In full-duplex UAV communication systems, the difficulty is to eliminate SI in time-varying channels. In this paper, we propose a pilot-aid digital self-interference cancellation (SIC) method. First, the pilot is inserted into the data sequence uniformly, and the time-varying SI is modeled as a linear non-causal function. Then, the time-varying SI channel is estimated by the discrete prolate spheroidal basis expansion model (BEM). The error of block edge channel estimation is reduced by cross-block interpolation. The result of channel estimation is convolved with the transmitted data to obtain the reconstructed SI, which is subtracted from the received signal to achieve SIC. The simulation results show that the SIC performance of the proposed method outperforms the dichotomous coordinate descent recursive least square (DCD-RLS) and normalized least mean square (NLMS) algorithms. When the interference to noise ratio (INR) is 25 dB, the performance index normalized least mean square (NMSE) is reduced by 5.5 dB and 4 dB compared with DCD-RLS and NLMS algorithms, which can eliminate SI to the noise floor, and the advantage becomes more obvious as the INR increases. Full article

In recent years, significant progress has been made in the multi-target tracking (MTT) of unmanned aerial vehicle (UAV) swarms. Most existing MTT approaches rely on the ideal assumption of a pre-set target trajectory. However, in practice, the trajectory of a moving target cannot be known by the UAV in advance, which poses a great challenge for realizing real-time tracking. Meanwhile, state-of-the-art multi-agent value-based methods have achieved significant progress for cooperative tasks. In contrast, multi-agent actor-critic (MAAC) methods face high variance and credit assignment issues. To address the aforementioned issues, this paper proposes a learning-based factored multi-agent soft actor-critic (FMASAC) scheme under the maximum entropy framework, where the UAV swarm is able to learn cooperative MTT in an unknown environment. This method introduces the idea of value decomposition into the MAAC setting to reduce the variance in policy updates and learn efficient credit assignment. Moreover, to further increase the detection tracking coverage of a UAV swarm, a spatial entropy reward (SER), inspired by the spatial entropy concept, is proposed in this scheme. Experiments demonstrated that the FMASAC can significantly improve the cooperative MTT performance of a UAV swarm, and it outperforms existing baselines in terms of the mean reward and tracking success rates. Additionally, the proposed scheme scales more successfully as the number of UAVs and targets increases. Full article

In Republic of Korea, cracks in concrete structures are considered to be objective structural defects, and the constant maintenance of deteriorating facilities leads to substantial social costs. Thus, it is important to develop technologies that enable economical and efficient building safety inspection. Recently, the application of UAVs and deep learning is attracting attention for efficient safety inspection. However, the currently developed technology has limitations in defining structural cracks that can seriously affect the stability of buildings. This study proposes a method to define structural cracks on the outer wall of a concrete building by merging the orthoimage layer and the structural drawing layer with the UAV and deep learning that were previously applied during a safety inspection. First, we acquired data from UAV-based aerial photography and detected cracks through deep learning. Structural and non-structural cracks were defined using detected crack layer, design drawing layer defined the structural part, and the orthoimage layer was based on UAV images. According to the analysis results, 116 structural parts cracks and 149 non-structural parts cracks were defined out of a total of 265 cracks. In the future, the proposed method is expected to greatly contribute to safety inspections by being able to determine the quality and risk of cracks. Full article

There has been increased attention paid to autonomous unmanned aerial vehicles (UAVs) recently because of their usage in several fields. Human action recognition (HAR) in UAV videos plays an important role in various real-life applications. Although HAR using UAV frames has not received much attention from researchers to date, it is still a significant area that needs further study because of its relevance for the development of efficient algorithms for autonomous drone surveillance. Current deep-learning models for HAR have limitations, such as large weight parameters and slow inference speeds, which make them unsuitable for practical applications that require fast and accurate detection of unusual human actions. In response to this problem, this paper presents a new deep-learning model based on depthwise separable convolutions that has been designed to be lightweight. Other parts of the HarNet model comprised convolutional, rectified linear unit, dropout, pooling, padding, and dense blocks. The effectiveness of the model has been tested using the publicly available UCF-ARG dataset. The proposed model, called HarNet, has enhanced the rate of successful classification. Each unit of frame data was pre-processed one by one by different computer vision methods before it was incorporated into the HarNet model. The proposed model, which has a compact architecture with just 2.2 million parameters, obtained a 96.15% success rate in classification, outperforming the MobileNet, Xception, DenseNet201, Inception-ResNetV2, VGG-16, and VGG-19 models on the same dataset. The proposed model had numerous key advantages, including low complexity, a small number of parameters, and high classification performance. The outcomes of this paper showed that the model’s performance was superior to that of other models that used the UCF-ARG dataset. Full article

Path planning using handcrafted waypoints is inefficient for a multi-task UAV operating in dynamic environments with potential risks such as bad weather, obstacles, or forbidden zones, among others. In this paper, we propose an automatic path planning method through natural language that instructs the UAV with compound commands about the tasks and the corresponding regions in a given map. First, we analyze the characteristics of the tasks and we model each task with a parameterized zone. Then, we use deep neural networks to segment the natural language commands into a sequence of labeled words, from which the semantics are extracted to select the waypoints and trajectory patterns accordingly. Finally, paths between the waypoints are generated using rapidly exploring random trees (RRT) or Dubins curves based on the task requirements. We demonstrate the effectiveness of the proposed method using a simulated quadrotor UAV that follows sequential commands in four typical tasks with potential risks. Full article