Search Results (29)

Efficient and accurate recognition of highway pavement cracks is crucial for the timely maintenance and long-term use of expressways. Among the existing crack acquisition methods, human-based approaches are inefficient, whereas carrier-based automated methods are expensive. Additionally, both methods present challenges related to traffic obstruction and safety risks. To address these challenges, we propose a fixed pan-tilt-zoom (PTZ) vision-based highway pavement crack recognition workflow. Pavement cracks often exhibit complex textures with blurred boundaries, low contrast, and discontinuous pixels, leading to missed and false detection. To mitigate these issues, we introduce an algorithm named contrast-enhanced feature reconstruction (CEFR), which consists of three parts: comparison-based pixel transformation, nonlinear stretching, and generating a saliency map. CEFR is an image pre-processing algorithm that enhances crack edges and establishes uniform inner-crack characteristics, thereby increasing the contrast between cracks and the background. Extensive experiments demonstrate that CEFR improves recognition performance, yielding increases of 3.1% in F1-score, 2.6% in mAP@0.5, and 4.6% in mAP@0.5:0.95, compared with the dataset without CEFR. The effectiveness and generalisability of CEFR are validated across multiple models, datasets, and tasks, confirming its applicability for highway maintenance engineering. Full article

This article presents a cost-effective camera network system that employs neural network-based object detection and stereo vision to assist a pan–tilt–zoom camera in imaging fast, erratically moving small aerial targets. Compared to traditional radar systems, this approach offers advantages in supporting real-time target differentiation and ease of deployment. Based on the principle of knowledge distillation, a novel data augmentation method is proposed to coordinate the latest open-source pre-trained large models in semantic segmentation, text generation, and image generation tasks to train a BicycleGAN for image enhancement. The resulting dataset is tested on various model structures and backbone sizes of two mainstream object detection frameworks, Ultralytics’ YOLO and MMDetection. Additionally, the algorithm implements and compares two popular object trackers, Bot-SORT and ByteTrack. The experimental proof-of-concept deploys the YOLOv8n model, which achieves an average precision of 82.2% and an inference time of 0.6 ms. Alternatively, the YOLO11x model maximises average precision at 86.7% while maintaining an inference time of 9.3 ms without bottlenecking subsequent processes. Stereo vision achieves accuracy within a median error of 90 mm following a drone flying over 1 m/s in an 8 m $\times$ 4 m area of interest. Stable single-object tracking with the PTZ camera is successful at 15 fps with an accuracy of 92.58%. Full article

Many transportation agencies have deployed pan–tilt–zoom (PTZ) closed-circuit television (CCTV) cameras to monitor roadway conditions and coordinate traffic incident management (TIM), particularly in urbanized areas. Pre-programmed “presets” provide the ability to rapidly position a camera on regions of highways. However, camera views occasionally develop systematic deviations from their original presets due to a variety of factors, such as camera change-outs, routine maintenance, drive belt slippage, bracket movements, and even minor vehicle crashes into the camera support structures. Scheduled manual calibration is one way to systematically eliminate these positioning problems, but it is more desirable to develop automated techniques to detect and alert agencies of potential drift. This is particularly useful for agencies with large camera networks, often numbering in the 1000’s. This paper proposes a methodology using the mean Structured Similarity Index Measure (SSIM) to compare images for a current observation to a stored original image with identical PTZ coordinates. Analyzing images using the mean SSIM generates a single value, which is then aggregated every week to generate potential drift alerts. This methodology was applied to 2200 images from 49 cameras over a 12-month period, which generated less than 30 alerts that required manual validation to determine the confirmed drift detection rate. Approximately 57% of those alerts were confirmed to be camera drift. This paper concludes with the limitations of the methodology and future research opportunities to possibly increase alert accuracy in an active deployment. Full article

Border surveillance and the monitoring of critical infrastructure are essential components of regional and industrial security. In this paper, our purpose is to study the intricate nature of surveillance methods used by hybrid monitoring systems utilizing Pan–Tilt–Zoom (PTZ) cameras, modeled as directional sensors, and UAVs. We aim to accomplish three occasionally conflicting goals. Firstly, at any given moment we want to detect as many intruders as possible with special attention to newly arriving trespassers. Secondly, we consider it equally important to observe the temporal movement and behavior of each intruder group as accurately as possible. Furthermore, in addition to these objectives, we also seek to minimize the cost of sensor usage associated with surveillance. During the research, we developed and analyzed several interrelated, increasingly complex algorithms. By leveraging RL methods we also gave the system the chance to find the optimal solution on its own. As a result we have gained valuable insights into how various components of these algorithms are interconnected and coordinate. Building upon these observations, we managed to develop an efficient algorithm that takes into account all three criteria mentioned above. Full article

Automated progress monitoring of construction sites using cameras has been proposed in recent years. Although previous studies have tried to identify the most informative camera views according to 4D BIM to optimize installation plans, video collection using fixed or pan-tilt-zoom cameras is still limited by their inability to adapt to the dynamic construction environment. Therefore, considerable attention has been paid to using camera-equipped unmanned aerial vehicles (CE-UAVs), which provide mobility for the camera, allowing it to fit its field of view automatically to the important parts of the construction site while avoiding occlusions. However, previous studies on optimizing video collection with CE-UAV are limited to the scanning of static objects on construction sites. Given the growing interest in construction activities, the existing methods are inadequate to meet the requirements for the collection of high-quality videos. In this study, the following requirements for and constraints on collecting construction-activity videos have been identified: (1) the FOV should be optimized to cover the areas of interest with the minimum possible occlusion; (2) the path of the UAV should be optimized to allow efficient data collection on multiple construction activities over a large construction site, considering the locations of activities at specific times; and (3) the data collection should consider the requirements for CV processes. Aiming to address these requirements and constraints, a method has been proposed to perform simulation-based optimization of path planning for CE-UAVs to allow automated and effective collection of videos of construction activities based on a detailed 4D simulation that includes a micro-schedule and the corresponding workspaces. This method can identify the most informative views of the workspaces and the optimal path for data capture. A case study was developed to demonstrate the feasibility of the proposed method. Full article

Pan–tilt–zoom cameras are commonly used for surveillance applications. Their automation could reduce the workload of human operators and increase the safety of airports by tracking anomalous objects such as drones. Reinforcement learning is an artificial intelligence method that outperforms humans on certain specific tasks. However, there exists a lack of data and benchmarks for pan–tilt–zoom control mechanisms in tracking airborne objects. Here, we show a simulated environment that contains a pan–tilt–zoom camera being used to train and evaluate a reinforcement learning agent. We found that the agent can learn to track the drone in our basic tracking scenario, outperforming a solved scenario benchmark value. The agent is also tested on more complex scenarios, where the drone is occluded behind obstacles. While the agent does not quantitatively outperform the optimal human model, it shows qualitative signs of learning to solve the complex, occluded non-linear trajectory scenario. Given further training, investigation, and different algorithms, we believe a reinforcement learning agent could be used to solve such scenarios consistently. Our results demonstrate how complex drone surveillance tracking scenarios may be solved and fully autonomized by reinforcement learning agents. We hope our environment becomes a starting point for more sophisticated autonomy in control of pan–tilt–zoom cameras tracking of drones and surveilling airspace for anomalous objects. For example, distributed, multi-agent systems of pan–tilt–zoom cameras combined with other sensors could lead towards fully autonomous surveillance, challenging experienced human operators. Full article

Traditional object detection methods using static cameras are constrained by their limited perspectives, hampering the effective detection of low-confidence targets. To address this challenge, this study introduces a deep reinforcement learning-based visual perception enhancement technique. This approach leverages pan–tilt–zoom (PTZ) cameras to achieve active vision, enabling them to autonomously make decisions and actions tailored to the current scene and object detection outcomes. This optimization enhances both the object detection process and information acquisition, significantly boosting the intelligent perception capabilities of PTZ cameras. Experimental findings demonstrate the robust generalization capabilities of this method across various object detection algorithms, resulting in an average confidence level improvement of 23.80%. Full article

In the past, plant layouts were regarded as highly static structures. With increasing internal and external factors causing turbulence in operations, it has become more necessary for companies to adapt to new conditions in order to maintain optimal performance. One possible way for such an adaptation is the adjustment of the plant layout by rearranging the individual facilities within the plant. Since the information about the plant layout is considered as master data and changes have a considerable impact on interconnected processes in production, it is essential that this data remains accurate and up-to-date. This paper presents a novel approach to create a digital shadow of the plant layout, which allows the actual state of the physical layout to be continuously represented in virtual space. To capture the spatial positions and orientations of the individual facilities, a pan-tilt-zoom camera in combination with fiducial markers is used. With the help of a prototypically implemented system, the real plant layout was captured and converted into different data formats for further use in exemplary external software systems. This enabled the automatic updating of the plant layout for simulation, analysis and routing tasks in a case study and showed the benefits of using the proposed system for layout capturing in terms of accuracy and effort reduction. Full article

This study proposes a visual tracking system that can detect and track multiple fast-moving appearance-varying targets simultaneously with 500 fps image processing. The system comprises a high-speed camera and a pan-tilt galvanometer system, which can rapidly generate large-scale high-definition images of the wide monitored area. We developed a CNN-based hybrid tracking algorithm that can robustly track multiple high-speed moving objects simultaneously. Experimental results demonstrate that our system can track up to three moving objects with velocities lower than 30 m per second simultaneously within an 8-m range. The effectiveness of our system was demonstrated through several experiments conducted on simultaneous zoom shooting of multiple moving objects (persons and bottles) in a natural outdoor scene. Moreover, our system demonstrates high robustness to target loss and crossing situations. Full article

With the increasing application of dual-PTZ (Pan-Tilt-Zoom) cameras in intelligent unmanned systems, research regarding their calibration methods is becoming more and more important. The intrinsic and extrinsic parameters of dual-PTZ cameras continuously change during rotation and zoom, resulting in difficulties in obtaining precise calibration. Here, we propose a general calibration method for dual-PTZ cameras with variable focal length and posture under the following conditions: the optical center of the camera does not coincide with the horizontal and pitch rotation axes, and the horizontal and pitch rotation axes are not perpendicular to each other. We establish a relationship between the intrinsic and extrinsic parameters and the feedback parameters (pan, tilt, zoom value) of dual-PTZ cameras by fitting and calculating previous calibration results acquired at specific angles and zoom values using Zhang’s calibration method. Subsequently, we derive the intrinsic and extrinsic parameter calculation formula at arbitrary focal length and posture based on the camera’s feedback parameters. The experimental results show that intrinsic and extrinsic parameters computed using the proposed method can better meet precision requirements compared with the ground truth calibrated using Zhang’s method. The average focal length error is less than 4%, the cosine similarity of the rotation matrix between the left and right cameras is more than 99.8%, the translation vector error is less than 1%, and the recalculated Euler angle errors are less than 1 degree. Our work can quickly and accurately obtain intrinsic and extrinsic parameters during the use of the dual-PTZ camera. Full article

The recent trend toward the development of IoT architectures has entailed the transformation of the standard camera networks into smart multi-device systems capable of acquiring, elaborating, and exchanging data and, often, dynamically adapting to the environment. Along this line, this work proposes a novel distributed solution that guarantees the real-time monitoring of 3D indoor structured areas and also the tracking of multiple targets, by employing a heterogeneous visual sensor network composed of both fixed and Pan-Tilt-Zoom (PTZ) cameras. The fulfillment of the twofold mentioned goal was ensured through the implementation of a distributed game-theory-based algorithm, aiming at optimizing the controllable parameters of the PTZ devices. The proposed solution is able to deal with the possible conflicting requirements of high tracking precision and maximum coverage of the surveilled area. Extensive numerical simulations in realistic scenarios validated the effectiveness of the outlined strategy. Full article

Multiple unmanned multirotor (MUM) systems are becoming a reality. They have a wide range of applications such as for surveillance, search and rescue, monitoring operations in hazardous environments and providing communication coverage services. Currently, an important issue in MUM is coverage control. In this paper, an existing coverage control algorithm has been extended to incorporate a new sensor model, which is downward facing and allows pan-tilt-zoom (PTZ). Two new constraints, namely view angle and collision avoidance, have also been included. Mobile network coverage among the MUMs is studied. Finally, the proposed scheme is tested in computer simulations. Full article

To solve the problems of poor exploration ability and convergence speed of traditional deep reinforcement learning in the navigation task of the patrol robot under indoor specified routes, an improved deep reinforcement learning algorithm based on Pan/Tilt/Zoom(PTZ) image information was proposed in this paper. The obtained symmetric image information and target position information are taken as the input of the network, the speed of the robot is taken as the output of the next action, and the circular route with boundary is taken as the test. The improved reward and punishment function is designed to improve the convergence speed of the algorithm and optimize the path so that the robot can plan a safer path while avoiding obstacles first. Compared with Deep Q Network(DQN) algorithm, the convergence speed after improvement is shortened by about 40%, and the loss function is more stable. Full article

Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will result in false negative errors. With moving cameras, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises a background modeler and a deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, which was pre-trained with a specific video dataset. It can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving cameras. The F-measure results, obtained from the pan-tilt-zoom (PTZ) videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high-ranking background subtraction methods by more than 3%. Full article

Explosive ordnance disposal (EOD) robots can replace humans that work in hazardous environments to ensure worker safety. Thus, they have been widely developed and deployed. However, existing EOD robots have some limitations in environmental adaptation, such as a single function, slow action speed, and limited vision. To overcome these shortcomings and solve the uncertain problem of bomb disposal on the firing range, we have developed an intelligent bomb disposal system that integrates autonomous unmanned aerial vehicle (UAV) navigation, deep learning, and other technologies. For the hardware structure of the system, we design an actuator constructed by a winch device and a mechanical gripper to grasp the unexploded ordnance (UXO), which is equipped under the six-rotor UAV. The integrated dual-vision Pan-Tilt-Zoom (PTZ) pod is applied in the system to monitor and photograph the deployment site for dropping live munitions. For the software structure of the system, the ground station exploits the YOLOv5 algorithm to detect the grenade targets for real-time video and accurately locate the landing point of the grenade. The operator remotely controls the UAV to grasp, transfer, and destroy grenades. Experiments on explosives defusal are performed, and the results show that our system is feasible with high recognition accuracy and strong maneuverability. Compared with the traditional mode of explosives defusal, the system can provide decision-makers with accurate information on the location of the grenade and at the same time better mitigate the potential casualties in the explosive demolition process. Full article