Generalized Net Model of Forest Zone Monitoring by UAVs

Abstract

The paper presents a generalized net (GN) model of the process of terrain observation with the help of unmanned aerial vehicles (UAVs) for the prevention and rapid detection of wildfires. Using a GN, the process of monitoring a zone (through a UAV, which is further called a reconnaissance drone) and the localization of forest fires is described. For a more indepth study of the terrain, the reconnaissance drone needs to coordinate with a second UAV, called a specialized drone, so that video and sensory information is provided to the supervising fire command operational center. The proposed GN model was developed to assist in the decision-making process related to the coordination of the operation of both UAVs under dynamically changing terrain circumstances, such as those related to preventing or quickly containing wildfires. It describes the stages (transitions), logical determinants (transition predicate matrices), and directions of information flow (token characteristics) within the process of localization of fires using the pair of reconnaissance and specialized drones.

1. Introduction

For the early prevention of disasters and accidents, the quick discovery of the location of the crisis, risk assessment, and countermeasures aimed at minimizing the potential hazards on human lives and health, on the environment, and on the infrastructure are of paramount importance. With the recent occurrence of forest fires at unprecedented locations, such as behind the Arctic Circle, and unusual periods of the year, with threatening frequency and severity [1], it is critically important to find the exact localization of the origin, the velocity of fire spread, and to perform risk analysis for the environment, people, settlements, and critical infrastructure, in order to ensure that timely and accurate information is provided for deciding on how to best extinguish the fire and suppress its spread. Traditionally used means to detect forest fires, such as fire lookout towers, mobile fire groups, and forest rangers are not sufficient for a prompt reaction to this type of natural disaster. Acoustic monitoring [2], satellite imaging, and unmanned aerial systems (UAS) provide opportunities to prompt action and prevent the spread of forest fires. The advantages of drone usage include the lack of human risk, time saving, promptness, and the level of detail of the information. A number of recent publications on this topic revealed different aspects of the use of unmanned aerial vehicles (UAVs, drones) against this versatile destructive phenomenon [3,4,5,6,7,8,9,10,11,12,13].

In modern technological conditions, UAVs are increasingly used for this purpose. The technology and availability of the most common, namely, so-called drones, are quickly developing, and it is estimated that they will number 1.5 million by 2023, according to research by Business Insider Intelligence [14].

Recently, their use for various purposes is growing. Even in areas such as agriculture, they are increasingly used. For example, some authors proposed solutions for person classification, search, and rescue with airborne optical sectioning [15,16], and photogrammetry and remote sensing [17].

In this paper, generalized nets (GNs [18,19,20]), a tool for modelling of parallel processes, are applied for the description of the process of forest zone monitoring for fire localization using UAVs. Short remarks for GNs in the Mathematics journal are given in [21]. Here, we only mention that a generalized net consists of transitions and places, similar to Petri nets and other of their modifications. The difference with these modifications is that, in the generalized net, tokens enter with initial characteristics and, as they move within the network, they obtain subsequent characteristics that they may during their stay in the net. Each transition is associated with an index matrix (see [22]) whose elements are predicated, and depending on the truth value of the predicate at a certain moment for a certain set of token characteristics, tokens are allowed or not to transition from an input to an output place of a transition, thus changing the characteristic of the process modelled by the token.

In the literature, there are recent approaches to the mathematical modelling of detecting and managing of fires in forest areas: fuzzy set modelling [23,24], agent-based modelling [25,26], and others [27,28]. Here for the first time, the modelling apparatus of generalized nets is implemented, which allows for describing both the analytics of the process via the characteristic of the tokens and the logical conditions for the development of the modelled process by the predicates in index matrices, determining the token transfer. Models may thus become much more detailed than existing ones are.

GNs can open opportunities for different online applications, such as the search for optimal conditions, learning on the basis of experimental data, and control on the basis of expert systems. Until now, the apparatus of GNs was used as a tool for a description of parallel processes in several areas, such as economics, transport, medicine, bioprocess, and computer technologies [18]. The facility of obtaining GN models demonstrates the flexibility and efficiency of generalized nets as a modelling tool in different fields, e.g., artificial intelligence (neural networks, expert systems, decision making, pattern recognition, etc.), biotechnology, medicine, optimization, and intercriterion analysis [18,29].

We employed GN capabilities for description and modelling in order to investigate how to make better use of existing technological advancements. This is the first generalized net model devoted to the topic considered here. The model allows for following in real time how the process unfolds, and for further complexification.

2. Materials and Methods

Drones equipped with thermovision cameras for fire patrolling in forests and agricultural fields are increasingly used, for instance, such a project was implemented by the University of Ruse [30], and the Emergency Command Bureau of Laixi, Qingdao, China even using drones (series EH216F, EH216, and EHang Falcon) for extinguishing fire in tall buildings [31].

With the development of artificial intelligence, intelligent flying devices can be created that can make decisions on choosing the optimal route, avoiding obstacles, radio detection, and situation analysis. Recently, intensive work has been conducted on the creation of intelligent sensors in which the data collection system itself (camera, etc.) has built-in applications for preliminary image analysis and the immediate determination of target objects to which specialized sensors should be directed for closer monitoring [32].

UAV capabilities to acquire visual sensory information from a given fire zone depend on the parameters of the chosen flight platform, the available data transfer and control communications, and on information-acquisition tools, such as surveillance cameras and used sensors [32,33].

The requirements of the flight platform (drone) are based on the need for quickly gathering and processing necessary information, and have the following characteristics:

• drone resilience to unfavorable environmental influence;
• carrying capacity which allows for the transportation of necessary equipment (1–5 kg; in fire extinguishing, up to 30 kg);
• sufficient flight duration to explore a large perimeter of the observation zone (30–120 min);
• drone adaptability for installing (embedding) different types of video and sensory equipment (video cameras, thermovision cameras, LiDAR, sensor for air parameter analysis, mapping sensors, etc.);
• speed for approaching the object and precise positioning (for instance, 50–60 km/h for the approach, and capability for positioning the drone in the air over the object);
• ease of use (comfortable joystick for manipulation and helm or tablet for observation);
• price for acquisition, exploitation, and servicing.

Communication requirements are determined by the parameters of the planned communication modules and interfaces for the transmission of the obtained video and sensory data, and for controlling the drone. They have the following characteristics:

• transmission speed from UAV to base station (depot, center of operations), which may reach 20 Mbit/s for high-resolution video streaming;
• distance of communication: required distance for ensuring lossless data transmission (e.g., 20 km);
• energy consumed for the communication;
• the size and weight of the device.

Tool requirements for obtaining information are determined by the parameters of the video and thermovision cameras installed on the UAV, sensors for the detection of smoke, toxic and suffocating gases, and the presence of aerosols and solid particles. It is essential to ensure reliable protection of both the data transmitted between drone and ground station, and high security of the control channels. The cybersecurity of communications is increasingly important, for which a number of studies and models are considered [34].

To model the process of using UAVs for the early detection of forest fires, this paper proposes the use of a generalized net. The principle of operation is as follows. The fire service operates a pair of drones: the first is lighter and faster, and is sent first to patrol the observation zone (patrolling the forest area). The second is larger, allowing for the installation of specialized video and sensor equipment for precise fire detection and the assessment of environmental parameters. The first, called a reconnaissance drone, is equipped with a thermovision camera with a wide viewing angle and a long range of thermal-radiation detection (from a few hundred meters to several kilometers). That drone traverses the observation zone, starting from one end till other end; after relocating, according to the set field of view, it sequentially moves sideways to enter the next belt of the zone (see Figure 1). If the drone is equipped with an AI system, it can move by itself to a specific object, spotted by the thermovision camera and to analyze whether it is a source of fire. If it lacks the required software for analysis and image recognition, its direction may be performed by an operator who monitors the video camera transmission. When it flies over a specific region that is suspected of a potentially dangerous object or phenomenon, the drone may automatically (or, in the second case, assisted by an operator) sway from its designated path by approaching and lowering to inspect in more detail the object or phenomenon before returning to its original path trajectory (see Figure 2).

Multifaceted information obtained from the reconnaissance drone (coordinates of potential objects, photo and video information, wind speed and direction, presence of smoke, gases, etc.) is transmitted both to the supervising center of operations and to the second drone. The second, so-called specialized, drone, aims to fly only over sites identified by the first drone as potential fires, examining them with high-precision onboard equipment. The more detailed information that it obtains is analyzed in the supervising center of operations, and instructions for action are issued to the specialized firefighting squad. The GN model of this process is described below.

3. Description of Generalized Net Model of Forest Terrain Monitoring

The GN model (see Figure 3) contains 7 transitions, 21 places, and 3 types of tokens as follows:

• $\delta$ tokens, which represent the drones. First, they are in place $l_3$, which denotes the drone depot or center of operation. Drones are either reconnaissance drone ${\delta }^{\prime }$ or specialized drone ${\delta }^{{}^{\prime \prime }};$
• $\phi$ tokens, which represent the firefighting command located at place $l_{16}$, and particular squad ${\phi }^{\prime }$ that attends the detected zone of the fire;
• $\sigma$ tokens, which represent communicated signals from drones to the operational center and the firefighting command or from the firefighting squad at the fire site to the center of operation.

The GN transitions are the following.

$$Z_1=\langle \{l_3,l_4,l_9,l_{14},l_{19}\},\{l_1,l_2,l_3\},\begin{array}{cccc}& l_1& l_2& l_3\\ l_3& W_{3,1}& W_{3,2}& true\\ l_4& false& false& true\\ l_9& false& false& true\\ l_{14}& false& false& true\\ l_{19}& false& false& true\end{array}\rangle ,$$

where

• $W_{3,1}=$ “The moment for take-off of subsequent reconnaissance drone has come”;
• $W_{3,2}=$ “There is a signal from the reconnaissance drone regarding the presence of potential danger”.

In some previously determined time moments, token $\delta$, staying permanently in place $l_3$, splits into two tokens.

In the case that $W_{3,1}$ is true, the token splits to original token $\delta$ and new token ${\delta }^{\prime }$, which enters place $l_1$, where it obtains characteristic “time of take-off”.

In the case that $W_{3,2}$ is true, the token splits into original token $\delta$ and new token ${\delta }^{\prime \prime }$, which enters place $l_2$ with characteristic “time of take-off; coordinates of the area for observation”.

$$Z_2=\langle \{l_1,l_6,l_{12}\},\{l_4,l_5,l_6\},\begin{array}{cccc}& l_4& l_5& l_6\\ l_1& false& false& true\\ l_6& W_{6,4}& W_{6,5}& W_{6,6}\\ l_{12}& false& false& true\end{array}\rangle ,$$

where

• $W_{6,4}=$ “The fly-through of the region is completed, and no accidents have occurred”,
• $W_{6,5}=$ “Detected fire or potential danger for its occurrence”,
• $W_{6,6}=$ “The fly-through of the region has not been completed”.

In place $l_4$, the ${\delta }^{\prime }$ token obtains characteristic “no fire or potential danger detected”; in place $l_5$, it obtains characteristic “fire or potential danger detected; precise coordinates of the detected danger”. In place $l_6$, it does not obtain any new characteristic.

$$Z_3=\langle \{l_2,l_{10}\},\{l_7,l_8,l_9,l_{10}\},$$

$$\begin{array}{ccccc}& l_7& l_8& l_9& l_{10}\\ l_2& false& false& false& true\\ l_{10}& W_{10,7}& W_{10,8}& W_{10,9}& W_{10,10}\end{array}\rangle ,$$

where

• $W_{10,7}=$ “Fire detected by reconnaissance drone”;
• $W_{10,8}=$ “Additional information about the fire must be gathered”;
• $W_{10,9}=$ “The task of the drone is completed”;
• $W_{10,10}=$ “The drone is still traveling to the the area for observation”.

The ${\delta }^{\prime \prime }$ token from place $l_{10}$ enters place $l_9$ when predicate $W_{10,9}$ is true. In that moment, the token splits into two tokens: token ${\delta }^{\prime \prime }$ that enters place $l_9$, and token ${\sigma }_1$ that enters place $l_7$ if predicate $W_{10,7}$ is true, or token ${\sigma }_2$ that enters place $l_8$ if predicate $W_{10,8}$ is true.

In place $l_7$, the ${\sigma }_1$ token obtains a new characteristic depending on the result of observation, which in the event of confirmed fire is “fire alert signal; precise coordinates of the location of the fire site, additional information (area, terrain type, etc.)”; in case that fire is not confirmed, it is “signal canceling the alert”.

In place $l_8$, the ${\sigma }_2$ token obtains characteristic “additional information about the fire site (area, terrain type, etc.”.

In place $l_9$, ${\delta }^{\prime \prime }$ token obtains characteristic “the task is completed”. In place $l_{10}$, it does not obtain any new characteristic.

$$Z_4=\langle \{l_5,l_{13}\},\{l_{11},l_{12},l_{13}\},\begin{array}{cccc}& l_{11}& l_{12}& l_{13}\\ l_5& false& false& true\\ l_{13}& W_{13,11}& W_{13,12}& false\end{array}\rangle ,$$

where

• $W_{13,11}=$ “There is fire or potential danger of fire occurrence.”
• $W_{13,12}=$ “The check for fire or potential danger of fire occurrence is complete”.

The ${\delta }^{\prime }$ token from place $l_{13}$ enters place $l_{12}$ when the predicate $W_{13,12}$ is true. In that moment, the token splits into two tokens: original token ${\delta }^{\prime }$ that enters place $l_{12}$, and token ${\sigma }_3$ that enters place $l_{11}$ if predicate $W_{13,11}$ is true.

In place $l_{11}$, the ${\sigma }_3$ token obtains characteristic “fire alert signal; coordinates”.

In place $l_{12}$, token ${\delta }^{\prime }$ obtains characteristic “resum original fly-through path of the region”. In place $l_{13}$ it does not obtain any new characteristic.

$$Z_5=\langle \{l_7,l_{11},l_{16},l_{20}\},\{l_{14},l_{15},l_{16}\},\begin{array}{cccc}& l_{14}& l_{15}& l_{16}\\ l_7& false& false& true\\ l_{11}& true& false& true\\ l_{16}& false& W_{16,15}& true\\ l_{20}& false& false& true\end{array}\rangle ,$$

where

• $W_{16,15}=$ “Fire is detected”.

The ${\sigma }_3$ token splits into two tokens, ${\sigma }_3^{\prime }$ and ${\sigma }_3^{\prime \prime }$, which enter places $l_{14}$ and $l_{16}$, respectively, keeping the characteristic of token ${\sigma }_3$.

The $\phi$ token from place $l_{16}$ stays permanently in place $l_{16}$, but when the predicate $W_{16,15}$ is true, it splits into two tokens: original token $\phi$ and a new token ${\phi }^{\prime }$, which enters place $l_{15}$.

In place $l_{15}$, the ${\phi }^{\prime }$ token obtains characteristic “members of the deployed firefighting squad; equipment; hour of departure”.

$$Z_6=\langle \{l_{15},l_{18}\},\{l_{17},l_{18}\},\begin{array}{ccc}& l_{17}& l_{18}\\ l_{15}& false& true\\ l_{18}& W_{18,17}& W_{18,18}\end{array}\rangle ,$$

where

• $W_{18,17}=$ “The firefighting squad has arrived at the location of the fire”;
• $W_{18,18}=$ “The firefighting squad is still on the way”.

In place $l_{17}$, the ${\phi }^{\prime }$ token obtains characteristic “hour of arrival; travel duration.”. In place $l_{18}$, it does not obtain any new characteristic.

$$Z_7=\langle \{l_8,l_{17},l_{21}\},\{l_{19},l_{20},l_{21}\},\begin{array}{cccc}& l_{19}& l_{20}& l_{21}\\ l_8& false& false& true\\ l_{17}& false& false& true\\ l_{21}& W_{21,19}& W_{21,20}& W_{21,21}\end{array}\rangle ,$$

where

• $W_{21,19}=$ “Specialized drone must be dispatched to gather additional information”;
• $W_{21,20}=$ “The fire is extinguished, and the firefighting squad must return to the base”;
• $W_{21,21}=$ “The fire has not been extinguished yet”.

From place $l_{17}$, the ${\phi }^{\prime }$ token enters place $l_{21}$, where it stays by the moment when predicate $W_{21,20}$ is true and it enters place $l_{20}$. If the firefighting squad needs additional information, token ${\phi }^{\prime }$ from place $l_{21}$ splits into two tokens: original token ${\phi }^{\prime }$ that continues to stay in place $l_{21}$, and token ${\sigma }_4$ that enters place $l_{19}$.

In place $l_{19}$ the ${\sigma }_4$ token obtains characteristic “request for a specialized drone to gather additional information”.

In place $l_{20}$, the ${\phi }^{\prime }$ token obtains characteristic “the fire is extinguished; time taken to extinguish fire”. In place $l_{21}$, it obtains characteristic “current situation of firefighting squad and the region of the fire””.

The so-constructed GN model describes the logic of the parallel processes involved in a UAV-assisted monitoring of forest terrains for the localization of potential fires and hotspots. While the presented model is simplistic, it can easily be upgraded to include more details about the actual procedures and protocols, and additional monitoring and control capabilities, thus enabling real-time decision making. Each transition can be replaced with a new, separate generalized net through the flexibility and efficiency of the hierarchical operators over GNs [19,20].

4. Conclusions

Prevention and quick response in case of fires and other disasters are greatly important to rapidly overcome these emergencies. The exact location of the forest fire and the provision of timely information about possible factors for its growth are key to dealing with the situation. The use of UAVs for this purpose helps fire services to quickly and correctly assess a disaster, make adequate decisions, and implement them in a timely manner.

To assist in the decision-making process in such topical problems, such as preventing or quickly containing wildfires, the apparatus of generalized nets is proposed. For the first time, a GN model for forest zones monitoring was developed. The presented original GN model describes in real time the stages (transitions), logical determinants (transition predicate matrices), and directions of information flow (token characteristics) within the process of localization of fires using a pair of UAVs (a reconnaissance drone and a specialized drone), UAV movements and communications between themselves, and between any of them and the supervising fire command operational center. The GN model presented here is an attempt to use GNs to both model these processes, and further use them for control and system optimization.