Wildfires are among the most destructive disasters. These catastrophes have an enormous impact in populated regions. It applies to the United States (Western states) [1
]; Canada (South-Western) [2
]; Mediterranean Europe (Portugal, Spain, France, Italy and Greece) [3
] and South Eastern Australia [4
]. As can be seen, this is a worldwide problem and not just on less developed countries. The means available for fighting the wildfires are clearly insufficient for their efficient suppression. The answer to this problem must be not only reactive, but also proactive. The prevention of fires and the implementation of strategies to help the firefighting are imperative.
One of the possible strategies is the implementation of a fire break network (FBN). A fire break (FB) is a strip of land that has been strategically and artificially modified, where vegetation density is reduced to break up the continuity of fuel. It acts as a barrier to slow or stop the progress of wildfire, thus improving fire control opportunities. Technically, in the Portuguese FBN, an FB is a land strip with 125 m wide composed by three regions: the road network, with a minimum width of 5 m; the fuel interruption with a minimum width of 10 m where all vegetation is cut; the fuel reduction composed by two zones, where a minimum distance between the tree tops is imposed (the complete information is available in [6
]). The technical specifications are synthetized in Figure 1
and a ground observation of an FB can be seen in Figure 2
Since the vegetation is always growing, the monitorization of an FB is essential for its efficiency. The Portuguese Institute of Nature and Forest Conservation (Instituto de Conservação da Natureza e das Florestas, ICNF) planned a priority network of FBs to help control wildfires and decrease their burnt area. The implementation and maintenance of FB is crucial for its efficiency and should be ensured by the Local Authorities. The verification by the National Authorities (ICNF) of the previous premises can be made by ground observation, but it is expensive and time consuming if a wide network is implemented (the FBN was defined as having 11,125 km, having 1600 km already implemented). The current practice is the visualization of satellite imagery, pointing out the cases that need special attention. This technique is also a time-consuming process and prone to errors. In this paper, a remote sensing semi-automatic methodology for the detection of maintenance operations in an FB is presented.
With the launch of the Sentinel 2 (the first satellite of the constellation in June 2015 and the second in March 2017) free satellite imagery with a spatial resolution of 10 m and a temporal resolution of five days was made available to the community. This new high spatial resolution allowed the analysis of the FB conditions, which in current work, was to detect when maintenance operations are performed. It is common to divide remote sensing applications into two groups: land cover classification [7
] and change detection [11
]. The proposed methodology fits into the second group. In this kind of applications, the output gives the information of the occurrence of an event in the study area. In Hamunyela et al. [12
], forest disturbances were detected with resource to two observations and spatio-temporal features and in Hermosilla et al. [13
]; annual composites were generated to detect changes. In [15
], different approaches were used to identify changes in the land cover and the kind of changes. In [16
], pixel-based methods were implemented, with three and six observations, respectively, while in [15
], an object-based technique was used. It should be noticed that in these works [11
] Landsat imagery was used, which means that the best achievable detection period is 16 days (but due to the atmospheric conditions the period is usually worse). Additionally, with the exception of [15
], more than one observation of previous data is needed for the detection. Usually, the change detection relies on the slope of the time-series and consecutive variations in vegetation indices. These applications showed the importance of the temporal and spatial resources for disturbances detection.
The purpose of this work is to identify when maintenance operations occur in the FBs. Although there is not an evaluation of the quality of the operation, only complete operation in the FB are to be detected. Relatively to the common change detections methods the goals were:
Use of Sentinel-2 data instead of Landsat imagery, due to its increased frequency and spatial resolution;
Identify only a specific kind of operation efficiently, dealing with the phenology and different types of vegetation;
Use of common vegetation indices and other indices;
Reduce the previous data used, identifying the maintenance as soon as possible, allowing a classification whenever a new observation is made, as in [15
To achieve these objectives and based on current literature, several requirements were defined as follows:
Object-based classification—since an FB is a well-defined area, it will be defined as an object. This approach can—better capture its spatial characteristics;
Temporal dynamics—the use of time-series allows the determination of the temporal dynamics, which is essential in change detection methods;
Machine learning—the use of artificial intelligence techniques to enhance the change detection classification.
There are two main stages: the data extraction, and the maintenance operations detection with an artificial neural network (ANN). In the first stage, the time-series and the datasets are created. It includes the geolocation correction of the observations, calculation of the mean values of the reflectance bands within the FB, followed by the application of a noise reduction filter and finally, computation of the spectral indices. The second stage includes a preliminary training step for feature selection, ANN training and error estimation, and a classification step to identify the months where a maintenance operation was executed.
The structure of this paper is as follows: Section 2
describes the study area, the data used for the implementation, and presents the dataset and the detection methodology; Section 3
show the results of the intermediary steps for designing the ANN and the detection results followed by their discussion in Section 4
. Finally, the conclusions are in Section 5
The results will be divided into two sections. On the first, the results that supported the ANN design are presented, while on the second the results of the implemented methodology are presented.
3.1. Feature Selection and Artificial Neural Network Sizing
Feature selection techniques were applied to the training set, to reduce the number of features and use the most discriminant ones. This process was applied to both median and mean filtered data. The results presented here will be only for the median filter, due to its similarity with the mean filter. The total number of samples used in the ANN training were 633, which accounted for 67% of all the clear observations in the four study areas multiplied by the 13 FBs.
According to the SelectKBest feature selection algorithm, the best features in this analysis are: B04, B05, B11, B12, NMDI, NDI, ExG, ExR, ExGR, and MExG. Additionally, the Pearson absolute correlations between these features were calculated, to identify which ones give more information together.
Analysing the mean correlation values between a feature and all the others (Table 4
), the more independent features are the NMDI and ExG, with 0.751 and 0.748, respectively. The least correlated features pairs were ExG/B05 and ExG/B11, with 0.600 and 0.602, respectively. From this analysis, it was concluded that ExG is probably the most discriminant feature, and consequently it was one of the selected features to be used in the detection. The higher correlated pairs were B11/B12 (0.986) and ExR/ExGR (0.984). In the first pair, the features have a similar behaviour, but B12 has a higher mean correlation, therefore it was rejected. Regarding the other pair, ExR is less correlated to ExG than ExGR, but ExGR works better with B05 and B11 than ExR. The NDI do not enhance the occurrence of a maintenance operation. Finally, B04 is extremely correlated with B05 (0.975), and the same happens with MExG and ExG (0.958). However, with B04 and MExG the same happens as with the NDI.
To access the performance of the classifier for different feature combinations, several groups of features were defined, as presented in Table 5
First, all the groups were tested on an ANN with one hidden layer, varying the number of neurons in the interval (5,100) with steps of five. In the cross validation, five folds were used and the process was repeated 10 times, with the mean error being used. The results presented in Figure 13
a show that is possible to divide the groups into three sets according to the cross-validation error: the first two tend to a relative error of approximately 6%–8%, while on the third set (group 5, 6, 9, 11) the error can be less than 4%. The best results were verified for the second category, i.e., groups 5, 6, 9 and 11. The recall, precision and F1 score for each class (detection and no detection), also demonstrated a better performance for this third set.
b shows the recall, used to check the ability to avoid false negatives. Again, groups 5, 6, 9, and 11 were confirmed as better suiting this problem. Although they present similar results, groups 5 and 6 use less features; in these groups NDMI index is not included. If this was used in the data extraction stage, two more bands would be needed (B8A and B12), as well as an extra calculation and two extra neurons in the input layer. This means more processing time, particularly in the data extraction. For these two groups, an ANN with two hidden layers was tested, with the goal of reducing the number of neurons and improve the detection. This test did not show any improvements, so it was defined as an ANN with just one hidden layer.
To decide which of these two groups will be used and the final number of neurons, a final test was done varying the number of neurons in the interval (45,60) (where the error is approximately 3%–5%, and the recall and precision are above 90%), with steps of one neuron. Here, both groups have nearly the same error. Since B05 had better results in the feature selection stage, group 5 was chosen. For the ANN structure, one hidden layer with 53 neurons was defined (with an error lower than 3%), with a penalty of 8. For the mean filtered dataset, the selected features were also from Group 5, and the ANN was composed by an ANN with 53 neurons (the error is approximately 3%), with a penalty of 8. In both, during the training stage and structure definition, a penalty of 10 was used. After that, the penalty was decreased until the error metrics started deteriorating.
3.2. Maintenance Operations Detection Results
In this section, the classification results, using both median and mean filtered data, will be presented. Since this methodology will be used to validate the execution of maintenance operations in an FB, it must avoid the false positives. However, there are few true positive examples in the available data, and this is the reason why there is a thin balance between the recall and precision of each class. If the ANN is trained to detect all operations, it will decrease the precision of the classifier.
For the validation dataset, 10 ANN were generated with the previous specifications, to assess the sensitivity of the ANN classifier. The results are presented in Table 6
and Table 7
. The identification of maintenance operations is not possible in all cases. With the median filtered data, the detection results are worse in the validation dataset. Although, it is important to remember that in the validation dataset only six examples correspond to maintenance operations, so an error in just one example has a huge impact in these metrics. The precision of 57% for the detection of the operations is low, but is needed to guarantee that false positives rarely occur. Additionally, the recall value of 87% shows that most of the operations were detected.
A more detailed analysis of the results for the validation dataset showed that the errors usually occur in continuous operations, just one of the months being identified. Evaluating the use of the mean filter, it can be verified that with the validation dataset, there is more precision in the detection (64%), i.e., there are less false detections of maintenance operations. The drawback is that the recall decreases to 77%, being more difficult to detect the real operations. So, the median filter allows more operations detections, but due to its increased sensibility to transitions in the data, it produces more false detections than using the mean filter.
Both classifiers were applied to the real case scenario (Marisol) and correctly detected one maintenance operation in March 2018. Unfortunately, there were also two false detections in May and June of 2017. These can be explained with the fact that there was not any clear observation in May 2017 in Marisol, which led to more significant changes in the data in this period. The observations that confirm the results are in Figure 14
Finally, relative to the incomplete operations test dataset, the nine cases were divided in three groups of operation completeness. The results by group are presented in Table 8
. (they were the same for both noise filters). Only one case was wrongly detected and it was in the 50%–75% group, where completeness of the operation on the FB was 73%. Additionally, in the tested FBs with a partial intervention of less than 50%, no operations were detected. This was a good indicator that incomplete operations will not be detected, which is important for the FBN monitoring.
Monitoring of the forest is a fundamental task in fire prevention. Not only is it important to characterize the land cover, but also to understand what is changing and why it is happening, or to validate the execution of land management operations. As is known and verified during this study, the phenology is a major concern in change detection problems, since it represents transitions in the time-series without occurring any modification in the coverage. Additionally, when earth observation satellites such as Sentinel-2 are used, the variations in the luminosity add more background noise to the data. Another concern about using optical satellites is its sensitivity to adverse atmospheric conditions, which leads to greater periods without available observations. Finally, a methodology to detect changes in any kind of forest needs to handle the different behaviour of vegetation types.
The results from the feature selection revealed that the better indices for the detection were based on the visible spectrum, namely ExR and ExR, which is understandable, since an operation is easily visible in TCI observations. Additionally, a further analysis of the time-series shows that these indices are more robust of the seasonal effects. This showed the importance of exploring other regions from the electromagnetic spectrum in remote sensing.
Compared with previous works, one of the achievements of this work was to achieve a monthly timestep for the change detection. This is important, since in [12
], several observations were needed, and by using Landsat more than one month was needed to detect changes. In [13
], annual composites were used, which only enabled the creation of annual change maps and in [15
], the used observations were spaced by more than one month. The drawback was that the use of less data led to the occurrence of false positives (due to less confirmation steps) and the imposition of a month timestep obliged to estimate month values whenever there were no usable observations. In this work the error is approximately the same as in [12
], but was tested in geographically separated areas, with different kinds of vegetation and during two full years, representing a more uncontrolled environment. In [10
], the composites were generated in specific seasons of the year, avoiding the phenology effects. The proposed ANN was trained to detect only a specific kind of change, without discriminating changes as in [15
The two steps for the noise reduction (object-based analysis and the application of noise filters) improved the quality of data and reduced false detections. Additionally, for filtering, two approaches were presented. The mean filter revealed more robustness to the false positives, although less ability to identify the desired events than the median filter. This is verified by the lower recall and higher precision of the mean approach. Although these methods helped to separate the phenological trends from the maintenance operations, misclassifications are still occurring. Since, in this case, it is very important to avoid the false positives, the mean filter was found to be a better solution.
The results in the test datasets were also promising, since in Marisol the expected operations were detected. Additionally, the results in the incomplete FBs test verified that this methodology could distinguish a complete operation from an incomplete. Although there is not a classification of the quality of the operation, if an operation is detected, it is known with good certainty that it was a correct operation.
The first and most important conclusion is the applicability of the presented methodology to the detection of maintenance operations in defined FBs. The results range between the 2.5% and 3.3% of error in the training and validation datasets. However, the F1-Scores in the validation dataset were in the order of 70%, which reveals that there still some false positives and false negatives. With the test dataset, the operations were only detected if more than 50% of the area was operated.
The wanted improvements, relative to other methods, were achieved; there is no need of confirmation if a change really happened; Sentinel-2 data, which give future works a better frequency of observations, were used; a specific kind of change is detected and usually is not confused with the background effects of phenology, and the less common vegetation indices had good performances in the detection.
As future work is expected to develop techniques for obtaining more observations, this compensates those affected by partial cloudy conditions. This implies the application of techniques to recover deteriorated images due to atmospheric conditions, and with the resource to radar data from Sentinel-1 (generating time-series of this data, as in [31
]). The last may be used just for the image recovery, but also to get more information. Additionally, testing the ANN with the difference between the feature values in consecutive months instead of using the values of the two months is intended. This will lead to an ANN with less features; consequently, a simpler ANN. Finally, a different class for continuous operations could be defined, that is characterized by smother changes, which can be misclassified as seasonal changes due to the phenology.