Monitoring Forest Cover Dynamics Using Orthophotos and Satellite Imagery

Abstract

The assessment of changes in forest coverage is crucial for managing protected forest areas, particularly in the face of climate change. This study monitored forest cover dynamics in a 6535 ha mountain area located in north-west Romania as part of the Apuseni Natural Park from 2003 to 2019. Two approaches were used: vectorization from orthophotos and Google Earth images (in 2003, 2005, 2009, 2012, 2014, 2016, 2017, and 2019) and satellite imagery (Landsat 5 TM, 7 ETM, and 8 OLI) pre-processed to Surface Reflectance (SR) format from the same years. We employed four standard classifiers: Support Vector Machine (SVM), Random Forest (RF), Maximum Likelihood Classification (MLC), Spectral Angle Mapper (SAM), and three combined methods: Linear Spectral Unmixing (LSU) with Natural Breaks (NB), Otsu Method (OM) and SVM, to extract and classify forest areas. Our study had two objectives: (1) to accurately assess changes in forest cover over a 17-year period and (2) to determine the most efficient methods for extracting and classifying forest areas. We validated the results using performance metrics that quantify both thematic and spatial accuracy. Our results indicate a 9% loss of forest cover in the study area, representing 577 ha with an average decrease ratio of 33.9 ha/year⁻¹. Of all the methods used, SVM produced the best results (with an average score of 88% for Overall Quality (OQ)), followed by RF (with a mean value of 86% for OQ).

1. Introduction

The importance of temperate forest ecosystems in the carbon cycle, as well as their aesthetic, social, and anti-erosion, or hydrological control functions, are well-established [1,2,3]. However, the long-term effects of climate change on these ecosystems, which have already begun to affect their functionality in various ways [4,5], remain poorly understood. The observed and predicted increases in temperature in temperate areas of Central and Eastern Europe since 1950 [6] have led to changes in the phenology of temperate forests, with an extension in the period of growth being positively appreciated for biomass production and carbon sequestration rates based on several studies [4,5,7,8]. However, the increase in periods with extreme temperatures, droughts [9], and heavy precipitation events has also led to an increase in natural disturbances specific to European temperate forests, such as windthrow, bark beetle infestation (Ips typographus (L.)), and root rot (Heterobasidium annosum, e.g.) [10]. Senf and Seidl [11] reported that of the 227 million hectares of forests in Europe [12], 17% were disturbed between 1986 and 2016 due to natural and/or anthropogenic causes. Similarly, Forzieri et al. [13] quantified the vulnerability of European forests to fires, windthrows, and insect outbreaks and estimated that 33.4 billion tonnes of forest biomass could be affected by these disturbances. The results of Wang et al.’s [3] analysis of the effects of climate change, CO₂ increase in the atmosphere, and nitrogen deposition on aboveground net primary production in a temperate forest suggest that the total negative effect induced by climate change will not be fully compensated by the total positive effect.

Remote sensing and GIS (geographic information system) techniques are widely used in the scientific community to monitor and assess forest ecosystems quantitatively and qualitatively [14,15]. These efforts include studying the spatial extension and alterations of forest vegetation at global, regional, and local levels, with clear implications for biodiversity decline [16,17]. In recent decades, research in this area has diversified from identifying and extracting forest vegetation classes [18,19,20,21,22,23] or tree species [24,25,26,27,28,29] to analyzing the spatial-temporal evolution of forested areas based on various ranges of satellite images [30,31,32,33,34,35,36,37,38,39,40,41,42]. Many studies focus on comparative analyses of methods for extracting, classifying, and quantifying vegetation, including forests, and some combined this with time series imagery.

Huang et al. [43] tested Support Vector Machines (SVM), Maximum Likelihood Classification (MLC), Neural Network (ANN), and Decision Tree (DT) algorithms on a modified Landsat TM (1985) image from eastern Maryland, USA, with six land coverage classes (closed forest, open forest, woodland, non-forest land, land-water mix, and water). They evaluated the classifiers’ performance based on thematic accuracy and highlighted SVM’s results, depending on the kernel type used. Shafri et al. [44] tested MLC, Spectral Angle Mapper (SAM), ANN, and DT to map Malaysian tropical forests using hyperspectral data and highlighted MLC’s superior performance in conditions of maximum biotic heterogeneity. Otukei and Blaschke [45] used DT, SVM, and MLC to assess land cover changes using two Landsat scenes (5 TM from 1986 and 7 ETM+ from 2001) in eastern Uganda and found that all methods were effective with acceptable accuracy, with slight advantage for DT.

Comparative research in this field includes studying the behavior of different combinations of entry data used in classifications, the behavior of classifiers with varying training sample sizes, and the performance of combinations between classification methods. For example, Forkuor et al. [46] used SVM, Random Forest (RF), and Stochastic Gradient Boosting (SGB) to test different combinations of Landsat 8 OLI and Sentinel-2 images for land-use and land-cover mapping in Burkina Faso. They found that using the two Red Edge bands from Sentinel-2 improved land-use land-cover (LULC) mapping and SGB outperformed SVM and RF in overall performance. Thanh Noi and Kappas [47] compared RF, k-Nearest Neighbor (kNN), and SVM for six Land Use Land Cover (LULC) types, including forest, using 14 different training sample sizes in the Red River Delta of Vietnam based on a Sentinel-2 image. Their study emphasized the need to adjust parameters for the compared classifiers and indicated SVM’s lower sensitivity to training sample sizes. Adugna et al. [48] tested SVM and RF for regional land cover mapping in East Africa based on FY-3C scenes with a resolution of 1 km. Nguyen et al. [49] used 446 Sentinel-2 images from 2017 and 2018 to map LULC, including tropical forest classes, in Dak Nong, Vietnam, using four classification methods (Multinomial Logistic Regression—MLR, Improved kNN, RF, and SVM) tested for four time sequences (wet season, dry season, the entire year 2017, and a combination of wet-dry seasons). Ruggeri et al. [50] combined Linear Spectral Unmixing (LSU) with Object-Based Image Analysis (OBIA) and Iterative Self-Organizing Data Analysis Technique (ISODATA) with OBIA for land cover mapping in high mountain areas in Colombia, using a classification scheme adapted from CORINE Land Cover (CLC), which includes forest classes. Dabija et al. [51] used Random Forests and SVM on Sentinel-2 and Landsat 8 OLI satellite images covering three regions from three European Union countries to test CLC classes, including forest classes such as broad-leaved and coniferous forests.

All of the studies presented aim to identify the classification methods with the highest efficiency for the given entry data and intended purposes. Our study seeks to evaluate the dynamics of forest cover in a 6535 ha mountainous area located in NW Romania over a 17-year period (2003–2019) using GIS techniques, as well as extraction and classification methods for forest vegetation on multispectral scenes. Specifically, we employed LSU combined with Natural Breaks (NB), Otsu Method (OM), and SVM, SAM, SVM, RF, and MLC, respectively. Our primary objectives were to determine the exact loss/growth of forest cover in the area during the given time period and to validate the best method or methods for extracting/classifying forest cover from satellite data, using GIS data as the reference basis. In other words, we aimed to identify which methods, applied to medium-resolution satellite scenes, produced results closer to the GIS data obtained from high-resolution images. To assess the accuracy of the results, we used two types of entry data: continuous data and discrete data [52].

2. Materials and Methods

2.1. Study Area

The study area, Padiș, is located in the north of the Bihor massif in the Apuseni Mountains, which are part of the Romanian Western Carpathians (Figure 1). This area encompasses over 90% of one of the most complex and spectacular karst morphosystems in Romania, including the closed basin Padiș-Cetățile Ponorului, an endorheic area of 36 km² with underground drainage that resembles a karst plateau situated at an altitude of 1250–1280 m [53], the Someșului Cald-Cetatea Rădesei gorge sector, and Galbenei gorges. Padiș falls administratively under Bihor County, and ecological management is provided by the Apuseni Natural Park, a protected area of national interest of category V IUCN, established through Ministerial Order no. 7/1990 and reconfirmed by Law no. 5/2000.

The dominant vegetation in the area consists of coniferous forests (Norway spruce—Picea abies), broad-leaved forests (European beech—Fagus silvatica), and, to some extent, mixed forests (beech and spruce) [53,54]. Vegetation inversions occur frequently due to microclimatic conditions, with spruce growing in deep karst depressions. Grasslands (meadows) composed mainly of grass species also occupy significant areas within the karst plateaus. In some places, intensive grazing and woodland exploitation have led to a decrease in the density of herbaceous vegetation cover, resulting in decay. The Forest CLC classes (2000–2018) [55] that cover the area include coniferous forest, broad-leaved forest, and mixed forest. As a protected area, the anthropogenic activities that put pressure on these natural components are related to wood exploitation, animal husbandry, and intensive tourism during the spring-summer season.

2.2. Data Acquisition and Pre-Processing

For this study, we used two types of data, which we will generically refer to as Data for GIS processing and Remote Sensing Data (

Table 1. The data sets used in the study.

Year	Data for GIS Processing	Remote Sensing Data
2003	Google Earth	Landsat 7 ETM—27 May 2003
2005	Orthophoto	Landsat 5 TM—18 July 2005
2009	Orthophoto	Landsat 5 TM—26 May 2009
2011	Google Earth 2011, Orthophoto 2012	Landsat 5 TM—12 July 2011
2014	Google Earth	Landsat 8 OLI—4 July 2014
2016	Orthophoto	Landsat 8 OLI—26 August 2016
2017	Google Earth	Landsat 8 OLI—14 September 2017
2019	Google Earth	Landsat 8 OLI—19 August 2019

). The selection of these data was based on several criteria, such as cloud cover and the period of the year with the most intense physiological activity of the forest, but the primary consideration was their temporal correspondence (year). We identified eight data sets covering a 17-year period.

The first type of data comprised orthophotos with a 0.5 m resolution from the National Agency for Cadastre and Land Registration (ANCPI) and Google Earth images (captured with the Google Earth Pro application) which were used for digitizing forest polygons. Prior to use, all Google Earth data underwent an image-to-image registration procedure [56] based on the orthophoto from 2012 with the highest location precision (20 cm). The scale at which they were taken from the application is 1: 2000. For 2011, the Google Earth image was supplemented in the southern part with the 2012 orthophoto.

The Remote Sensing Data used in this study were the multispectral images from Landsat 5 TM, 7 ETM, and 8 OLI obtained from the United States Geological Survey (USGS) [57] as Level 1 Products and geometrically calibrated by the satellite data providers. We mention that the Landsat 7 ETM scene does not have any data gaps, with it being prior to 31 May 2003, when the Scan Line Corrector (SLC) failed. All data underwent standard pre-processing operations in ENVI 5.3, including spatial subset, radiometric calibration for the FLAASH module, and atmospheric correction. Radiometric calibration for the FLAASH module involved the transfer of data from digital number (DN) format to radiation values with a scale factor of 0.1. For the conversion of radiation values into Top of Atmosphere Reflectance (ToA), we used the Fast Line-of-sight Atmospheric Analysis of Spectral Hypercubes (FLAASH) with a single scale factor of 1 for all bands, as well as the mid-Latitude Summer Atmospheric Model, Tropospheric Aerosol Model, and Kaufman-Tanre Aerosol Retrieval.

To reduce the shading effect on ToA and transition to Surface Reflectance (SR), we used the Cosine Correction Method [58] in SAGA GIS, with a Digital Elevation Model (DEM) with a 5 m resolution generated by ANCPI in the Laki2 project [59] from LiDAR data.

2.3. Methods

Figure 2 and Figure 3 summarize the workflow for the methods used in this study. To obtain the forested area from orthophotos and Google Earth images, which we refer to as Forest_GIS (F_GIS) in this study, we vectorized in ArcGIS 8.1 the land cover/use classes (forest, pasture, bare rock, road, and built area) in the series of eight images used. Subsequently, we extracted the vectorial layers related to the forest using a specific tool (“Select”). We assumed a linear error of 1 m among the GIS data and 4 m between these and the ones from the satellite scenes, respectively. After the overlay operations, errors ranging from 16 ha to 18 ha resulted from the shifting to a unique system of data projection (WGS_1984_UTM_Zone_34N), in addition to handling errors.

For the extraction of forest cover from SR, we used four standard classifiers (SAM, MLC, SVM, RF) and LSU.

Linear Spectral Unmixing (LSU) and Spectral Angle Mapper (SAM) are physics-based methods that share a procedural approach to some extent. They both rely on the extraction of endmembers, which correspond to the spectral response of material components that make up the pixels in an image [60]. However, the mathematical models used for extracting information from these spectral signatures differ between the two methods. LSU is a standard technique for processing multispectral and hyperspectral scenes, developed by Adams et al. [61]. It allows for the extraction of material component fractions (abundances) at the pixel level, viewed as endmembers. The model assumes that the spectral signatures from each pixel level can be expressed as a linear combination of endmembers balanced by their abundance [62]. SAM is a classification method developed by Kruse et al. [63] that determines the spectral similarity between reference spectra (spectral signatures of components viewed as endmembers) and the spectral response at the pixel level. The model considers only the angular difference expressed in radians between two compared spectra, treated as vectors in a space with n dimensions (given by the number of spectral bands), rather than the length of the spectral vectors. As a result, it is expected to be less sensitive to alterations in lighting conditions [64,65].

For both methods, we used the same set of endmembers, one for each year, identified at the pixel level, and extracted the spectral signatures directly from the reflectance of the year in question using the Pixel Purity Index algorithm [66,67]. We combined this with a supervised assessment using orthophotos, Google Earth images, and fieldwork.

Each set of spectra in our study comprises two spectral signatures for the forest, two for the grassland, one for the road and built areas (due to resolution limitations), and sequentially, one for bare rock (not for all years). From 2009 onwards, we added another spectral signature for recently clear-cut land. Field observations showed that the two spectral signatures for the forest are primarily explained by the dominant species (coniferous forest and broad-leaved forest) and the different lighting/exposure of forest areas. For grassland, we distinguished between normal grassy surfaces and nearly degraded ones based on the continuity of the grassy vegetal cover.

In the scientific literature, there are several procedures for interpreting and analyzing fractions with forest vegetation or only with vegetation [68,69,70,71,72,73,74]. We selected two procedures based on image segmentation according to threshold values (NB and OM) and one that involves the use of a classifier (SVM).

In all three procedures, we worked with rasters resulting from the sum of fractions with forest vegetation from each LSU (for each year) using the Band Math tool in ENVI. Mathematically, this procedure is correct because if the LSU processing is done correctly, the overall sum of fractions at the pixel level cannot exceed 1 (the values are between 0 and 1 because we used the “unit sum constraint” conditioning).

NB, also known as Jenks Natural Breaks Method, is widely used in GIS software [75] and remote sensing studies [76,77,78]. We used NB to obtain an initial separation of rasters into two classes, after which we analyzed the threshold values (break values) and noticed a small degree of variation between 0.56–0.64 (with five out of seven values close to 0.6). To homogenize the procedure, we chose the threshold value of 0.6 to separate the two classes. In other words, pixels with a proportion of trees exceeding or equal to 60% were classified as forest. We reclassified each raster to transform it into an integer format and then into a vector layer from which we extracted the forest polygons forming F_{UNMIX_NB} for the forthcoming analysis.

OM is a non-parametric and unsupervised method of automatic threshold selection for image segmentation [79], widely described in the literature [80,81] and commonly used in remote sensing [82,83,84]. We implemented it as Binary Thresholding in ArcGIS Pro [85] to separate forest/non-forest classes on rasters with added-up forest abundances.

The use of a classifier on LSU derivatives is a frequent procedure [66,69,86] for establishing LULC or resolving some problems of classification/ambiguity at the pixel level [87]. We chose SVM, and the details of this method are presented below. The forest polygons resulting from the conversion of new rasters derived through LSU with OM and LSU with SVM, respectively, form the vector layers called F_{UNMIX_OM} and F_{UNMIX_SVM} in this study.

Regarding the processing of rasters resulting from SAM, we performed two Post Classification operations on each of them: Combine Classes for compatibility and comparison with cu F_GIS and Majority Analysis for spatial homogenization of the classes. Furthermore, we exported the SAM rasters in ArcGIS, converted them into vectorial formats, and extracted forest polygons, which we refer to as Forest_SAM (F_SAM) in this study.

Support Vector Machine (SVM) and Random Forest (RF) are two supervised classification methods, non-parametric, from the Machine Learning family, requiring training samples acquired from the field or images with excellent resolution. Although similar in terms of entry data requirements, SVM and RF differ fundamentally in terms of the mathematical logic of class separation. SVM is based on the optimal margin method for separable data, identifying a hyper-plane that separates the data, and the kernel method for tracking them with a non-linear transformation to a higher dimensional space [88,89]. RF is based on plotting decision trees by using the Classification and Regression Trees Algorithm (CART), decision trees that make independent predictions by training on randomly extracted subsets from the entry data, and the final prediction is a result of combined individual predictions. [90,91]. SVM is based on the principle of statistical learning theory, conceptualized by Vapnik [92], and probably at present, alongside RF, is one of the most used methods of classification of multi and hyperspectral data [93,94] because it can be used with good results even when the territorial complexity is higher, and the information providing a consistent training data is lower. We used SVM with the Radial Basis Function kernel type, and for RF, we selected 150 decision trees, bearing in mind the actual recommendations [95].

Maximum Likelihood Classification (MLC) is a parametric classifier that uses the Bayesian Decision Rule to calculate the probability of an element being a member of each n class [96,97], and that requires, much like SVM and RF, training samples as entry data, alongside multispectral images.

The training data set for each year was extracted using orthophotos and Google Earth images (

Table 2. Number of pixels for training data sets used in SVM, RF, and MLC.

Year	Coniferous Forest	Broad-Leaved Forest	Pasture	Pasture with Sparse Vegetation	Bare Rock	Road and Build	Total Number of Pixels
2003	12306	3012	2673	548	21	27	18587
2005	14696	4509	2951	387	21	23	22587
2009	13750	3943	2827	399	21	23	20963
2011	12560	3092	2870	660	21	27	19230
2014	12562	3084	2883	663	21	27	19240
2016	12306	2947	2748	1035	21	27	19084
2017	12303	2940	2742	1035	21	27	19068
2019	11923	2883	2080	1120	21	24	18051

), representing approximately 27% of the 72,600 pixels covering the study area. The same pixels were used for SVM, RF, and MLC for comparison purposes, and the LULC classes were kept the same as for SAM and LSU, except for the clear-cut surfaces, which were integrated with pastures.

The SVM, RF, and MLC rasters underwent the same post-classification operations as those from SAM, and the resulting vector layers with forest were named Forest_SVM (F_SVM), Forest_RF (F_RF) and Forest_MLC (F_MLC), respectively. We mention that all the final data used in the comparative analysis for forested areas are also in vector format. Extraction and overlay vectorial (intersect, symmetrical difference) techniques were then applied in order to obtain the necessary data for the comparative analysis and evaluation for the accuracy of the results.

2.4. Accuracy Assessment

In this study, we conducted an assessment of both thematic accuracy [98] and spatial accuracy [99] in line with our intended objectives.

For the assessment of thematic accuracy, we worked with two versions of Producer’s Accuracy and User’s Accuracy, which we refer to as PA₁/UA₁ and PA₂/UA₂, respectively, to avoid confusion. Both versions have the same significance: Producer’s Accuracy indicates the probability of a reference sample being correctly classified, and User’s Accuracy indicates the probability that a sample classified represents that category on the ground [100]. The differences arise from the type of data used for validation samples. We used training samples at the pixel level for PA₁/UA₁, with a Stratified Random sampling scheme and 200 samples on each raster in a forest/non-forest format, in accordance with current recommendations [101,102].

We introduced PA₂/UA₂, which was developed by Zhan et al. [103] for assessing the accuracy of geospatial objects, to match our aim of conducting comparative assessments between F_GIS and F_{UNMIX_NB}, F_{UNMIX_OM}, F_{UNMIX_SVM}, F_SVM, F_RF, and F_MLC.

$${\mathrm{PA}}_2=\frac{\left|\mathrm{C}\cap \mathrm{R}\right|}{\mathrm{R}}$$

(1)

$${\mathrm{UA}}_2=\frac{\left|\mathrm{C}\cap \mathrm{R}\right|}{\mathrm{C}}$$

(2)

where C represents the area of the classified object (in our case, the forest area from classification), R represents the area of the reference object (in our study, the forest area from digitization—F_GIS), and $\left|\mathrm{C}\cap \mathrm{R}\right|$ represents the area of the intersection between C and R.

For spatial accuracy assessment, we used Overall Quality (OQ) [99,103]:

$$\mathrm{OQ}=\frac{\left|\mathrm{C}\cap \mathrm{R}\right|}{\left|\neg \mathrm{C}\cap \mathrm{R}\right|+\left|\mathrm{C}\cap \neg \mathrm{R}\right|+\left|\mathrm{C}\cap \mathrm{R}\right|}$$

(3)

where $\left|\neg \mathrm{C}\cap \mathrm{R}\right|$ represents the area of R that is not covered by C, and $\left|\mathrm{C}\cap \neg \mathrm{R}\right|$ represents the area of C that is not covered by R.

3. Results

3.1. Monitoring Forest Area with GIS Data

We analyzed the dynamics of forest cover in the study area using data obtained through digitization (F_GIS). These data are presented spatially in Figure 4 and quantitatively in

Table 3. The forest areas derived from vectorization (F_GIS) are expressed in hectares (ha) and as percentages.

Year	Total Area (ha)	FGIS (ha)	Percent FGIS (%)
2003	6535.81	5037.16	77.07
2005		5028.74	76.94
2009		4911.17	75.14
2011		4705.92	72.00
2014		4627.28	70.80
2016		4616.95	70.64
2017		4558.50	69.75
2019	6535.81	4459.80	68.24

, indicating a decrease in the forest area of 577 hectares from 2003 to 2019, resulting in an average rate of decrease of 33.9 hectares per year for the forest area. In terms of percentage, we observed a reduction in the degree of forestation from 77% to approximately 68% for the analyzed territory over just 17 years.

Two aspects need to be addressed here. Firstly, the value of 577 hectares should be understood as a net quantitative loss, as there are areas where the forest has extended into the territory due to natural regeneration during the 17 years analyzed. Symmetrical Difference operations performed on the vector layers reveal a spatial increase of 164 hectares and a decrease of 741 hectares.

Secondly, regarding the average rate, there were no significant negative alterations (decrease) or positive ones between 2003 and 2005, and the time frame was too short for any increase phenomenon to be (noticeable) considerable. From 2005 to 2009, there was a moderate decrease in the forest by 117 hectares, with an average decrease rate of 23.4 hectares per year for five years. However, from 2009 to 2011, there was a dramatic decrease of 205.25 hectares, with an average rate for the three years of 68.41 hectares per year, as shown in the diagram in Figure 5.

The rate of decrease in forest cover during the last three years (2017–2019) is consistent with the average rate observed over the 17 years analyzed.

3.2. Comparative Analysis

The comparative analysis of the methods used, from the perspective of the results achieved (F_GIS, F_{UNMIX_NB}, F_{UNMIX_OM}, F_{UNMIX_SVM}, F_SAM, F_SVM, F_MLC, F_RF), focuses on three key aspects: the quantitative correspondence of absolute values (i.e., the value in hectares obtained from each layer) of forest cover, the thematic accuracy (i.e., how well the forest area is extracted), and the spatial accuracy of classifications.

To assess the quantitative correspondence of absolute values, we analyzed the forest areas of F_{UNMIX_NB}, F_{UNMIX_OM}, F_{UNMIX_SVM}, F_SAM, F_SVM, F_MLC, and F_RF (as shown in

Table 4. The absolute forest areas were obtained using these methods: F_{UNMIX_NB}, F_{UNMIX_OM}, F_{UNMIX_SVM}, F_SAM, F_SVM, F_MLC, F_RF, and F_GIS.

Year	FGIS (ha)	FUNMIX_NB (ha)	FUNMIX_OM (ha)	FUNMIX_SVM (ha)	FSAM (ha)	FSVM (ha)	FMLC (ha)	FRF (ha)	Masked Pixels (ha)
2003	5037.16	5099.94	5202.99	5144.49	4441.05	4992.84	4309.65	4752.27	0
2005	5028.74	5139.09	5010.84	5211.00	4931.28	4897.71	4518.72	4784.49	0
2009	4911.17	4972.05	4929.39	4889.43	4223.70	4996.89	4266.18	4483.17	57.15
2011	4705.92	4961.61	4970.70	4697.82	4707.27	4805.82	3934.71	4555.17	134.28
2014	4627.28	4590.81	4480.47	4425.03	3987.27	4795.11	3847.41	4470.03	191.61
2016	4616.95	4528.26	4945.05	4665.78	3989.97	4836.96	4422.06	4587.75	57.97
2017	4558.50	4526.37	5187.69	5045.40	4555.89	4829.31	4521.87	4541.49	0
2019	4459.80	4506.93	4275.99	4445.01	4449.87	4640.49	4449.33	4345.50	0

) in relation to disparities compared to FGIS (as shown in

Table 5. The difference between F_GIS and F_{UNMIX_NB}, F_{UNMIX_OM}, F_{UNMIX_SVM}, F_SAM, F_SVM, F_MLC, F_RF.

Year	Difference FGIS_FUNMIX_NB (ha)	Difference FGIS_FUNMIX_OM (ha)	Difference FGIS_FUNMIX_SVM (ha)	Difference FGIS_FSAM (ha)	Difference FGIS_FSVM (ha)	Difference FGIS_FMLC (ha)	Difference FGIS_FRF (ha)
2003	62.78	165.83	107.33	−596.11	−44.32	−727.51	−284.89
2005	110.35	−17.90	182.26	−97.46	−131.03	−510.02	−244.25
2009	60.88	18.22	−21.74	−687.47	85.72	−644.99	−428
2011	255.69	264.78	−8.10	1.35	99.90	−771.21	−150.75
2014	−36.47	−146.81	−202.25	−640.01	167.83	−779.87	−157.25
2016	−88.69	328.10	48.83	−626.98	220.01	−194.89	−29.20
2017	−32.13	629.19	486.90	−2.61	270.81	−36.63	−17.01
2019	47.13	−183.81	−14.79	−9.93	180.69	−10.47	−114.30
Sum (–) Difference	−157.29	−348.52	−246.89	−2660.58	−175.35	−3675.6	−1425.66
Sum (+) Difference	+536.82	+1406.12	+825.32	+1.35	+1024.94	0	0
Sum Total Difference	694.11	1754.64	1072.21	2661.93	1200.29	3675.60	1425.66

), which were categorized as positive (overestimation) or negative (underestimation) and summed up to obtain terms for comparison. This type of analysis is known as “non-site-specific analysis” in the literature [101].

As shown in Figure 6, the smallest deviations were observed for F_{UNMIX_NB} and F_{UNMIX_SVM}, followed by F_SVM, while the largest deviations were observed for F_MLC and F_SAM. For F_{UNMIX_NB}, there were five situations of overestimation (one of which was insignificant at +47 hectares) and three years of negative deviations (two of which were significant). The average value of total deviations (694.11 hectares) for the eight years for F_{UNMIX_NB} was 86.76 hectares. For all the forest area forms derived from LSU, there was a prevalence of positive deviations, with an additional 1406 hectares for F_{UNMIX_OM}.

In the case of F_SVM, there was a clear tendency of overestimation, with positive deviations observed in six out of the eight years for which we have data. The average value of total deviations (1200.29 hectares) for the eight years for F_SVM was 150 hectares. For F_RF, the deviations are only negative, with an average of 170 hectares.

These data provided a partial comparison of the values obtained through the methods used in this study, as they do not provide information on the quality of classifications or the spatial correspondence of forest areas.

To assess thematic accuracy, we used two versions of Producer’s Accuracy and User’s Accuracy (PA/UA), which offer multiple possibilities for comparison. These include comparative assessments of thematic accuracy for the methods used, indirect assessments (through comparison) of accuracy for F_GIS, and comparative analyses between the two versions of calculation for PA and UA.

Table 6. The values expressed in percentages for PA₁ (Producer’s Accuracy based on discrete data) and PA₂ (Producer’s Accuracy based on area). LSU_OM—Linear Spectral Unmixing with Otsu Method; LSU_NB—Linear Spectral Unmixing with Natural Breaks; LSU_SVM—Linear Spectral Unmixing with SVM.

Year	LSU_OM		LSU_NB		LSU_SVM		SAM		SVM		MLC		RF
	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)	PA1 (%)	PA2 (%)
2003	96	93	93	92	95	93	80	82	90	94	83	84	90	91
2005	93	93	96	94	96	95	83	85	93	93	87	87	93	92
2009	93	91	93	92	93	91	80	77	90	94	82	84	96	86
2011	96	94	95	94	94	91	92	87	96	94	86	81	92	91
2014	88	88	88	88	88	87	74	73	97	94	81	79	94	90
2016	97	95	91	91	94	93	85	79	94	95	88	90	95	91
2017	98	98	94	90	97	96	92	91	93	96	92	92	94	91
2019	96	90	98	93	99	92	97	90	96	95	98	92	93	91
Average	95	93	94	92	95	92	86	83	94	94	87	86	93	90

lists the values for PA₁ and PA₂.

As individual values, PA ranges from 99% (PA₁ for LSU_SVM in 2019) to 73% (PA₂ for SAM in 2014), with an average of 90.8%. Only five PA values out of 112 are in the acceptable accuracy class (70–79%) [104], while the others belong to the high accuracy class (80–89%, 25 values) and very high accuracy class (over 90%, 82 values). Comparing PA₁ and PA₂, the difference between the pair values is generally not significant, with 80% of cases ranging between 1–3% in favor of PA₁ (with only 11 situations out of 56 obtaining higher scores for PA₂). LSU_OM and LSU_SVM (95%) have the best score for PA₁ as the mean value for the entire time frame, while SVM has the best result (94%) for PA₂. SVM also stands out for its consistently high values on both series of results (94% for PA₁ and PA₂).

Statistics for UA (

Table 7. The values expressed in percentages for UA₁ (User’s Accuracy based on discrete data) and UA₂ (User’s Accuracy based on area).

Year	LSU_OM		LSU_NB		LSU_SVM		SAM		SVM		MLC		RF
	UA2 (%)	UA1 (%)	UA2 (%)	UA1 (%)	UA2 (%)	UA1 (%)	UA2 (%)	UA2 (%)	UA1 (%)	UA2 (%)	UA1 (%)	UA2 (%)	UA1 (%)	UA2 (%)
2003	95	90	96	93	96	92	95	92	98	95	99	98	98	97
2005	94	93	93	92	93	92	86	87	96	96	97	97	97	96
2009	94	91	94	91	94	91	90	90	93	92	98	96	99	96
2011	94	89	94	89	95	91	91	87	96	92	97	97	98	94
2014	95	91	95	88	94	91	87	84	95	91	99	95	95	93
2016	92	89	95	93	93	92	90	92	92	91	93	94	97	92
2017	90	86	94	91	92	87	93	91	94	90	96	93	98	92
2019	97	94	95	92	95	92	95	90	93	91	93	93	99	93
Average	94	90	95	91	94	91	91	89	95	92	97	95	97	94

) show a net prevalence of values belonging to the very high accuracy class and high accuracy class, with two maximum values of 99% obtained by MLC for the UA₁ version and a minimum of 84% for SAM in the UA₂ version. Comparing UA₁ and UA₂, the proportion of the difference between the pair values that range between 1–3% is only 69%, with UA₂ exceeding UA₁ in only three cases out of 56. The MLC and RF classifiers have the best scores for UA₁ and UA₂, followed by SVM and LSU_NB based on mean values for both UA₁ and UA₂.

Overall Quality (OQ) for single-class assessment [99,103] quantifies spatial accuracy by considering both the degree of overlap and non-overlap (as shown in Equation (3)).

The results for OQ, presented numerically in

Table 8. The values expressed in percentages for Overall Quality (OQ).

Year	OQ_LSU_OM (%)	OQ_LSU_NB (%)	OQ_LSU_SVM (%)	OQ_SAM (%)	OQ_SVM (%)	OQ_MLC (%)	OQ_RF (%)
2003	85	85	85	77	90	83	89
2005	87	94	88	76	90	85	89
2009	83	84	83	71	87	81	83
2011	84	84	84	77	87	79	85
2014	81	80	80	65	87	77	85
2016	85	85	85	74	87	85	84
2017	84	82	84	83	87	86	84
2019	84	85	85	82	87	86	85
Average	84	85	84	76	88	83	86

and graphically in Figure 7, exceed the critical value of 50%, which is the lower threshold for validation and use in assessing spatial accuracy [99]. The best score for each value separately was recorded by LSU_NB in 2005 (94%), while the lowest OQ was recorded by SAM in 2014 (65%). Statistically, the range of 80–90% (45 values out of 56, or 80% of the results) has the highest frequency in the series of results for OQ.

Based on the average series of values, SVM has the best spatial accuracy with an 88% OQ, followed by RF with 86%. If we exclude SAM, which has a lower OQ with an average of 76%, all the other methods consistently obtained scores above 80%.

The main problem that this study aims to address is identifying the best method for extracting a forest area that corresponds to the actual situation on site.

Table 9. The ranking positions and Overall Ranking for LSU_OM, LSU_NB, LSU_SVM, SAM, SVM, MLC, and RF in relation to performance metrics.

Method	Ranking from PA1	Ranking from PA2	Ranking from UA1	Ranking from UA2	Ranking from OQ	The Average of Ranking Positions	Overall Ranking
LSU_OM	1	2	3	5	4	3	4
LSU_NB	2	3	2	4	3	2.8	3
LSU_SVM	1	3	3	4	4	3	4
SAM	5	6	4	6	6	5.4	6
SVM	2	1	2	3	1	1.8	1
MLC	4	5	1	1	5	3.2	5
RF	3	4	1	2	2	2.4	2

provides a ranking of the methods used based on the results assessed through performance metrics. By using a simple arithmetic average of values without weighting criteria importance (equivalent to performance indices), we can obtain an average ranking position that translates easily into an Overall Ranking (OR) for the methods used. Comparing the positions in this OR with those in OQ, it is noticeable that the situation is identical, which further emphasizes the importance of OQ as a measure of spatial accuracy.

SVM stands out not only for its first two positions in PA₂ and OQ but also for its consistently high positions in all the accuracy assessments presented. In contrast, SAM records the lowest positions in the hierarchies listed in Table 9.

It is worth noting that although the differences between the scores obtained by the methods used are small, only a 1% difference actually corresponds to a difference of 40–50 hectares (depending on the year). RF occupies the second position, with the best results for UA₁ and two second positions at UA₂ and OQ. Much like SVM, it stands out for consistently high results regarding OQ. LSU_NB closely follows RF (with a difference of only 0.6 in the average ranking positions) but does not register any first place for the performance indicators used. Although LSU_SVM has the best score for PA₁ (along with LSU_OM), it is outperformed by SVM and LSU_NB for other performance measures. LSU_OM has very good results for PA, but it overestimates the forest area, which explains its fourth and fifth-place rankings in OQ and UA₂, respectively. It is important to note that none of the LSU combinations were able to accurately assess the situation in 2014, which had a cloud cover of 2.93%, not even when combined with a non-parametric method like SVM. The effects of the masked pixels in 2014 are best seen in the scores for PA (Table 6) and OQ (Table 8). Only SVM and RF maintained their high percentage for all performance indicators in that year, which can be explained by the way these classifiers work (as discussed in Section 2.3). There is no sensible explanation for the behavior of LSU_SVM in 2014. However, both LSU_SVM and SVM were able to identify the sudden decrease in forest area between 2009 and 2011, while other versions of LSU and SAM were not able to capture it. The explanation for this can be found in the metadata of the images used in the study. The multispectral Landsat 5 image used was from July 2011, while the Google Earth image was from October 2011, which suggests that the land clearing may not have been captured by the Landsat image. Unfortunately, the study does not have data to precisely show the period of logging.

One interesting finding in this study is the rather unusual situation observed with MLC, which occupies the first position at UA, regardless of the version, with very high scores, but weaker results at PA, OQ, and QC. This suggests that MLC is very effective at classifying pixels identified as part of a particular class (in this case, the forest), but it struggles with omission errors.

An important aspect of any comparative study regards highlighting the behavior of the used classifiers when the entry data are changing. Accordingly, a small experiment was conducted using a Sentinel 2 MSI-Level 2A (SR) satellite scene from July 2019, obtained from the Copernicus Scientific Hub [105]. Four spectral bands with a resolution of 10 m (Blue, Red, Green, and Near-Infrared) and six with a resolution of 20 m (Red-edge 1, Red-edge 2, Red-edge, NIR narrow, and two SWIR bands) were extracted from the scene. The bands with a resolution of 20 m were resampled to 10 m, and then all 10 bands were combined to create two multi-band files: one having only Sentinel bands, and the other, in which were combined the spectral bands from Landsat 8 image (2019, resampled to 10 m) used in this study with the three red-edge bands dedicated to vegetation in Sentinel. These files were then used with the SVM and RF classifiers, which offered the best results in this study. The resulting classified images (those presented in Figure 8 are derived from the first multi-band file—Sentinel 10 m) were processed and evaluated for accuracy according to the methodology used. If we compare the accuracy indices values obtained on these 2 data sets (

Table 10. The performance metrics values obtained by SVM and RF for multi-band file classification.

	SVM					RF
Multi-band file	PA1 (%)	PA2 (%)	UA1 (%)	UA2 (%)	OQ (%)	PA1 (%)	PA2 (%)	UA1 (%)	UA2 (%)	OQ (%)
Sentinel, 2019	95	93	99	93	87	98	94	98	93	87
Landsat 8 plus Sentinel Red-edge bands, 2019	93	92	99	93	86	99	92	99	93	86

) with the performance indices average values registered by SVM and RF on the classified images from Landsat scenes, we observe that RF obtains scores up to 6% higher than the average indices (at PA₁), while SVM has moderate increases or stagnates. This aspect would indicate a higher potential for RF, even if it obtains second place in our hierarchy.

4. Discussion

One of the guidelines of this study follows the use of some performance indices to express the spatial accuracy of the results obtained. If we had not used performance measures for assessing spatial accuracy and instead relied solely on continuous data for thematic accuracy, such as removing PA₂, UA₂, and OQ, then RF would occupy fourth position in terms of performance, and LSU_OM would be in first position, along with SVM, followed by LSU_SVM. However, if we had used Overall Accuracy (OA), then RF would have been considered the best performer (with a mean value of 93% for OA), and SVM would only occupy the third position. We did not use OA in this study because previous research [99,103,105,106] has shown that OA takes into account those pixels that are not part of the class of concern (in this case, the forest). Therefore, for a single-class assessment, it is advisable to use PA and UA.

Regarding the occurrence and spatial distribution of errors that appear in the classification procedures carried out, it can be observed that irrespective of their type (omission and commission) and the classification method used (which can affect the extent of errors), as illustrated in Figure 9, errors are predominantly localized in marginal forest strips that transition to pastures, recently deforested areas, and regions with sparse arboreal vegetation.

As a secondary aspect of our analysis, we address the issue of interpreting LULC classes from the CORINE 2018 database [55], specifically the Transitional woodland-shrub class (code 324). Some polygons within this class represent recent clear-cut areas where the forest has completely disappeared, at least in the territory we studied (Figure 10). Although we did not incorporate the CORINE Land Cover data in our comparative analysis due to the mismatch between the scales (1:100,000) used in the dataset, which covers almost a continental-size territory and cannot represent polygons under 25 km² [107], and the finer resolution (0.5 m) of our digitized data, it is crucial to highlight that for adjacent years, the differences between the CLC and F_GIS datasets are noteworthy. For instance, the differences in CLC and F_GIS datasets for forest cover in 2000 and 2006 are negligible (175 ha underestimation in 2000 and 40 ha overestimation in 2006). However, for CLC 2016 and 2018, compared to F_GIS 2016 and 2017, the differences are more significant (231 ha overestimation for 2016 and 326 ha overestimation for 2018, both values with a positive sign), despite the superior quality of satellite data (Landsat 8 OLI and Sentinel 2) used by CLC. The discrepancies in our study area can be attributed to the interpretation of CLC classes and the fragmentation of areas with dense forest.

Upon comparing our results with those obtained from high-resolution data sets, Dominant Leaf Type and Forest Type, available online at land.copernicus.eu [108], we observed even greater differences, particularly in overestimating the forest extent for the latter. The data indicated that the forest extent was 5290.24 ha in 2012 and 4953.7 ha in 2018. Additionally, considerable discrepancies were found when comparing our results with the sets of global data Forest extent—2000 and Forest extent—2020 [37], produced by the GLAD Landsat Analysis Ready Data team [36], which are also accessible online [109]. According to our processed data for Padiș, the forest extent was 5706 ha in 2000 and 5407 ha in 2020.

We wish to clarify the causes of the significant decrease in forest area in Padiș. Based on our research and observations, we assert that these causes are multifaceted. Natural factors, such as severe storms [110,111,112], causing windthrows (the latest one occurred on 17 September 2017), as mentioned in the PNA environmental reports [113,114,115,116,117,118], and pests, such as the European spruce bark beetle (Ips typographus) [119,120], have contributed to this decline. Furthermore, the damaged wood provides an opportunity for exploitation, even in protected areas, as allowed by Romanian laws. However, the amount of wood harvested exceeds the amount naturally damaged, resulting in legal loggings being transformed into illegal ones. Such illegal activities are well documented in the literature [121,122].

How do our findings fit into the context of similar studies that focus on the comparative evaluation of the methods employed? Taking into account the studies cited in Section 1, alongside additional ones, we can place this study in the “group” that obtained better results for SVM and RF. This assertion is because the accuracy scores of important classifiers in other studies vary [94]. However, it can be challenging to make quantitative comparisons between the results due to the different performance metrics used [123,124,125,126].

For SVM, we mention several studies with better results, including Pal and Mather [127], Rokni Deilmai et al. [128], Abe et al. [66], Maxwell et al. [129], Thanh Noi and Kappas [47], Hasan et al. [130], Nguyen et al. [49], Zagajewski et al. [29], and Dabija et al. [51]. The latter [51] obtained the best scores for SVM with Radial Basis Function compared to RF for the CLC classes. For classes 311 (deciduous forests) and 313 (mixed forests), the overall accuracy ranges between 63–98% and 75–83%, respectively. For class 324 (Transitional woodland-shrub), the overall accuracy ranges from a minimum of 56% to a maximum of 82%.

Among the comparative studies in which RF has the best results or obtains similar results to SVM, we mention those carried out by Adugna et al. [48], Rodriguez-Galiano et al. [131], Christovam et al. [132], Tomala et al. [133], Bayrakdar et al. [134], and Avci et al. [135]. The scores obtained for RF by Forkuor et al. [46] are also remarkable, which surpass 90% OA, even if they are slightly outrun by SGB.

For the Unmixing approach, Lu et al. [20] used LSU with the threshold method to classify vegetation from the Amazon basin and compared the results with MLC, resulting in an extra 7.4% OA for LSU. Taureau et al. [73] used LSU and NDVI in a comparative study of the mangrove forest canopy obtained from hemispherical photographs and found an excellent correlation for LSU (R²—0.95). Ruggeri et al. [50] found the best results for the LSU-OBIA combination, with a User’s Accuracy of 88%. However, Medina Machín et al. [86] obtained weaker results for the LSU with SVM (81.4% OA) version in a study for the classification of vegetation in a coastal-dune ecosystem, despite using various combinations of spectral bands, vegetation indices, textural parameters, and fractional abundances from LSU.

Regarding our results from MLC and SAM, they align with the line of research that employs these methods. Sohn et al. [136] used a version of SAM in a study on deforestation and forest recovery stages in Yucatan, which yielded results exceeding 70% UA. Shafri et al. [44] obtained 85.56% OA for MLC and only 48.83% OA for SAM. Li et al. [137] evaluated various spectral combinations from Landsat TM, vegetation indices, and classification methods, including MLC, concluding that MLC and OBIA offered the best results.

5. Conclusions

This study analyzes changes in forest cover in a protected area by integrating remote sensing processing results with GIS to assess accuracy, which is the most sensitive aspect of this approach. The complementary nature of remote sensing and GIS provides real support for monitoring forest areas in protected areas, given the easy access to satellite data, continuous method improvements, and minimal financial costs and time resources.

To ensure the best methods of work and quality assessment, it is crucial to select appropriate measures. Therefore, we advocate for the widespread use of spatial accuracy indices, particularly in situations where the use of performance metrics alone at the pixel level may lead to incorrect interpretations, as we hypothetically presented.

Regarding our study area, which is integrated into PNA, no other studies monitor territorial forest cover changes, regardless of the CLC database. CLC is only tested for thematic accuracy [107]. Our study reveals that the forest area decreased continuously in two time sequences (2005–2014, 2016–2019) within the analyzed time frame of 17 years (2003–2019), resulting in a net loss of 577 ha, which equates to a 9% decrease in forestation.

Of all the methods used, SVM and RF estimate this situation the best. SVM stands out by a consistent behavior, slightly better than RF, as demonstrated by the obtained performance scores and by the fact that it highlights the reduction in forested areas more effectively. RF responds better to changes in entry data, indicating that its performance can be improved.