Land Cover Mapping with Convolutional Neural Networks Using Sentinel-2 Images: Case Study of Rome

Abstract

Land cover monitoring is crucial to understand land transformations at a global, regional and local level, and the development of innovative methodologies is necessary in order to define appropriate policies and land management practices. Deep learning techniques have recently been demonstrated as a useful method for land cover mapping through the classification of remote sensing imagery. This research aims to test and compare the predictive models created using the convolutional neural networks (CNNs) VGG16, DenseNet121 and ResNet50 on multitemporal and single-date Sentinel-2 satellite data. The most promising model was the VGG16 both with single-date and multi-temporal images, which reach an overall accuracy of 71% and which was used to produce an automatically generated EAGLE-compliant land cover map of Rome for 2019. The methodology is part of the land mapping activities of ISPRA and exploits its main products as input and support data. In this sense, it is a first attempt to develop a high-update-frequency land cover classification tool for dynamic areas to be integrated in the framework of the ISPRA monitoring activities for the Italian territory.

1. Introduction

Background

The availability of updated and reliable land cover (LC, all the acronyms and abbreviations are reported in Abbreviations part) and land use (LU) data is a crucial element in monitoring the effect of human activity on the territory, on ecosystems and on their ability to provide ecosystem services, in order to guide decisions for sustainable soil management [1].

In Europe, the Copernicus Land Monitoring Service (CLMS) [2] offers users and stakeholders a wide range of land cover and land use products to support land monitoring activities (

Table 1. Copernicus Land Monitoring Service land cover and land use data with reference to the Global, Pan-European and Local component; LC = Land Cover classes, LU = Land Use classes, CLMS = Copernicus Land Monitoring Service.

	Name	Data Type	Classes	MMU
CLMS Global Component	Global Land Cover	Raster	23 (LC)	Pixel 100 × 100 m
CLMS Pan-European Component	CLC Plus Backbone	Raster	12 (LC)	Pixel 10 × 10 m
	CORINE Land Cover	Vector	44 (LC, LU)	25 ha (status)
				5 ha (changes)
CLMS Local Component	Coastal Zones	Vector	55 (LC, LU)	0.5 ha
	Natura 2000
	Riparian Zones
	Urban Atlas		27 (LC, LU)	0.25 ha (class 1)
				1 ha (class 2–5)

).

As part of the Global component of the CLMS, a 100-m-resolution annually updated global LC map is provided for the period 2015–2019 [3]. The CLMS pan-European component includes the CORINE Land Cover (CLC, vector data updated every 6 years, with a minimum mapping unit of 26 hectares and a 33-class land cover and land use classification system) [4], the four High-Resolution Layers raster (with a resolution of 10 m for the land cover classes of artificial surfaces, forest, grassland and water and wetness) [5], and the new CLC Plus Backbone, which offers a 10-m-resolution LC mapping in raster format for 2018 [6]. As part of the CLMS Local component, the Urban Atlas, Riparian Zones, Natura 2000 and Coastal Zones [7] layers are available, which offer high spatial resolution mapping for specific areas of the European territory (in order, urban areas, riparian zones, Natura 2000 protected areas and a buffer zone of 10 km of coastline). The CLMS Local data adopt an LC and LU classification system in compliance with the Mapping Assessment of Ecosystem Services (MAES) ecosystem types.

These data are reliable and widely used for land monitoring but have limitations in terms of low update frequency and classification system with mixed land cover and land use classes. In this perspective, the growing availability of satellite data has contributed to the development of new mapping techniques [8] which constitutes an important support tool for the creation of new land monitoring products. Actually, there are numerous experiences of LC and LU mapping by exploiting the 10-m-resolution multispectral information of the Copernicus Sentienel-2 optical satellite and the C-band Sentinel-1 SAR data [9,10,11,12]. Among the main products with European coverage, Malinowski et al. [13] proposed Sentinel-2 based 14-class land cover and land use classification for 2017, ESA also introduced Sentinel-1 data for a 12-class land cover classification [14] while Venter and Sydneham [15] integrated Sentinel data with the Land Use and Cover Area frame Survey (LUCAS), producing a 2018 map with 90% accuracy. Also of note are global products such as the ESA WorldCover [16,17], the Finer Resolution Observation and Monitoring of Global Land Cover (FROM-GLC10) [18] or the Esri 2020 LULC map [19].

However, the land cover classification of Remote Sensing images is extremely challenging, due to the wide variety of anthropogenic and natural elements (buildings, roads, trees, agricultural activities, water bodies) that correspond to different colors, shapes and textures. Particularly important and complex is also the production of tools for the detailed and high-frequency monitoring of dynamic contexts, such as peri-urban and suburban areas, and of impactful and rapid phenomena such as illegal building and construction sites.

Image processing methods have improved in recent years [20]. In many studies, deep learning has been shown to be useful for land cover classification [21], with successful applications in urban areas [22,23], agricultural land [24], forestry [25,26] and wetlands [27,28].

Deep learning models used for land cover classification are based on Artificial Neural Network algorithms [29], which include Convolutional Neural Networks (CNNs) [30,31], Recurrent Neural Networks (RNNs) [32] and Generative Adversarial Networks (GANs) [33]. CNNs are currently the most commonly used in remote sensing image processing [34,35,36], and its structure consists of an input layer, several convolutional and pooling layers and a final output which is used for the classification of the image [37].

CNNs have proven to be effective alternatives to traditional Machine Learning methods such as Decision Trees [38], Random Forests [39], Support Vector Machines [40] and K-Nearest Neighbors [41]. Using CNNs, information can be learned automatically and directly from data, without requiring human intervention, allowing also for solving complex and non-linear problems [42,43]. They have better performances than ML since they can handle a larger amount of data faster and adapt to changes [44]. Furthermore, convolutional neural networks are more generalized than traditional machine learning algorithms, making them suitable for a wide range of tasks, as well as performing well on new data [45]. Finally, CNNs are able to learn from the spatial context of the data [46,47].

There are a variety of CNN architectures that can be used for LC classification, such as the VGGNet [48], AlexNet [49], ResNet [50], DenseNet [51] and U-Net [52]. Several of these algorithms have been used successfully on RGB [53,54], multispectral [55,56] and hyperspectral data [57,58,59], and the state of the art reveals many publications regarding classification of not freely available high-resolution images [54,60,61]. However, various DL methods were recently utilized to generate LU and LC maps using free data, which are receiving particular attention because their high temporal resolution allows for significant cost savings in high-frequency monitoring [62,63,64], e.g., Hu et al. [65] used Landsat-8 images for wetland cover classification with a VGG, Di Pilato et al. [66] focused their study on the detection of changes in urban areas using a CNN on Sentinel-2 images, Mirmazloumi et al. [12] developed a workflow to generate a LULC map of Europe using Sentinel and Landsat-8 images.

The Higher Institute for Environmental Protection and Research (ISPRA) produces and updates tools for mapping and monitoring land cover, land use and land consumption for the Italian territory. ISPRA is responsible together with the National Environmental Protection System (SNPA) for updating the National Land Consumption Map (LCM) [1,67], a nationwide 10-m-resolution raster that maps land consumption with a tree-level classification system. The LCM is updated annually through photo-interpretation of the entire national territory with the aid of support layers obtained through the classification of Sentinel data [68]. ISPRA also produces land cover and land use maps with an EAGLE-compliant classification system integrating Copernicus data [69,70] and develops innovative methodologies for the classification of land cover and land consumption by the classification of satellite images [71], also through deep learning techniques [54].

This research aims to define a methodology that integrates as much as possible with the main ISPRA land monitoring products, in order to introduce valuable high-update-frequency land mapping tools to the land monitoring framework already in place for the Italian territory. This study shows the first results of land cover mapping by applying three CNN imaging techniques to Sentinel-2 multispectral images in order to analyze their performance in the production of classifications on the Italian territory. The VGG16 [72,73,74], DenseNet121 [61,75] and ResNet50 [72,73] algorithms were compared for the identification of five EAGLE-compliant land cover classes with respect to the classification of a single image and multi-temporal data.

2. Materials and Methods

2.1. Study Area

The considered study area is the Municipality of Rome located in Central Italy, in the Latium region (Figure 1).

The municipality is located in the Tiber valley and is spread over an area of 1287 km². Rome is the largest municipality in Italy and is one of the largest European capitals, with a territory with a low population density due to the presence of numerous green urban areas. The territory has reliefs and hills alternating with flat areas and extends to the sandy coasts on the Tyrrhenian Sea. According to the Köppen climate classification, Rome belongs to the Csa zone, i.e., a temperate climate with hot summers. The average annual rainfall is around 800 mm in the thirty years 1971–2000, with maximum peaks in the months of November, December and April.

Initially a portion of the municipality with an area of 64 km² was selected for the comparison between different deep learning algorithms and different input data. For this purpose, the area was divided into two parts, the larger (indicated with an “B” in Figure 2) was used for the predictive model training phase, while the smaller (indicated with a “A” in Figure 2) was used in the validation phase. In the following, the two areas will be called the “training area” and “validation area”, respectively.

The best combination of the input data and algorithm was then applied to the land cover mapping of the whole municipality of Rome in the testing phase.

2.2. Overview

The proposed research applies deep learning algorithms on Sentinel-2 images for the production of a land cover map based on an EAGLE-compliant land cover classification system [71,76]. Figure 3 shows the main phases of the methodology, which foresees a first phase of data collection and pre-processing, followed by the modeling phase. Modeling consists of a model-training phase, a validation phase and a final application to the area of interest through the testing phase.

In the study, three CNN algorithms were tested on different sets of label data in order to identify the most promising configuration, which was then applied to the prediction area and used for the land cover classification of the whole municipality of Rome.

The process was developed using the Python-friendly open-source library TensorFlow (https://www.tensorflow.org/, accessed on 15 February 2023), and the accuracy assessment was based on the Olofsson methodology [77].

2.3. Data Collection and Pre-Processing

The 10 Sentinel-2 bands with a spatial resolution of 10 and 20 m were selected with reference to 22 March, 25 July and 8 October 2019. Data were downloaded through the Semi-Automatic Classification Plugin for QGis [78], considering atmospherically corrected Surface Reflectance Level-2A images with low cloud cover. The bands were stacked to create three mono-temporal images for the three reference dates (with 10 bands each) and merged into a fourth 30-band multitemporal image, according to the procedure described by Wakulinska [62].

A PCA (principal component analysis) was used to reduce the dimensionality of the data to three spectral bands, in order to optimize the processing times and the complexity of the models. In order to create the training and validation datasets, a test area of 64 km² was defined and then divided into two sections, which were used for the training phase and for the validation phase (Figure 2).

2.4. Land Cover Classification System

Table 2. EAGLE-compliant classification system adopted for land cover mapping activities.

	I Level		II Level
1	Abiotic non-vegetated surfaces	1.1	Artificial abiotic
		1.2	Natural abiotic
2	Biotic vegetated surfaces	2.1	Woody vegetation
		2.2	Herbaceous vegetation
3	Water surfaces

shows the classification system adopted for the land cover classification described in this research, which is based on the EAGLE Land Cover Components [76] and coincides with the second level of the land cover classification systems used for other ISPRA activities [1,69,70,71].

The first classification level distinguishes three main land cover classes, two of which are further developed at the second classification level.

• Abiotic non-vegetated surfaces include any unvegetated surfaces, with or without anthropogenic influence or impact, either covered with man-made artificial structures or natural materials. At the second classification level, the class is subdivided into artificial abiotic surfaces (i.e., permanent and reversible consumed land, according to the definition of ISPRA-SNPA [1]) and natural abiotic surfaces (i.e., any kind of surface in its natural form, either with or without anthropogenic influence, such as unvegetated rocky areas, sand, bare soil).
• Biotic vegetated surfaces include any surface with spontaneous, semi-natural or artificial vegetation, with or without anthropogenic influence. At the second classification level, woody vegetation and herbaceous vegetation are distinguished [76].
• Water surfaces include water in its liquid or solid state of aggregation, both of artificial origin or natural formation (water basins, rivers, streams, stagnant waters, glaciers).

2.5. Modeling

2.5.1. Deep Learning Algorithms

Three of the most popular and efficient CNN algorithms were selected with the aim of evaluating the best-performing models:

• DenseNet-121

The basic idea of a DenseNet is similar to ResNet: connect shallow layers with deep layers using shortcuts. As a matter of fact, DenseNet architectures [51] modify the standard CNN structure by building shorter connections between each layer. In order to maximize the flow of information, DenseNets connect every layer to every other one, meaning that the input of a layer is the result of concatenation of feature maps from previous ones. In this work, a DenseNet121 with a depth of 121 layers was employed, as shown in Figure 4 [75,79].

• VGG-16

VGG is a standard deep CNN introduced by Simonyan et al. [48]. VGG consists of several configurations (VGG-13, VGG-16, VGG-19, etc.), each following the same generic design. In a VGG, an image is passed through a stack of convolutional layers with very small filters to extract features. These architectures are based on the basic structure of convolutional neural networks. The VGG-16 used consists of 13 convolutional layers and 3 fully connected layers. The complexity of the VGG network can be increased by adding more convolutional layers [8,48]. Figure 4 displays the VGG-16 architecture.

• ResNet-50

ResNet (Residual Network) and all its variants (ResNet-18, ResNet-34, ResNet-50, ResNet-101 and ResNet-152) were proposed in 2015 by He [50] with the aim to reduce the problem of vanishing gradients. It was partially solved through “shortcut connections”, a technique used to create connections between layers that are not directly adjacent [54,75]. Figure 4 shows the ResNet-50 architecture.

Each of the three classification algorithms was applied to the three single-date multiband rasters and to the multitemporal multiband raster. In the following, each pairing between a certain multiband raster and a certain classification algorithm is defined as an “experiment”, and overall, 12 experiments were performed (

Table 3. Main characteristics of the experiments performed by applying the three classification algorithms to the four multiband rasters.

Experiment	Type	Date	Algorithms	N° Bands
1	single-date	22 March 2019	DenseNet121	10
2		25 July 2019
3		8 October 2019
4		22 March 2019	VGG16	10
5		25 July 2019
6		8 October 2019
7		22 March 2019	ResNet50	10
8		25 July 2019
9		8 October 2019
10	multi-temporal	22 March 2019/25 July 2019/8 October 2019	DenseNet121	30
11			VGG16
12			ResNet50

).

2.5.2. Predictive Model Training, Validation and Testing

In the training phase, the labelling activity was carried out through the LC Map produced and updated by ISPRA-SNPA, which is available in vector format only for the municipality of Rome. The LC map was reclassified according to the classes of Table 2, and 150,000 training areas were extracted from the LC Map to create the training dataset; moreover, vector elements were combined with ground truth provided by Sentinel-2. A CNN patch training was involved. This method is common in land cover applications and trains a convolutional neural network on a large dataset of small land cover patches rather than training on batches of individual pixels.

At first, a simple 2D CNN was applied to each of the 12 experiments in order to evaluate which provides the best results. Typical CNN models include an input layer, a series of convolutional and pooling layers, and an output layer [37].

The training procedure consisted of 100 epochs of iteration. The model was compiled using an Adam optimizer [80], which iteratively updates network weights by using training data rather than the traditional stochastic gradient descent method [81] with a learning rate of 0.0001. The model output consists of a vector with one entry for each class. Using the softmax activation function, the output is normalized, and a probability distribution is generated. Next, a weighted cross-entropy was chosen and calculated as the network loss function. In addition, batch normalization was implemented to improve the model’s accuracy and to reduce training time and overfitting [82].

In order to select the best experiment, three of the most common DL metrics were calculated on the land cover products obtained in the validation phase (Equations (1)–(3)):

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}{\mathrm{T}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}+\mathrm{F}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e} \mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}$$

(1)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}{\mathrm{T}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}+\mathrm{F}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e} \mathrm{N}\mathrm{e}\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}$$

(2)

$$\mathrm{F}1 \mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=2\times \frac{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}+\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}$$

(3)

The experiment that showed the best results was used to map the land coverage of the entire territory of the municipality of Rome in the testing phase.

2.6. Accuracy Assessment of the Land Cover Map of Rome

The land cover map of the municipality of Rome was validated with the photointerpretation of a sample of points, which was determined using Olofsson methodology [77].

Quantitative evaluation was conducted evaluating the Overall Accuracy (OA) for the overall classification (Equation (4)) and the User Accuracy (PA) and Produser Accuracy (UA) for each land cover class (Equations (5) and (6)).

$$\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l} \mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\left(\mathrm{\%}\right)=\frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}}\times 100$$

(4)

$$\mathrm{s} \mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\left(\mathrm{\%}\right)=\frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} \mathrm{i}}{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{s} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} \mathrm{i}\left(\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{r}\mathrm{o}\mathrm{w}\right)}\times 100$$

(5)

$$\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{d}\mathrm{u}\mathrm{c}\mathrm{e}\mathrm{r}’\mathrm{s} \mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\left(\mathrm{\%}\right)=\frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} \mathrm{i}}{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{k}\mathrm{n}\mathrm{o}\mathrm{w}\mathrm{n} \mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} \mathrm{i}\left(\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{c}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{n}\right)}\times 100$$

(6)

Accuracy was also calculated by deleting a strip of edge pixels from each patch (“erosion”) to estimate the impact of mixed pixels on overall accuracy. The calculation of the sample size is reported in Appendix A. The accuracy assessment was conducted for the single-date and multi-temporal maps, both in the original version and in the version with the erosion of a row of pixels at the edges of the patches.

3. Results

3.1. Results Overview

This section presents the results of the algorithm validation and testing activities. The validation phase is carried out by applying the 12 experiments on the validation area, and the accuracy metrics of the resulting classifications are reported. In the validation phase, the VGG16 algorithm applied to the summer image (Experiment 5) and to the multi-temporal one (Experiment 11) give the best results. Such configurations were used in the subsequent testing phase for the production of two land cover maps of Rome, which were then subjected to accuracy assessment through the Olofsson methodology.

3.2. Validation

Table 4. Overall accuracy (OA), Precision, Recall and F1-Score of the validation phase for each of the 12 experiments.

Experiments	Type	Algorithms	OA	Precision	Recall	F1-Score
1	single-date	DenseNet121	0.67	0.59	0.63	0.59
2			0.72	0.65	0.68	0.64
3			0.70	0.63	0.67	0.63
4		VGG16	0.75	0.68	0.73	0.68
5			0.80	0.75	0.80	0.76
6			0.74	0.66	0.68	0.65
7		ResNet50	0.67	0.60	0.62	0.58
8			0.64	0.57	0.58	0.56
9			0.63	0.58	0.62	0.58
10	multi-temporal	DenseNet121	0.69	0.59	0.70	0.60
11		VGG16	0.87	0.79	0.88	0.77
12		ResNet50	0.67	0.62	0.64	0.62

shows the results of the accuracy assessment carried out on the validation area, with reference to the application of the 12 experiments.

Comparing the metrics and the maps for the twelve experiments, the best results are obtained with the VGG16 algorithm (Figure 5).

This algorithm performs better than both ResNet50 and DenseNet121 with multi-temporal data (experiment 11) and with each of the three single-date data experiments (experiments 4–6). With reference to the input data, the best results are achieved using multi-temporal data (experiments 10–12), while among the single-date images the summer image (25 July 2019, used for experiments 2, 5 and 8) shows the best classification.

3.3. Testing

Experiment 5 (Figure 6) and Experiment 11 (Figure 7) were selected for the production of the land cover map of Rome (LCR).

The accuracy assessment carried out with the Olofsson methodology provided the results of

Table 5. Accuracy assessment of the LCR. The results in single dates refer to the classification based on the summer image of 25 July 2019. P.A. = Producer Accuracy, U.A. = User Accuracy, O.A. = Overall Accuracy.

Land Cover Class	Single Date				Multi-Temporal
	P.A.	U.A.	O.A.	O.A. Erosion	P.A.	U.A.	O.A.	O.A. Erosion
Abiotic surfaces	0.75	0.73	0.62	0.71	0.78	0.80	0.59	0.76
Artificial abiotic	0.63	0.77			0.62	0.72
Natural abiotic	0.39	0.23			0.67	0.45
Vegetation	0.81	0.87			0.78	0.88
Woody vegetation	0.62	0.52			0.49	0.50
Herbaceous vegetation	0.58	0.73			0.55	0.64
Water surfaces	0.94	0.66			0.96	0.47

.

The single-date classification shows higher accuracy values than the multi-temporal one in all classes, except for the natural abiotic surfaces. By eliminating edge pixels from each patch (erosion), an increase in overall accuracy is observed in both single-date and multi-temporal classification.

4. Discussion

The methodology compares the three CNN algorithms VGG16, DenseNet121 and ResNet50 for land cover mapping according to five EAGLE-compliant land cover classes through Sentinel-2 multispectral image classification. The methodology also refers to the land monitoring activities carried out by ISPRA for the national territory, in an attempt to enhance its standards and products for the creation of new land mapping tools.

In detail, three Setinel-2 acquisitions relating to spring (22 March 2019), summer (25 July 2019) and autumn (8 October 2019) were selected, processed individually and combined into a stack, then the three algorithms were applied to the four datasets, obtaining 12 land cover classifications. The accuracy assessment of single-date classifications shows that all three algorithms are affected by seasonal land cover variations. For the DenseNet121 and the VGG16, the best results were related to the summer image, while for the ResNet50, spring classification was slightly better. In all three cases, the multi-temporal classification shows results in line with the best single-date one, demonstrating that the models work well even on a single date, with positive implications in terms of time and computational costs. On the other hand, the best result relates to the summer period, which is the same as the ISPRA data used to extract the training areas; therefore, to obtain products with high update frequency (less than one year), it will be necessary to take into account the seasonal variability of the land cover in the selection of the training dataset, especially in agricultural areas and those with mixed vegetation. The VVG16 algorithm gave the best results in the training and validation activities and was used for the land cover mapping of the municipality of Rome, with respect to the summer image (experiment 5) and the multitemporal stack (experiment 11).

The accuracy assessment carried out on the two land cover maps of Rome shows better results as regards the classification on a single date (which reaches an overall accuracy of 62%), while the land cover map based on the multi-temporal stack reaches 59%.

The algorithm is efficient in distinguishing the three macro-classes of Abiotic surfaces, vegetation and water bodies, while for analyzing the second classification level, natural abiotic surfaces have the highest omission and commission errors due to the small number of training areas, which limits the effectiveness of the algorithm (Figure 8).

Most of the herbaceous vegetation omission errors (which instead has few commissions) are mainly located in peri-urban and rural areas, above all on the edges of the shelterbelt of trees and riparian vegetation, due to the woody vegetation commission errors. Most of the woody vegetation omissions are concentrated in urban areas, particularly in small green urban areas (for example, sparse trees alternating with a herbaceous cover) and in rows of trees. Permanent water bodies show few omission errors, while commission errors are concentrated at the edge of the water bodies and on small patches of arable land.

Most of the classification errors occur on the edges of the classes and are due to the spectral signature of the mixed pixels. These errors affect both the single-date classification (which, however, has fewer errors on woody vegetation) and the multi-temporal one. A second assessment conducted by excluding the border pixels of all patches shows a significant increase in overall accuracy, which reaches 71% for single-date classification and 76% for multitemporal classification.

The artificial abiotic surfaces show good results, commission errors mainly concern the bare soil in the agricultural area and omissions are mainly located in urban areas with a simultaneous presence of many small patches of different land cover classes (such as tree-lined streets) and along the road network.

This methodology is based on Sentinel-2 data and adopts an EAGLE-compliant classification system, resulting in it being integrated well into the Copernicus framework and suitable for a wide range of improvements, such as the introduction of Sentinel-1 SAR data or the expansion of the classification system (e.g., distinguishing trees from shrubs and separating permanent and periodic herbaceous vegetation), evaluating the seasonal variability of the land cover and integrating high-resolution or hyperspectral data into the methodology. It will also be important to evaluate the sensitivity of the algorithm with respect to the use of a higher number of Sentinel-2 spectral bands.

The map is more accurate in urbanized areas, since the ISPRA land cover map of the municipality of Rome used to extract the training areas represents the consumed land with more geometric detail than the non-consumed land [83]. Anyway, the use of ISPRA products for the extraction of training areas allows for progressively improving the reliability of the training dataset without additional photo-interpretation costs, since the annual and highly accurate update of the land cover and land consumption maps is one of the institutional tasks of ISPRA and SNPA.

Already in this preliminary phase, the map shows good results in single-date classification, with notable positive implications in terms of the timeliness of updates and reduction in computational costs. In this sense, the methodology constitutes a first attempt to create new tools for high-frequency land monitoring in critical areas, for example, with reference to the monitoring of urban densification phenomena and the management of green spaces included in the consolidated urban fabric.

5. Conclusions

This research compares for the first time three CNN algorithms for the definition of a land cover mapping methodology that exploits the data produced by ISPRA for Italy to optimize the collection of training areas. Actually, compared to other traditional or machine-learning-based classification techniques, CNNs require little pre-processing of the data and marginal human intervention once the model is calibrated. However, the creation of the training dataset is cumbersome and delicate, since the correct training of the algorithm requires a large number of well-selected training areas, whose collection is generally carried out through photo-interpretation.

The use of the ISPRA land cover map significantly optimizes this onerous phase, since a dataset is exploited which is already physiologically subject to manual updating and which must guarantee high accuracy. In this way, a training dataset that is reviewed and updated every year is obtained, which allows the production of a land cover map on an annual basis with limited training area selection efforts. Achieving higher update frequencies requires an in-depth analysis of the selection of training areas relating to the classes most subject to land cover variations during the year, such as agricultural areas, so that the algorithms understand their behavior during the year. Once the reasonably achievable update frequency has been defined on the basis of data availability, optimization of computational costs and adaptation of training areas to seasonal variations, the methodology will be able to support high-frequency monitoring of rapid and impactful phenomena, such as large infrastructure construction sites, illegal building or forest disturbances. Anyway, already in the preliminary phase, the results are encouraging to consider applications to the monitoring of urban areas. By 2050, more than 75% of the world’s population will live in urban areas, with repercussions in terms of exposure to extreme events, significant environmental impacts and lower resilience in dealing with the climate crisis and phenomena such as the urban heat island. Controlling land transformations in urban areas is therefore essential because it affects people’s health and well-being. This theme is included among the priorities of the 7th of the 8th Environmental Action Programme, is included in the “Biodiversity Strategy for 2030”, is the theme of the UN Sustainable Development Goals number 11 and requires the adoption of effective policies based on reliable and timely observations. Actually, the methodology stands as a promising tool to deepen the monitoring of urban areas and in particular the green infrastructures in urban areas and the related ecosystem services, the evaluation of the cooling index, the SDG 11.7.1 [84] associated with accessibility, and maps of vulnerability in terms of physical access to green spaces. Experiences such as the recent “Dynamic world” [85] show the potential of deep learning techniques in land cover mapping, even on a large scale and with high update frequency. With appropriate refinements in the selection of training areas, the methodology will be also improved for high-frequency monitoring of peri-urban and rural areas, with a view to maximizing integration with other land monitoring tools available nationwide.