Earth Observation Data and Geospatial Deep Learning AI to Assign Contributions to European Municipalities Sen4MUN: An Empirical Application in Aosta Valley (NW Italy)

Abstract

Nowadays, European program Copernicus’ Sentinel missions have allowed the development of several application services. In this regard, to strengthen the use of free satellite data in ordinary administrative workflows, this work aims to evaluate the feasibility and prototypal development of a possible service called Sen4MUN for the distribution of contributions yearly allocated to local municipalities and scalable to all European regions. The analysis was focused on the Aosta Valley region, North West Italy. A comparison between the Ordinary Workflow (OW) and the suggested Sen4MUN approach was performed. OW is based on statistical survey and municipality declaration, while Sen4MUN is based on geospatial deep learning techniques on aerial imagery (to extract roads and buildings to get real estate units) and yearly Land Cover map components according to European EAGLE guidelines. Both methods are based on land cover components which represent the input on which the financial coefficients for assigning contributions are applied. In both approaches, buffers are applied onto urban class (LC_b). This buffer was performed according to the EEA-ISPRA soil consumption guidelines to avoid underestimating some areas that are difficult to map. In the case of Sen4MUN, this is applied to overcome Sentinel sensor limits and spectral mixing issues, while in the case of OW, this is due to limits in the survey method itself. Finally, a validation was performed assuming as truth the approach defined by law as the standard, i.e., OW, although it has limitations. MAEs involving LC_b, road lengths and real estate units demonstrate the effectiveness of Sen4MUN. The developed approach suggests a contribution system based on Geomatics and Remote sensing to the public administration.

1. Introduction

The European Space program Copernicus, with its Sentinels missions, has allowed the creation of many research projects and prototypes aiming at developing several services, many of them already available based on earth observation data [1,2,3,4,5,6]. Unfortunately, there are still few available or prototype services based on an applied use of geomatics and remote sensing, despite an expansion of Space Economy. The distribution systems of regional contributions to municipalities are overwhelmingly based on territorial components. Therefore, the geospatial component and the technological transfer offered by the growing development of geomatics and remote sensing in this sector would be enormous, allowing for important impacts on administrative processes currently carried out with systems that are often onerous, as well as poorly efficient and inaccurate. To achieve real digitalization in public administration is necessary to develop services capable of responding to ordinary questions in a modern way, guaranteeing scalability and a certain degree of standardization. In fact, nowadays there is a growing need of standardization and validation procedures of geographical and earth observation data/product/services with special concerns about the expected roles they can have within monitoring/control actions from institutional subjects (e.g., CAP controls. natural hazards, etc.).

One of these, still under development, and one of the few that aims to use remote sensing in a highly economic key is the Sen4CAP system for the payment of contribution and control in agriculture, according to the Common Agricultural Policy (CAP) in light of EU regulation (N. 746/2018) [7,8,9]. The EU’s Common Agricultural Policy (CAP) seeks to raise agricultural productivity in Europe in a sustainable manner while maintaining a respectable level of living for EU farmers. The CAP, which has an annual budget of about EUR 59 billion, uses a variety of strategies such as direct payments, market reforms, and rural development, to improve the sustainability and competitiveness of agriculture in Europe [10]. The Integrated Administration and Control System (IACS) oversees and manages the majority of the CAP budget with the goal of protecting the program’s finances and assisting farmers in submitting their declarations. [11]. Earth observation (EO) satellites are regarded to play a bigger part in the CAP reform satellite’s enhancement and cost-effectiveness of the IACS. It is important to note that the CAP reform has established the role of EO data as being of utmost importance. Their adoption is expected to be required in 2024–2025, and may also extend to future planned missions like IRIDE, which the Italian Space Agency (ASI) and the European Space Agency (ESA) are working on together. Sentinel missions—Sentinel-1 and Sentinel-2 (hereinafter called Sentinels) in particular—play a major role. In the years after 2020, the initiative will give special consideration to demonstrating how data obtained from Sentinels may help modernize and streamline the CAP. Sen4CAP was established by the ESA in response to requests from European payment agencies like other EO services [12,13,14].

Also, other space agencies worldwide like NASA (US), JAXA (Japan), CNES (France), DLR (Germany), and more recently the ASI (Italy) with the ambitious program IRIDE, have developed or have planned to strengthen their services addressed to the private and public sector, not only for research purposes, but to promote technological transfer in many sectors [15], but at the present time fewer are related to contribution systems in different sectors which play a huge role. The space race has opened up new frontiers of investment for individuals and enterprises at different levels, not only the large economic giants of the ICT and geospatial services such as Planet, SatVu, Albedo, Maxar, e-Geos, etc. Despite several services being available from forest management monitoring to precision agriculture, to the management of migratory flows passing through the management of urban areas and planning of smart cities and much more [16], almost none are related to contributions system. The reason is related to the fact that, on one hand, many services and products offered are often too general and not specific, or worse, without solid scientific validation. On the other hand, public administration is too slow to embrace the new opportunities offered by applied sciences. It is often incapable of understanding them because they are not transferred in the correct way or suitable for use as a support and then a replacement for old methods without creating big changes (especially when it comes to money). Added to all this is bureaucracy and the need to legislate on new approaches so that they become operational and can complement and replace conventional ones. It is therefore not surprising that more bureaucratic countries are often slower to assimilate the news. Added to this is the need for foresight on the part of the political sector and of regional managers and technicians capable of understanding and getting involved, abandoning safe but often obsolete paths to support new ones that require hard work, experimentation, and continuous understanding [17].

Therefore, this work aims to create a tentative service based on European EO data and geospatially based processing about the distribution of contributions intended for municipalities called Sen4MUN, where Sen means Sentinels for municipalities. It is worth noting that in Italy, as in other EU countries, the municipalities receive income from different types according to different criteria [18]. Current incomes of taxes, contributions, and equalization nature are made up of four income items: taxes, duties, and similar income; tax sharing; equalization funds from central administrations; equalization funds from the region or autonomous province. Generally, equalization funds from central administrations and regions or autonomous provinces are the most important and the main core of the approach suggested [19,20]. Concerning contribution delivered by regions to municipalities, they are assigned following two possible approaches; the first one is based on the estimated expenditure, taking into account the previous year (the more you spend the more you receive money for local development within a budget defined at the regional and governmental level); the second is a little more rigorous, based on indicators of territorial development (in which, at least in Italy, the more the urbanized area develops, roads, housing units, factories and more in general areas for urban use, the higher the revenues) as a function of the resident population without going into the criticisms of the present approaches, which certainly should be reviewed from the point of view of environmental sustainability dealing with SDGs goals [21]. Since the second approach permits an exploitation of the possibility offered by EO data and GIS updated data; a system similar to the idea at the base of Sen4CAP has been suggested in this work. Urbanized components, like other territorial bio-physical surfaces, can potentially be mapped to the temporal resolution of Sentinels by translating the information produced by a medium-high resolution land cover into a datum that can be used by administrations at an economic level [22]. To date, the monitoring of land cover is carried out only from an environmental point of view (to map the territory or quantify land consumption or for other research purposes), but without, up to now, deriving any form of economic quantification. To try to bridge this gap, the Sen4MUN has been designed.

Sen4MUN aims to create a single and standardized approach for the item concerning government revenue or, as in the case of this study, regional revenues for local authorities considering also the environmental issues. Sen4MUN aims to replace the approach currently based on the estimation of territorial indicators obtained from statistical analyzes or very rough estimates used up to now by suggesting a more rigorous, efficient, and objective approach, based on the technology transfer offered by EO data trying to suggest new approaches by space economy and the actual and expected roles of geomatics within the next generation EU framework from science to public services.

In this regard, therefore, to strengthen the use of free European satellite data in Ordinary administrative workflows such as Sen4CAP, this work has been focused to assess the feasibility and prototypal development of a possible service called Sen4MUN for the distribution of contributions yearly allocated to local municipalities and scalable to all European regions.

The analysis was focused on Valle d’Aosta Region, North West Italy, considered complex in land cover mapping because of its geo-morphology and Sentinels geometrical resolution limits in alpine areas linked to SAR distortions [23] and spectral mixing, as well as phenology detection in pixels located in high slope degree in multispectral remote sensed data [24].

A comparison between the ordinary methodology based on the estimation of territorial indicators (OW), with the buffer zone retrieved with statistical surveys and the Sen4MUN approach, which is based on yearly land cover classification according to EAGLE guidelines, has been performed. Finally, due to the fact that some roads cannot be correctly mapped by Sentinels due to a GSD limiting factor, updated GIS geodatabases were included in the Sen4MUN prototypal approach.

2. Materials and Methods

2.1. Study Area

Sen4MUN was realized and tested within the Aosta Valley Region in the North West of Italy. Aosta Valley is a region in northwestern Italy which hosts the highest peaks in the whole alpine chain and in general in Europe. The region borders France in the Western part and Switzerland in the Northern. Figure 1 has reported Aosta Valley’s location in respect to Italy. Though it is the smallest in Italy, its geomorphology makes it one of the most complex. It is situated in the Western Alps [25].

2.2. Ordinary Workflow

This approach adopts municipality declaration and statistical data obtained from regional or national offices that starts from building practices or territorial surveys performed within a time range (for example every 5 years [26]. In the most likely cases, data comes from GIS databases (like the cadastre of buildings real estate units and municipalities streets lengths), and the land cover areas divided into three marco-classes derived from municipality declarations. These three land cover components are: urban areas which are the most important, semi-anthropic areas, and sterile areas, buffered 10–20 or 30 m (over-estimated) due to inhomogeneity data collection through the different times [22]. Nowadays, Ordinary Workflow (hereinafter called OW) does not use deep learning processing to update the GIS geodatabase and earth observation data. In fact, almost the entire OW data collection is based on municipality declaration and statistical surveys at different level with yearly temporal gaps. This is the case of the Aosta Valley region that follows the Cerutti’s approach since 1979, which is entirely based on statistical surveys and declarations [27,28,29]. This approach considered the following patterns: the municipal area, the urban and anthropic areas, the semi-anthropic areas (that includes all vegetated areas cultivated and not cultivated by humans), the sterile areas (that means all land unproductive surfaces, like water bodies and courses, snow and ice, rocks), roads lengths, and finally real estate units. All these components are properly weighted and normalized according to financial criteria defined at regional, national, and European levels. The financial criteria are based on the resources made available by the annual regional budget on the basis of a legal resolution, the expenses made in the current year, the actual resident population and the municipal area. These indicators, based on state and regional tax formulas, which vary over time because they are the result of political economy choices, define the coefficients to be applied to the territorial development components covered by this study and which characterize the contributions paid. In fact, the financial criteria are defined and regulated by national law and regional regulations, leaving greater freedom in their calculation to regions with special statutes such as the Valle d’Aosta Autonomous Region.

Therefore, OW is based on data collection campaigns for statistical purposes and declarations from municipalities. The municipal offices are required to communicate certain parameters such as the municipal area, resident population, roads belonging to the municipality and real estate units present in the municipality. The poor communication between offices at various levels often makes these data not always representative and homogeneous in qualitative terms for all municipalities.

It is worth noting that OW is based on a simple weighted sum of factors that sum to total land cover as reported in financial land components workflow in Figure 1 which is used also in Sen4MUN using a different collection data method. In this regard, the role of remote sensing and GIS is to properly map these areas instead of using periodic surveys as it happens nowadays and in the issue previously discussed. Equations (12) and (13) are the core of this approach, and each variable cab be obtained weighing each component (representing the input variable) properly mapped.

Table 1. Ordinary Workflow equations adopted to retrieve territorial inputs for the financial contribution counting. It is worth to note that these equations are adopted also in Sen4MUN approach.

Description	Algorithm
Land Cover Areas (LCA)	d = e + g + i (1) where, e = urban and anthropic area g = semi-anthropic areas i = sterile areas d = municipality’s administrative boundaries
LCA Weights	f = e × α (2) where, e = urban and anthropic area α = weight (in this case = 3) f = urban weighted area   h = g × β (3) where, g = semi-anthropic area β = weight (in this case = 1.5) h = semi-anthropic weighted area   l = i × γ (4) where, i = sterile area γ = weight (in this case = 0.5) l = sterile weighted area
Weighted areas	m = f + h + l (5) where, f = urban weighted area h = semi-anthropic weighted area l = sterile weighted area   $n=\frac{m}{\sum _{i=1}^{n}m}\times 100$ (6) where, m = conventional municipality area $\sum _{i=1}^{n}m$ = sum of all the municipalities in the regional areas n = conventional weighted municipality area   o = n × δ (7) where, n = conventional weighted municipality area δ = weight (in this case = 50%) o = sub-conventional weighted municipality area
Roads length	$q=\frac{p}{\sum _{i=1}^{n}p}\times 100$ (8) where, p = roads length $\sum _{i=1}^{n}p$ = sum of all the municipalities in the regional areas q = roads weighted length   r = q × ε (9) where, q = roads weighted length ε = weight (in this case = 30%) r = sub-conventional weighted roads length
Real estate units	$t=\frac{s}{\sum _{i=1}^{n}s}\times 100$ (10) where, s = real estate units $\sum _{i=1}^{n}s$ = sum of all the municipalities real estate units t = real estate weighted units   u = t × ζ (11) where, t = real estate weighted units ζ = weight (in this case = 20%) u = sub-conventional weighted real estate units

shows how each variable was obtained for the only purpose of complete clarity, although the procedure is as described previously, a simple weighing of weighed remote sensed variables.

The sub-conventional parameters, respectively obtained by Equations (7), (9) and (11) are used to obtain algorithms reported in Equations (12) and (13), respectively. These are adopted by the regional Aosta Valley Autonomous offices to assess the municipality contributions according to the financial budget foreseen annually.

$$v=o+r+u$$

(12)

where,

v = sum of all sub-conventional weighted parameters,

$$z=v\times \eta$$

(13)

where,

$\eta$ = a financial weight in this case 11.50%.

2.3. Sen4MUN

The Sen4MUN approach is based on remote sensing and, in particular, yearly land cover, GIS-updated geodatabases, and deep learning to assess the parameters necessary as input in the OW.

2.3.1. Earth Observation Data and Processing

In this regard, urban and anthropic areas, semi-anthropic areas, and sterile areas are computed from Aosta Valley yearly land cover classes aggregation available at SCT regional geoportal and eoVdA webpages. Road lengths are obtained by the GIS viability monthly update geodatabase, as well as real estate units also including deep learning (this part will be further discussed). The reference year 2020 was considered. Land cover (LC) complies with EAGLE guidelines and is realized according to the following methods [30] and it is based on Sentinels missions (S1–S2) [31,32,33,34] adopting Aosta Valley land cover [25,30]. It is worth to note that, to assist the semantic and technological foundation of a European harmonized information management capability for land monitoring, the EAGLE Group has been working on a solution and proof of concept since 2008. Mostly, but not only, in their capacities as Eionet members, land monitoring specialists from many European Environment Agency (EEA) member nations have formed the self-initiated and public EAGLE Group. As a result, the EAGLE Group uses a bottom-up methodology to compile information and experiences from current land cover (LC) and land use (LU) categorization methodologies and projects. The Copernicus Land Monitoring Service recognizes EAGLE as a key and necessary element to facilitate a broad change in emphasis from categorization to characterization. This has resulted in the enforcement of EAGLE compliance in new CLMS products. The EAGLE guidelines envisage dividing the land cover components according to a pyramidal hierarchical approach, starting from macro classes to reach detailed components.

Since LC classes are more detailed than those reported in the Cerutti’s approach (which are the same at EAGLE at macro-level), they were aggregated according to the Cerutti’s description to integrate the Sen4MUN approach into the OW as follows in

Table 2. Comparison between Land Cover EAGLE and Cerutti’s classes.

Land Cover EAGLE Class	Cerutti Class
Urban and anthropic areas	urban and anthropic area
Shrubland and transitional woods	semi-anthropic areas
Woody crops	semi-anthropic areas
Water surfaces	sterile areas
Water courses	sterile areas
Needle-leaved forests	semi-anthropic areas
Broad-leaved forests	semi-anthropic areas
Mixed forests and moors	semi-anthropic areas
Permanent snow and ice	sterile areas
Natural grasslands and alpine pastures	semi-anthropic areas
Lawn pastures	semi-anthropic areas
Bare rocks	sterile areas
Discontinuous herbaceous vegetation of medium-low altitude	semi-anthropic areas
Sparse herbaceous vegetation at high altitudes	semi-anthropic areas
Alpine wetlands	sterile areas

.

2.3.2. Geospatial Deep Learning Data and Processing

Roads and real estate units were extracted both from cadastral maps and deep learning adopting open-source libraries and Python scripts integrated with ESRI ArcGIS Pro v.2.9 for object detection and classification [35,36,37,38]. Roads and building footprints [39,40,41,42,43] were extracted using Convolutional Neural Network (CNN) techniques [44,45,46,47] onto the AGEA (Agency for Disbursements in Agriculture) 2020 ortho-rectified imagery, yearly available at the national level. In particular, ArcGIS pretrained deep learning models were adopted to extract roads and real estate units (that has been assigned from cadastre and municipal declaration of habitability to building footprint extracted using these models). Concerning road extraction, the deep learning model to extract roads from high resolution satellite imagery named: Road Extraction—Global in ESRI ArcGIS Pro v.2.9 was adopted. The implementation is based on the Sat2Graph model by [48]. Sat2Graph relies on a novel encoding scheme. It is capable of creating a three-dimensional tensor from the road network graph. This capability is named as Graph Tensor Encoding (GTE). By combining the benefits of segmentation-based and graph-based techniques, this graph-tensor encoding scheme enables the training of a basic, non-recurrent neural network model to directly translate the input satellite/aerial images into the road network graph (i.e., edges and vertices). In the case of a road network graph G = {V, E} covering a region measuring W meters by H meters, GTE employs a W^−λ × H^−λ × (1 + 3 · D_max) 3D-tensor (represented as T) to hold the graph’s encoding. In this case, D_max is the maximum number of edges that can be encoded at each λ × λ grid, and λ is the spatial resolution, which limits the encoded graph so that no two vertices can be co-located within a λ × λ grid. The two spatial axes in the two-dimensional plane are represented by the first two dimensions of T. To encode the graph information, we utilize the vector at each spatial point, u_x,y = [T_x,y,₁, T_x,y,2,..., T_x,_y,(1 + 3·D_max)]^T. Its first element (vertexness), p_v ∈ [0, 1], specifies the likelihood that a vertex would exist at point (x, y). D_max 3-element groups, which encode the data of a possible outgoing edge from location (x, y), come after the initial element. The first element p_ei ∈ [0, 1] (edgeness) of the i-th 3-element group encodes the likelihood of having an outgoing edge toward (dx_i, dy_i), that is, an edge pointing from (x, y) to (x + dx_i, y + dy_i). Since vertices with degrees greater than six are extremely uncommon in road network graphs, we have set D_max to six in this instance. GTE only use the i-th 3-element group to encode edges pointing toward a 360 D_max-degree sector from (i − 1) · 360 D_max degrees to i · 360 D_max degrees in order to minimize the number of possible distinct isomorphic encodings of the same input graph. It is simple to encode a road network graph into GTE. The encoding algorithm first interpolates the straight road segment in the road network graph for the purpose of road network extraction. To keep the distance between consecutive points under d meters, it chooses the bare minimum of equally spaced intermediate spots. By controlling the edge vector’s length in GTE, this interpolation technique stabilizes the training process.

Interpolation for stacked roads may result in vertices from two overlapped road segments at the same location. When this occurs, the pre-trained model in ArcGIS permits to move the endpoint vectors of the two edges using an iterative conflict-resolution process. The objective is to ensure that the separation between any two vertices (derived from the two edges that overlap). The GTE decoding technique returns a graph’s anticipated GTE, which is frequently noisy, to the standard graph format (G = {V,E}). There are two steps in the decoding algorithm: (1) vertex extraction and (2) edge connection. It has been considered vertices and edges with a probability larger than a threshold (referred to as p_thr), since both the edgeness and verticeness predictions are real numbers between 0 and 1.

During the vertex extraction phase, the decoding algorithm locates the local maxima of the vertexness map in order to extract possible vertices. The algorithm takes into consideration just those local maxima whose vertexness exceeds p_thr. The decoding algorithm joins the outgoing edges of each candidate vertex v ∈ V to other vertices in the edge connection stage. The following distance function (reported in Equation (14)) is used by the algorithm to calculate the distance of the i-th edge of vertex v ∈ V to all other neighboring vertices u.

$$\begin{gathered} \mathrm{d}\left(\mathrm{v},\mathrm{i},\mathrm{u}\right)=\left|\left|\left({\mathrm{v}}_{\mathrm{x}}+{\mathrm{d}\mathrm{x}}_{\mathrm{i}},{\mathrm{v}}_{\mathrm{x}}+\mathrm{d}{\mathrm{y}}_{\mathrm{i}}\right)-\left({\mathrm{u}}_{\mathrm{x}},{\mathrm{u}}_{\mathrm{y}}\right)\right|\right|+ \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\mathrm{w}·{\mathrm{c}\mathrm{o}\mathrm{s}}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}}\left(\left({\mathrm{d}\mathrm{x}}_{\mathrm{i}},\mathrm{d}{\mathrm{y}}_{\mathrm{i}}\right),\left({\mathrm{u}}_{\mathrm{x}}-{\mathrm{v}}_{\mathrm{x}},{\mathrm{v}}_{\mathrm{y}}-{\mathrm{v}}_{\mathrm{y}}\right)\right) \end{gathered}$$

(14)

where w is the weight of the cosine distance in the distance function and cos_dist(v1, v2) is the cosine distance of the two vectors. In this case, we set w to a high value, like 100, to prevent erroneous connections. Once this distance has been calculated, the decoding algorithm inserts an edge between v and u 0 and selects a vertex, u 0, that minimizes the distance function d(v, i, u). In order to prevent false edges from being added to the graph when there are no suitable candidate vertices nearby, we specified a maximum distance criterion of 15 m. Finally, Sat2Graph uses cross-entropy loss (denoted as L_CE) and L2-loss. The vertexness channel (p_v) and edgeness channels are subjected to the cross-entropy loss.

p_ei i ∈ {1, 2,..., D_max}), and the edge vector channels ((dx_i, dy_i) i ∈ {1, 2,..., D_max}) get the L2-loss. GTE varies across extended road sections.

In this instance, distinct ground truth labels in the GTE format can be mapped to the same road structure. We only calculate the losses for edgeness and edge vectors at position (x, y) when there is a vertex at that location in the ground truth due to this discrepancy. The overall loss function is displayed below in Equation (15) (ground truth is represented by $\widehat{\mathrm{T}}$ ${\widehat{\mathrm{p}}}_{\mathrm{v}}$, ${\widehat{\mathrm{p}}}_{\mathrm{e}\mathrm{i}}$, ${\widehat{\mathrm{d}\mathrm{x}}}_{\mathrm{i}}$, and ${\widehat{\mathrm{d}\mathrm{y}}}_{\mathrm{i}}$).

$$\begin{gathered} \mathrm{L}\left(\mathrm{T},\widehat{\mathrm{T}}\right)=\sum _{\left(\mathrm{x},\mathrm{y}\right)\in \left[1..\mathrm{W}\right]\times \left[1..\mathrm{H}\right]}({\mathrm{L}}_{\mathrm{C}\mathrm{E}}\left({\mathrm{p}}_{\mathrm{v}},{\widehat{\mathrm{p}}}_{\mathrm{v}}\right)+ \\ +{\widehat{\mathrm{T}}}_{\mathrm{x},\mathrm{y},1}\times (\sum _{\mathrm{i}=1}^{{\mathrm{D}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}({\mathrm{L}}_{\mathrm{C}\mathrm{E}}\left({\mathrm{p}}_{\mathrm{e}\mathrm{i}},{\widehat{\mathrm{p}}}_{\mathrm{e}\mathrm{i}}\right)+{\mathrm{L}}_2(\left({\mathrm{d}\mathrm{x}}_{\mathrm{i}},{\mathrm{d}\mathrm{y}}_{\mathrm{i}}\right),\left({\widehat{\mathrm{d}\mathrm{x}}}_{\mathrm{i}},{\widehat{\mathrm{d}\mathrm{y}}}_{\mathrm{i}}\right))))) \end{gathered}$$

(15)

Concerning building footprints, these were obtained using the deep learning model to extract building footprints from high-resolution aerial and satellite imagery named: Building Footprint Extraction—New Zealand. In this last case, real estate extraction is based on the Mask R-CNN model architecture implemented using ArcGIS API for Python [49].

Developed on top of Faster R-CNN, Mask R-CNN is a state-of-the-art model for instance segmentation. A region-based convolutional neural network called Faster R-CNN [50] provides bounding boxes together with a confidence score for each object’s class identification. Mask R-CNN works on two-stage mainly based on Faster R-CNN architecture: (a) (phase 1): there are two networks in the first stage; a region proposal network and a backbone network (ResNet, VGG, Inception, etc.) to provide a collection of region suggestions, these networks execute once for each image. The feature map’s regions that contain the object are called region proposals; (b) (phase 2): The network predicts object classes and bounding boxes for every suggested region that was acquired in phase one. While fully linked layers in the networks always need a constant size vector to produce predictions, each proposed region might have a variable size. The RoIAlign technique or RoI pool, which is quite similar to MaxPooling, are used to fix the size of these proposed regions. When segmenting scenes or objects with irregular borders, segmentation models may produce boundaries that are too smooth and may not be accurate. A point-based rendering neural network module named PointRend has been included as an upgrade into the pre-trained model in order to obtain a clear segmentation border. This module provides the segmentation problem with a rendering perspective, utilizing techniques from classical computer graphics. Labels are frequently predicted by image segmentation models on a regular grid with low resolution, such as 1/8th of the input. To upscale the forecasts to the original resolution in these models, interpolation is used. PointRend, on the other hand, upscales the predictions using an iterative subdivision technique by having a trained tiny neural network predict the labels of points at certain places. This technique efficiently produces output with great resolution [51].

In this work, in order to improve the quality of the results obtained and, in particular, the extraction of a building footprint to get real estate units, PointRend has been adopted into the pre-trained model. To enable PointRend within Mask R-CNN in the ESRI ArcGIS front-end or scripting console, the following parameters has been set as reported in Equation (16):

$$\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}=\mathrm{M}\mathrm{a}\mathrm{s}\mathrm{k}\mathrm{R}\mathrm{C}\mathrm{N}\mathrm{N}\left(\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}=\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a},\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{d}=\mathrm{T}\mathrm{r}\mathrm{u}\mathrm{e}\right)$$

(16)

An 8-bit, 4-band high resolution (0.20 m GSD) aerial AGEA imagery of 2020 of the whole Aosta Valley was adopted. Due to the high computation necessary to extract the features in the whole region, the processing was conducted in a workstation with the following main characteristics: 64 GB CPU memory; 2 TB SSD storage and with a graphic card Nvidia RTX A4000 16 GB GDDR6.

Then, from the roads extracted, they were checked with the viability geodatabase, and only municipality roads and areas were considered (as specifically reported in the regional laws) for the road length computation. Then, the land cover areas and roads were buffered 20 m for the following reasons: some sparse urban areas are difficult to map due to the GSD limits of Sentinels [52,53,54,55,56,57,58,59], then this threshold is adopted both in OW procedure and ISPRA Land Units in the case of soil consumption estimation [29]. Finally, the areas obtained were used as inputs in the OW equations reported in Table 1.

2.4. Validation

The validation of the Sen4MUN approach and in particular of the surface estimated adopting GIS [60,61,62,63,64] and earth observation data to retrieve the parameters adopted in the OW was realized by computing the Mean Absolute Error (MAE) according to Equation (17) here reported:

$$\mathrm{M}\mathrm{A}\mathrm{E}=\frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}|{\mathrm{p}}_{\mathrm{i}}-{\mathrm{o}}_{\mathrm{i}}|}{\mathrm{n}}$$

(17)

where p_i is the prediction (Sen4MUN component area), o_i is the OW component areas estimated without remote sensing methods, and n is the number of municipalities in the Aosta Valley autonomous area equal to 74).

It is worth noting that the EAGLE classification starts from the same macro-classes identified by Cerutti’s. The unique difference in the two approaches is represented by the names adopted (despite the description is perfectly the same). Therefore, the EAGLE approach starts from macro to arrive to detail with sub-classes while Cerutti’s stops to macro-level.

This is a key point because it permits comparability between the two methods even if different technical approaches are followed.

In order to sum up the suggested newer approach, a workflow of the Sen4MUN approach is provided in Figure 2.

3. Results

The Sen4MUN approach was validated by computing the MAE compared to the surfaces calculated with the traditional method from statistical surveys. It is worth noting that, for the reference year 2020, the main surfaces and inputs were computed both with OW and Sen4MUN for each municipality within the Aosta Valley. Furthermore, also deep learning pre-trained models concerning on roads and real estate units have been evaluated. The pre-trained ESRI ArcGIS model Building Footprint Extraction—New Zealand has produced an F1 score of 86.20 all over the region, with an overall precision of 0.885 and a recall of 0.861. Despite many building footprints being available in ArcGIS, this is one that works better in geomorphological complex areas, while the pre-trained ESRI ArcGIS model Road Extraction—Global showed a precision score of 0.804 in the area of study. However, including results without PointRend as reported in Equation (16), it had a score of 0.873. Excluding false positives due to rocks artifacts and after a manual refining due to photointerpretation, the final layer had an overall accuracy of 0.961. It is worth noting that a 4-bands aerial imagery was adopted to perform the extraction. In particular, to refine the accuracy of the models in ArcGIS Pro in order to reach a threshold score of 0.80, hyperparameter optimization has been performed, fine-tuning the hyperparameters of the model, such as learning rate, batch size, and optimizer settings, to improve the training process and enhance the model’s performance. Moreover, a repeat cycle approach has been followed by iteration through the training, evaluation, and refinement steps multiple times until the model’s performance reaches an acceptable level. Finally, MAEs were computed for land cover components, road length and real estate units.

Since urban areas play a major role in the present regional regulation in terms of township incomes, MAEs involving each municipality were computed also considering their road length and real estate units. The results obtained are reported in

Table 3. Overall MAEs and areas computed with both OW and Sen4MUN.

Urban & Anthropic Areas (km2)			Road Length (km)			Real Estate Units
Sen4MUN	OW	MAE	Sen4MUN	OW	MAE	Sen4MUN	OW	MAE
103.9	92.1	0.16	1686.7	1626.4	0.81	303,049	293,214	133

and a general

Table A1. MAEs and areas computed with both OW and Sen4MUN (S4M).

Aosta Valley Municipalities	Urban Areas (km2)			Roads Length (km)			Real Estate Units
	S4M	OW	MAE	S4M	OW	MAE	S4M	OW	MAE
Allein	0.28	0.33	0.05	16.37	15.32	1.05	643	638	5
Antey-Saint-André	1.04	1.00	0.04	17.98	15.67	2.31	3250	3208	42
Aosta	9.58	9.57	0.01	106.26	104.55	1.71	65,073	65,050	23
Arnad	1.68	1.41	0.27	29.34	26.73	2.61	2614	2601	13
Arvier	0.87	0.66	0.21	22.73	22.02	0.71	1970	1971	1
Avise	0.45	0.50	0.05	12.59	11.58	1.00	899	890	9
Ayas	2.84	2.85	0.01	21.28	19.04	2.24	10,022	10,021	1
Aymavilles	1.37	1.39	0.01	31.05	30.03	1.02	3349	3346	3
Bard	0.26	0.20	0.06	3.04	2.56	0.49	320	315	5
Bionaz	0.55	0.58	0.03	15.22	13.95	1.26	626	622	4
Brissogne	2.11	0.79	1.32	19.78	18.46	1.32	1510	1499	11
Brusson	1.55	1.86	0.31	28.91	28.75	0.16	5229	5222	7
Challand-Saint-Anselme	0.85	0.93	0.08	19.30	15.49	3.81	3231	3230	1
Challand-Saint-Victor	0.63	0.70	0.07	16.93	15.66	1.27	1776	1770	6
Chambave	1.39	0.88	0.52	24.36	25.13	0.77	1804	1799	5
Chamois	0.17	0.33	0.16	3.15	3.15	0.00	568	562	6
Champdepraz	0.94	0.65	0.29	17.60	15.78	1.81	1411	1412	1
Champorcher	0.69	1.00	0.31	22.97	21.72	1.25	2355	2351	4
Charvensod	1.55	1.42	0.13	9.95	9.90	0.05	3897	3890	7
Châtillon	2.97	2.28	0.69	47.63	43.32	4.31	7515	7510	5
Cogne	2.10	1.72	0.38	25.21	22.44	2.78	5528	5527	1
Courmayeur	3.37	2.90	0.47	53.52	52.90	0.62	15,334	15,301	33
Donnas	1.90	1.72	0.18	26.89	28.21	1.32	4069	4048	21
Doues	0.60	0.70	0.10	24.41	27.03	2.62	1329	1317	12
Emarèse	0.29	0.34	0.06	13.57	12.85	0.72	984	977	7
Etroubles	0.70	0.57	0.13	18.90	14.89	4.01	1314	1302	12
Fénis	1.47	1.28	0.18	27.13	26.30	0.83	3409	3405	4
Fontainemore	0.66	1.47	0.81	26.91	26.74	0.16	1560	1535	25
Gaby	0.61	0.80	0.19	9.51	9.25	0.26	1394	1337	57
Gignod	1.37	1.23	0.15	30.44	30.14	0.30	2692	2673	19
Gressan	2.84	2.52	0.32	27.43	26.26	1.17	8330	8313	17
Gressoney-La-Trinité	0.82	0.88	0.06	4.18	3.80	0.38	1944	1912	32
Gressoney-Saint-Jean	1.86	1.97	0.10	18.56	16.32	2.25	5350	5347	3
Hône	1.16	0.89	0.27	13.00	14.51	1.52	2226	2226	0
Introd	0.56	0.53	0.03	13.87	13.33	0.54	1444	1435	9
Issime	0.77	1.02	0.25	10.47	9.56	0.91	1263	1260	3
Issogne	1.50	1.08	0.42	26.35	30.83	4.48	2305	2291	14
Jovençan	0.72	0.43	0.28	10.45	13.32	2.87	974	959	15
La Magdeleine	0.28	0.36	0.08	5.46	5.62	0.16	1157	1101	56
La Salle	2.25	1.95	0.29	42.81	36.21	6.59	7445	7419	26
La Thuile	1.77	1.44	0.33	27.23	24.94	2.28	7223	7205	18
Lillianes	0.52	0.98	0.46	19.48	22.28	2.80	1160	1147	13
Montjovet	1.62	1.42	0.21	39.41	38.81	0.60	3271	3262	9
Morgex	2.33	1.66	0.67	22.97	19.41	3.57	6982	6973	9
Nus	2.44	1.88	0.56	54.40	51.67	2.73	5125	5125	0
Ollomont	0.42	0.55	0.13	8.80	7.68	1.12	965	960	5
Oyace	0.27	0.26	0.01	3.50	1.93	1.57	475	473	2
Perloz	0.46	0.91	0.45	19.45	21.95	2.51	1248	1237	11
Pollein	2.11	1.16	0.95	10.27	14.13	3.86	2217	2206	11
Pontboset	0.33	0.47	0.14	10.31	11.32	1.01	809	791	18
Pontey	1.17	0.50	0.67	6.80	4.72	2.07	1296	1294	2
Pont-Saint-Martin	2.00	1.61	0.39	21.39	21.49	0.10	5165	5160	5
Pré-Saint-Didier	1.10	0.91	0.19	20.58	19.46	1.13	5925	5917	8
Quart	3.90	2.56	1.34	59.69	58.83	0.86	6495	6489	6
Rhêmes-Notre-Dame	0.31	0.39	0.08	6.28	5.58	0.70	719	707	12
Rhêmes-Saint-Georges	0.31	0.39	0.08	6.19	5.57	0.62	820	808	12
Roisan	0.66	0.66	0.01	13.44	12.45	0.99	1455	1447	8
Saint-Christophe	2.98	1.93	1.04	44.18	44.32	0.15	5519	5511	8
Saint-Denis	0.49	0.53	0.04	9.98	6.92	3.06	1163	1161	2
Saint-Marcel	1.46	1.19	0.27	34.99	34.13	0.86	2308	2299	9
Saint-Nicolas	0.46	0.48	0.02	19.19	17.17	2.02	1184	1175	9
Saint-Oyen	0.35	0.22	0.14	7.68	5.72	1.96	556	555	1
Saint-Pierre	2.22	1.82	0.41	40.67	37.22	3.45	5225	5222	3
Saint-Rhémy-en-Bosses	0.88	0.73	0.15	19.48	18.51	0.98	1274	1266	8
Saint-Vincent	2.32	2.13	0.20	40.48	39.29	1.19	8956	8943	13
Sarre	2.74	2.18	0.56	41.53	40.33	1.21	6616	6591	25
Torgnon	1.10	1.10	0.00	21.17	19.56	1.61	4332	4313	19
Valgrisenche	0.36	0.54	0.17	14.00	13.78	0.22	859	851	8
Valpelline	0.60	0.62	0.02	13.58	14.25	0.67	1322	1300	22
Valsavarenche	0.48	0.66	0.18	8.46	9.29	0.83	1036	1023	13
Valtournenche	2.76	2.66	0.10	37.56	37.51	0.05	13,870	13,861	9
Verrayes	1.76	1.59	0.17	26.29	26.12	0.17	3218	3210	8
Verrès	2.14	1.27	0.87	13.50	10.74	2.77	3956	3953	3
Villeneuve	1.43	1.01	0.42	28.31	30.04	1.73	2192	2187	5
TOTAL	103.9	92.1	0.16	1686.7	1626.4	0.81	303,049	293,214	11

in Appendix A and in the graphs concerning on the MAEs reported in Figure 3.

Table 3 reports the overall MAE computed considering the same component retrieved with OW and Sen4MUN.

Figure 3 shows the MAEs for the three territorial components used for the calculation of municipal contributions by the Valle d’Aosta Autonomous Region. In particular, the following are reported in the section: (A) the MAEs for urban and anthropic areas expressed in square km; (B) the MAEs for the length of municipal roads expressed in km; (C) the MAEs for real estate units expressed in units. In all three sections of Figure 3, it is interesting to note that the largest MAEs generally occur in municipalities with fragmented urban areas with scattered villages and houses and roads in steep areas due to the limitations of the sensor and processing techniques or errors inherent in the traditional method, such as to cause the imbalances as illustrated below.

As reported in Table 3, an overall MAE of 0.16 km² was obtained involving urban and anthropic areas, while 0.81 km was obtained for road length and 11 units in the case of real estate units. An overall MAE of 0.82 km² was computed involving all EAGLE Land cover classes, respecting those computed with Cerutti’s approach. All the values obtained are significant and seem to suggest a validity of Sen4MUN. The values obtained in Table 3 for the two approaches are not very distant as indicated by the MAEs. It should be highlighted that the standard for assigning contributions is currently OW because it is regulated at a legislative level. It is worth noting that the errors came from Cerutti’s approach which overestimate some areas due to limits in this approach itself. Furthermore, it is interesting to note that in all the components, for example urbanized areas, road lengths and real estate units, MAEs are associated with municipalities that underwent more changes in at least one of the components in the reference year and with sparse villages. Unlike what can be derived from the MAEs here computed, the Sen4MUN system is more effective in monitoring territorial changes because it is based on satellite and GIS data updated at a high temporal frequency than the ordinary system. The MAEs obtained allow the transferability of the new approach and possible replacement of the ordinary one even in an Alpine reality such as the one investigated. In fact, MAEs are generally higher in mountainous areas than those obtainable in lowland areas due to the limiting factors affecting some remote sensing applications (land cover mapping in particular).

4. Discussions

The results show the consistency of the methods. Furthermore, the procedure adopts free worldwide coverage data. The creation of land covers on an annual, biennial, or monthly basis or even on a time scale of Sentinels (potentially every 5 days) allows the continuous monitoring of territorial dynamics. Moreover, this activity is part of the actions that each European country carries out for the monitoring of soil consumption and that the regions or provinces carry out to offer detailed products according to the guidelines of the European Environmental Agency. Furthermore, the availability for the administration of a regional air flight with very high resolution imagery (like an AGEA flight), albeit for agricultural purposes, allows this type of data to be integrated into other service chains. Such as in the case of Sen4MUN, to extract the footprint of buildings and roads thus makes public spending more efficient through the expansion of derivable products and services. Certainly, it is complex to create universally valid models, and locally suitable models are always preferable. But it must also be said that to create standards such as the Sen4CAP adopted by the European Union, compromises must be reached. Sen4MUN was specifically developed in a complex area such as to make it scalable in a much simpler way to plain and hilly areas which are the majority at a European level. The fact that the approach returns interesting results in an alpine and geomorphologically complex context is certainly encouraged from a scalability perspective. It should be remembered that the largest MAEs occur in municipalities that have more scattered settlements with isolated houses mixed with woods. Areas by which their nature are complex to map at Sentinel scale are not so numerous. Finally, it should also be remembered that many of the differences are linked not to a lack of accuracy and precision of Sen4MUN, which from a purely photo-interpretative analysis proved to be more performing than the traditional OW approach, but precisely as a result of errors in the dataset from statistical investigations of OW. This dataset, despite the errors it suffers from, was taken as a reference base as current regulations require this to be the data collection tool for the purposes of assigning contributions. During the analyses in some municipalities with high MAEs, it was found that Sen4MUN, on the contrary, mapped components that had escaped OW.

Naturally, the critical points in the Sen4MUN approach result in a correct mapping and knowledge of geomatics and remote sensing and their limits, such as the most suitable approach (hierarchical, single-directed, data fusion, etc.), the classification algorithms, as well as the input data that best respond to the components to be mapped, etc. Furthermore, the definition of the optimal number of training areas validation sets according to the area to be mapped is crucial [65,66,67,68,69,70,71,72,73,74,75,76] in obtaining high accuracies and minimization of errors [77,78,79,80,81,82,83,84,85,86,87]. Despite this work, a ready land cover has been adopted with an overall accuracy upper than 0.94, the whole procedure considering all these issues has been tackled in previous scientific literature involving the Aosta Valley territory [88]. In fact, in the case of a stand-alone and not prototype of this service at an application level not only on a regional or national scale, but on a European scale given the characteristics of the continent, the optimal solution would be to implement a regulated procedure, but which gives a certain degree of freedom in the algorithms and training sets (providing only thresholds) so that chains capable of responding best according to the area to be mapped are developed. Sen4CAP has been excessively standardized also in the algorithms making it underperform in the mountain areas [89]. The need to standardize as much as possible a procedure that moves economic contributions is crucial for large-scale regulation [47], but this is performed with the support of researchers, technicians, and academic experts and after several experiments in UE, to better define the operational leans and freedoms useful for developing consistent products and services for each European reality without creating disparities and differences. As far as the authors are aware, there are no applications in scientific literature for the monetarization of land cover and usage or AI geospatial deep learning at a national, regional, or local contribution scale for services provided by the public administration, therefore a comparison of the results obtained to date is extremely difficult. At the same time, fewer are scientific works on monetary attribution to the components of the coverage for the estimation of ecosystem services and nature based solutions [90,91], but with the intent of estimates of mere research and not of actual transfer of technological application to the public sector. Furthermore, the works themselves are on a large and non-municipal scale and involve geographical areas different from those under study. Therefore, we hope that this work will stimulate the Italian, European and international scientific and technical community to deepen and explore the application potential of geomatics in the operational workflows of public administration by offering new services and comparing the results and approaches obtained for the purpose of continuous improvement.

The use of the prototypal Sen4MUN Aosta Valley has made it possible to make the procedure for assigning contributions to municipalities more objective through the use of Sentinel data and updated GIS geodatabases, favoring technology transfer and the implementation of a possible new service within the Copernicus program and other future programs, such as an IRIDE capable of offering increasingly high spatial and temporal resolution data [37] useful for direct applications to the public sector [92,93,94,95,96].

5. Conclusions

Sen4MUN can be used as a standard procedure for assigning contributions to municipalities. The procedure is consistent and in line with the ordinary territorial workflow based on statistical surveys without the use of earth observation data, deep learning, and geodatabase. Although the system has been tested and used on a prototype level in the Aosta Valley, it can be scaled up to other Italian and European regions. The hope is that this service will become operational and spread to all member countries of the European Union as well as other national realities. Sen4MUN could join other services; coming from various Earth observation programs such as Copernicus, favoring an ever more massive technology transfer to the public sector by rationalizing activities and processes with plural activities with a view to the ever-increasing importance of geomatics in management flows and implementation planners of local, national and European policies. Finally, Sen4MUN seems to be capable of providing useful data for contribution purposes to public administration.