Meteonetwork: An Open Crowdsourced Weather Data System

Abstract

Citizen science has shown great potential for bringing large groups of people closer to science, thanks in part to cooperation with universities and research centers. In this context, amateur weather networks played a major role in the last few decades thanks to a constant growth in technology. An example is given by the Meteonetwork association, born in 2002, and mainly composed by atmospheric science enthusiasts, who built up in time a huge weather station network in Italy. In recent years, they have enlarged their horizons over Europe, displaying real time observations and daily maps coming from both personal weather stations and official networks. This study described how Meteonetwork has set up an open crowdsourced weather data system, how data are validated, and which products are generated and freely accessible for scientists and stakeholders for their own purposes. Two concrete use cases were described as examples: the Weatherness Project, which selects a subnet of Meteonetwork data for biometeorological and health purposes and the data assimilation process implemented to improve the initial conditions into the WRF meteorological model for daily weather forecasts.

1. Introduction

In recent decades, thanks to the constant development of information and communication technology (ICT) and to cost devices reduction, common people attracted by meteorology have constantly grown, leading to their role as citizen scientists capable of creating large amateur networks with increasingly reliable automatic measuring stations.

“Citizen science” (CS) is recognized as a self-developed significant activity in order to support, and often complement research and institutional monitoring conducted by universities, research centers, environmental control agencies, etc. [1,2,3]. It is a form of collaborative research involving volunteers, amateurs, and enthusiasts [4,5].

CS still presents a wide variety of definitions. Many attempts have recently been made to rationalize it [6], focusing on the work undertaken by communities of citizens to advance science, promote a broad scientific mentality, and/or encourage democracy in data dissemination in a commitment that allows society to tackle complex problems. For instance, the CS can be considered a technique of crowdsourcing, as was first termed by Howe [7] referring to the idea of outsourcing to the crowd, while Dickinson et al. [8] defined it as “getting an undefined public to do work, usually directed by designated individuals or professionals”. Traditionally, it was described as “obtaining data or information by enlisting services of a (potentially large) number of people”. However, due to recent innovations, this definition can now be expanded to include the use of a range of sensors, typically connected via Internet [9].

The great potential of crowdsourcing, given the widespread availability of observations, is limited in terms of data reliability, which is a major concern. Data usually come from a large number of operators who collect their measurements under many different conditions, which may not be standardized. In this context, the assessment of data quality [10] plays a major role. Evaluation is an essential methodological issue, requiring transparent and reliable procedures to ensure credibility and sustainability to citizen science projects. For this reason, the reputation of citizen science will grow only paying more and more attention to procedures for quality control of measurements (sampling plans, methods of sample collection and measurement in the field, calibration of instruments, etc.), as well as in transparent data quality and assurance criteria. In this regard, a review about the wealth of quality norms and methods was well described in Fiebrich et al. [11]. Therefore, to validate citizen science results, the same metrics used by professional researchers must be applied. Thereby, the policy makers can use citizen efforts to expand the knowledge provided by traditional monitoring [12].

1.1. Crowdsourcing and Amateur Weather Networks

The rapid spread of ICT over the past two decades facilitated the process of collecting and sharing data, and led to the growth of several online networks. Considering that traditional meteorological networks are in decline [13] and that, on the contrary, the demand for real-time, high space-time resolution data is increasing, the need for crowdsourcing weather data is clear. As computing power increases, the ability to process and use these kinds of data will also rise; therefore, it is necessary to explore their potential. Since the uncertainty associated with individual citizen observations are higher than for professional stations, the massive redundancy of the first ones helps to detect and improve anomalies in measurements.

The Meteonetwork (MNW) system is a typical example of citizen weather stations (CWSs), covering a wide territory with a high spatial density which allows a high redundancy of measures.

1.2. The Origin and Mission of the Metonetwork Association

The Meteonetwork (MNW) association was founded on 6 April 2002 in Seregno (Italy) by a group of friends and weather enthusiasts; a year later, it was officially registered as association in the city of Mantova (Italy).

MNW is a non-profit organization with the task to “promote and disseminate for the benefit of community the knowledge of meteorological, climatological, environmental, hydrological and glaciological sciences, and their multiple expressions on the territory”. Nowadays, it is comprised of about 200 associated members.

One of its most important goals has always been to develop an integrated and coordinated Automatic Weather Station (AWS) network made of various amateur meteorological sites. This is the main achieved result here presented: the creation of a CWS network, gathering a large number of citizen scientist observations, a goal made possible thanks to a well-established market of low-cost, easy-to-use devices, owned and maintained by private individuals who love to share their data on the Internet for the meteorological and climatological community; a way where people are, in fact, no longer simply consumers of data, but also producers [14].

The aims of this work were to describe the MNW network, and the procedures of data collection and processing with particular attention paid to the activated quality controls; in this way, data, coming from citizen scientists, through a rigorous methodological path, can represent a key tool in the knowledge of local meteorology with benefits in a wide variety of weather applications, including operational weather forecasting, health, energy, agriculture, and water management.

The paper is divided into two sections. The first one describes the open crowdsource weather data system implemented by MNW: the geographical area, the development, and actual consistency of the network, the information technology infrastructure, the main guidelines and ancillary data (the ‘metadata’), and the strict quality control procedures. The second section is focused on services available to all users, such as maps of real time weather data, and on two applied case studies: (a) the data assimilation process using the MNW values into the WRF (Weather Research Forecasting) forecasting model, and (b) the Weatherness project, aimed to set up biometeoclimatic indices for health and medical matters.

2. Materials and Methods

This chapter illustrates the evolution of MNW through years, the methodology, guidelines, and implemented quality control procedures.

2.1. The Network Geographical Area

The MNW operational area includes the whole European region (Figure 1a), with a particular focus over Italy (Figure 1b). At the end of January 2022, the network consisted of 6506 weather stations from 42 countries around the world (

Table 1. Number of operative weather stations in the MWN database.

Country	Stations (N)	Owner
Italy	4411	MNW (2128)/other national bodies (2283) *
Norway	866	Frost
Greece	444	MeteoGR (336)/Frost (8)/MNW (2)
France	367	Infoclimat (321)/Frost (29)/MNW (17)
Sweden	125	Frost
Finland	50	Frost
Spain	31	Frost (18)/MNW (11)/Infoclimat (2)
Poland	23	Frost
Germany	22	Frost
Svalbard Islands (Norway)	20	Frost
Romania	14	Frost
Russia	14	Frost
Swiss	14	Infoclimat (5)/MNW (5)/Frost (4)
Denmark	11	Frost
Ukraine	10	Frost
Belgium	9	Frost (5)/Infoclimat (4)
Netherlands	9	Frost (6)/Infoclimat (3)
Turkey	9	Frost
Portugal	8	Frost (5)/Infoclimat (3)
Ireland	6	Frost
Republic of San Marino	5	MNW
Belarus	4	Frost
Czech Republic	4	Frost
Slovakia	4	Frost
Bulgaria	3	Frost
Brazil	2	MNW
Canada	2	Infoclimat
Latvia	2	Frost
Slovenia	2	Frost (1)/MNW (1)
Albania	1	MNW
Antarctica	1	Frost
Austria	1	MNW
Croatia	1	Frost
Cyprus	1	MeteoGR
Estonia	1	Frost
Greenland (Denmark)	1	Frost
Iceland	1	Frost
Kosovo	1	MNW
Lithuania	1	Frost
Luxembourg	1	Frost
Malta	1	Frost
Moldova	1	MNW
Serbia	1	Frost
United Kingdom	1	MNW
Total number of stations	6506

* For a detailed classification about Italian weather stations, the reader is referred to Table 2.

). A detailed overview of the MNW network can be seen at this web site https://meteonetwork.eu/en (accessed on 15 May 2022).

Besides the acquisition of weather data coming from Italian citizen scientists, during the last decade, two main mutual data exchange agreements with European amateur associations have been signed: Infoclimat in France and MeteoGR in Greece. Furthermore, since 2020 additional data have been imported from the Frost network (https://frost.met.no/index.html (accessed on 15 May 2022)), managed by the Norwegian Meteorological Institute, which include quality-controlled daily, monthly, and yearly measurements of temperature, precipitation and wind data. Locally, official mutual data exchange covenants to automatically integrate data from the Regional Protection Environmental Agency (ARPA) of some Italian regions, like Emilia-Romagna, Veneto and Calabria, and from the meteorological agencies of the autonomous provinces of Trento and Bolzano have been signed.

In 2021, the MNW stations were included in the MISTRAL portal (Meteo Italian SupercompuTing poRtAL, https://www.mistralportal.it/ (accessed on 15 May 2022)); this led to a two-way relationship and to a mutual exchange of weather stations data inside both networks.

Table 2. Number of weather stations included in the MNW database and their relative percentage out of the total. The country column refers to the origin of the network owner; however, some weather stations are located outside the origin nation, hence, the term “et al.” can be here interpreted as “and other countries”.

Network	Country	Stations (N)	%
MNW (citizen scientists)	Italy et al.	2163	33
MISTRAL	Italy	1789	27
Frost	Norway et al.	1287	20
MeteoGR	Greece et al.	435	7
Infoclimat	France et al.	338	5
ARPA Calabria	Italy	163	3
Meteo Trentino	Italy	116	2
Civil Protection of Bolzano Province	Italy	84	1
ARPA Veneto	Italy	69	1
ARPA Emilia-Romagna	Italy	62	1

summarizes the number of weather stations and the network provenience in the MNW database.

2.2. The Grow and the Actual Consistency of the Network

The MNW network started in 2002 with 35 weather stations. At the beginning, weather instruments were only owned by a few numbers of participants, whose data usually remained in their computer or published on personal websites. At that time, real-time data of weather stations located in Italian territory were not available or easily obtainable on Internet. These were at least two main reasons that prompted the MNW association to decide the set up a meteorological network for “entry-level” stations.

Traditionally, weather stations with a good quality:price ratio are favored, such as Davis (Vantage Pro 2 and Vue versions), Ecowitt, Froggit, Sainlogic, Oregon Scientific, Bresser, PCE, Irox, and Lacrosse. Basically, they all measure air temperature, relative humidity, precipitation, wind speed and direction, and atmospheric pressure, but not all of them are equipped with instruments to monitor atmospheric variables, such as the incoming solar radiation or UV radiation.

Table 3. Number of sensors installed inside the MNW network. The percentage column is calculated in regard to the total number of operative weather stations (6506).

Meteorological Variable	Sensors (N)	%
Precipitation	6312	97
Air temperature	5926	91
Air relative humidity	4906	75
Wind speed and direction	4751	73
Atmospheric pressure	4410	68
Solar radiation (global)	1471	23
UV radiation	1132	17

summarizes the number of instruments for each meteorological variable available for the 6506 weather stations regularly registered into the MNW database, where, to be included, each sensor of the CWS has to go through the quality control process.

Among the 6506 stations, 4780 upload their data on the MNW database at least once every 24 h, and around 3400 are constantly on-line during the day.

Figure 2 depicts the development of the MNW network from the origin with an average growth rate of about 150 stations per year. The last decade saw a rapidly increasing growth, when other (national and foreign) weather networks were imported in the MNW database: Infoclimat (France, 2014), MeteoGR (Greece, 2018), Frost network (Norway, 2020), and MISTRAL (Italian official bodies, 2021). At the beginning of the current year, a quality control (QC) upgrade was implemented, and some weather sites were excluded from the system; this explains the drop in the number of stations in 2022.

Currently, the data stream generated by the Italian stations on MNW averages almost 6 records per second (518,000 data records per day, 21,583 records per hour). Figure 3 shows the sampling frequency of the operative MNW stations in Italy, on average, during the month of January 2022 (1134 stations). The range of the sampling frequency for each instantaneous datum spans between a minimum of 1 min up to 60 min, and most of the stations have a sampling frequency in the range from 3 to 5 min.

2.3. Guidelines and Metadata

The rules established by the World Meteorological Organization (WMO) in the “Guide to Meteorological Instruments and Methods of Observation WMO-No. 8” [15] for correct station siting are difficult to fully comply with, especially with stations located in highly urbanized areas.

Moreover, citizen weather stations typically have missing metadata about their siting and exposure. The lack of this information means the impossibility of an a priori knowledge of the typical data errors for each station. For this reason, many associations and research organizations are adopting installation rules for their own needs in order to guarantee a standard methodology in data collecting. This is the case of the MNW association, which has established its own guidelines, based on WMO rules, but with necessary adaptations to the context of CWS, often located in various types of sites, which allow for a categorization and regularization of sensors installed on roofs, gardens, or other surfaces.

2.3.1. Type of Siting

Proper siting of a weather station can be very challenging, especially in urban or suburban areas. Three types of positioning have been defined by MNW, approximately following the environment characteristics based on the Local Climate Zones (LCZ) classification system by [16].

-

Rural: an extra-urban placement in areas where building density is very low or even absent, equivalent to LCZ A, B, C, and D. The location is not affected by the urban heat island effect and it includes, for example, countryside far from town, mountain installations, etc.

-

Suburban: a semi-urban placement in areas of low building density and abundance of pervious land covers and scattered trees, comparable to LCZ 5, 6, and 7. The urban heat island effect is low, e.g., residential areas far from cities.

-

Urban: an urban placement in areas of high building density, with land cover mostly paved and few or no trees, similar to LCZ 1, 2, 3, 4, and 9. The urban heat island effect is present, e.g., downtown, industrial areas, etc.

For instance, considering the Italian weather stations only, which number 4411 as reported in Table 1, the number of different monitoring sites counts 651 stations in rural areas, 425 in suburban, and most of them (3335) in urban contexts.

2.3.2. Sensors Layout

Weather station sensors have different setting requirements according to the meteorological variable to be measured. For instance, temperature and humidity instruments are the most important group of devices in a CWS, but they are also the most affected by incorrect positioning. In order to be accepted in the MNW database, thermo-hygrometer sensors must be housed in a certified passive or fan aspirated radiation shield (such as Davis, Barani Design, MetSpec, Comet System, Vaisala, Campbell), and properly located, generally between 1.70–2.0 m above ground and at an adequate distance from the surrounding obstacles to not alter the measurement. However, distances and heights, respectively, from obstructions and above ground, are variable according to the positioning of each sensor: for a CS network, some compromises must be made, but, basically, the closer you are to the standards, the better. Generally, the presence of obstacles should not prevent a representative measurement of the surrounding area to be monitored.

MNW has, in fact, defined five different configurations for the installation place, which are listed below:

(i)

Open field: the weather station can be installed in an open field (e.g., countryside) with obstacles at least 10 m away from the station itself.

(ii)

Garden: weather sensors in gardens do not comply with all the requirements of the previous item; hence, they are accepted if obstacles are at least 4 m apart and their heights do not prevent an acceptable measurement.

(iii)

Courtyard: another option is to install the weather station in an area completely or partially enclosed by walls or buildings with further exceptions regarding the type of ground than the previous item (garden).

(iv)

Roof: this is one of the most critical installation places. In this case, the weather station must be installed at least 2 m above the roof surface and at a distance such that chimneys, air conditioning units, or other obstacles cannot influence the right measurement. This type of installation is similar to the historical meteorological observatories.

(v)

Balcony: usually, temperature and humidity sensors are installed separately from the rest of the weather station on a balcony or a terrace. In this case, there are additional limitations: sensors must be installed at least 1.50 m from the walls and at a distance such any obstacles cannot prevent a proper monitoring of the site.

For a complete review on how to place a weather station in order to be accepted in the MNW network, the reader can refer to these guidelines (in Italian): http://wiki.meteonetwork.it/index.php/Norme_OMM_e_Norme_MeteoNetwork (accessed on 15 May 2022).

The open field configuration is closer to the WMO standards (class 3) and, hence, if possible, it should be preferred. In

Table 4. Example of requirements for the configuration of a garden.

Sensor	Height above Ground (m)	Notes
Thermo-hygrometer	1.7 ÷ 2.0	The thermo-hygrometer must be housed in a certified radiation shield at a height above ground (a short grass or grass with not much gravel or sand surface) between 1.70 m and 2.00 m, or higher if necessary, to overcome hedges or other obstacles, and at least 4 m from the nearest building or obstacle.
Rain gauge	At least >0.50	The rain gauge must be placed at least 4 m from the nearest obstacle, and at a distance such that any vertical barrier (tree, building) cannot prevent acceptable measurement in case of oblique precipitation. The rain gauge can also be separately placed on the roof.
Anemometer	Variable	The anemometer must be placed at a height such that any vertical obstacle cannot prevent acceptable measurement of gusts. The influence of turbulence should be considered as well. The anemometer can also be separately placed on the roof.
Solar and UV radiation	-	The sensors must be placed so that they are never in the shade throughout the day; even these sensors can be separately placed on the roof.

an example with the requirements for configuration ii (garden), which is the most common case between the weather stations registered in the MNW network in Italy, is shown. In fact, among the total number of Italian weather stations in the MNW database (4411), we count 594 placements in open field, 2638 in garden, 111 in courtyard, 972 over roofs, and only 96 mounted over balconies.

The procedure for submitting personal weather stations to the MNW association involves the following steps:

-

An owner assessment, specifying in which type of area the weather station will be located (rural, suburban, and urban);

-

The choice for the best sensor layout based on the installation site, and compliance with related requirements, giving priority to temperature and humidity sensors;

-

Sending pictures of the weather station to the MNW team to check the necessary requirements;

-

Approval by the MNW committee and admission of the weather station into the network.

Once the weather station is accepted into the MNW network, an identity card with its siting specifications and measurement configurations will be available on the MNW web pages for each weather station inside the network. The procedure described above has proven to be remarkably effective in ensuring excellent reliability and data consistency, and encouraging station owners to provide the required metadata about their own station. In this way, the citizen has become, over the years, the ‘gatekeeper’ of their own individual sensor, installing it and ensuring its regular operation [17].

2.4. Information Technology Infrastructures

The actual MNW information technology (IT) is based on an edge solution with its own physical and cloud data center, where MNW has always been the IT owner, since the association was founded. In 2016, essential services for MNW activities have been moved to cloud and managed solutions, and in 2021, 6 CentOS Linux servers were installed for a total number of 60 processors (Intel and AMD) and 70 GB of RAM able to run the following weather chains every day:

-

the WRF-ARW (Weather Research Forecasting–Advanced Research WRF) model processing and post-processing;

-

the post-processing of images for the WRF ensembles, GFS (Global Forecasting System), ECMWF (European for Medium range Weather Forecasts), and MOLOCH models.

Data Storage

Between 2002 and 2005, meteorological data were manually uploaded by each citizen scientist; unfortunately, it was an unsustainable solution. Hence, an automatic acquisition process was implemented in 2006, as result of continuous developments in acquiring weather data from stations.

In detail, each weather station is interfaced via computer with a proprietary software which locally stores weather data (e.g., WeatherLink, Virtual Weather Display, etc.), and uploads them via FTP or cloud (e.g., weatherlink.com). MNW, through its automatic scripts, constantly collects raw data minute-by-minute with the following work flow:

-

identification of meteorological station ID and data source (by HTTP protocol);

-

identification of the interface to be used (depending on the format of data source and the software used by each collaborator). Currently, there are drivers for the following software/cloud archives: WeatherLink, WeatherLink IP, Cumulus, Anemos, WS2, Sint Wind PI, and there are automatic picking solutions to interface with the APIs of our partners, including Infoclimat, ARPA Veneto, Meteo Trentino, MeteoGr, and Mistral.

-

data download (daily archive or latest available data);

-

preliminary correction of errors (meteorological variables in a given min/max range, see Section 2.5 for further details);

-

identification of the reliability of single data (depending on the positioning of the weather station, the technical equipment, etc.);

-

data upload on the PostgreSQL database which has now reached a size of 600 GB, 450 of them only to archive raw weather station data.

2.5. Quality Control

Since applying appropriate quality control methods is an essential component when using these observations, specific guidelines, standards and protocols have been set up to achieve high-level of trustworthiness, guarantee robustness, and quantify the reliability of crowdsourced data (e.g., metadata protocols [18]; QA/QC procedures [19]).

The admission procedure for the MNW stations ensures a certain level of data quality for each sensor, since erroneous data reading caused by sensor malfunctions, hardware, or power supply error or changed environmental conditions can still occur. The MNW data are subjected to automatic validation and quality control procedures in order to reduce this possibility of measurement errors entering in the MNW production chain such as the WRF modeling or map generation processes (see Section 3).

The high-quality process has also been highlighted in two recent studies [20,21] where the MNW data, in particular temperature and precipitation, have been compared with official networks of ARPA Emilia-Romagna and Veneto, respectively. Both the summarized results underlined the worth of MNW efficiency, and, above all, how the joint use of the two networks has given a greater or, at worse null, benefit with respect to the ARPA networks only.

The actual automatic control of the data collected is available for the stations located in Italy, but it has not yet implemented for the imported networks previously described.

The automatic control follows two steps:

• Range test: delete clear incorrect data (e.g., minimum temperatures below −40 °C, maximum temperatures above +50 °C, and rainfall below 0 mm);
• Cross-validation through percentiles: for each atmospheric variable and each station, the distribution of values measured by the 15 nearest stations in a maximum of 30 km range is analyzed in order to obtain the 10th and 90th percentiles. A tolerance is calculated using the standard deviation of the distribution, which is added to the 90th percentile and subtracted from the 10th percentile to obtain the cut-off values of the quality check. The use of standard deviation for the tolerance computation considers the natural variability of the variable in a specific weather situation: if this variability of the given variable is predominantly accentuated (high standard deviation), the boundaries are more relaxed and less stringent. In a situation where the natural variability of the variable field is very reduced, the tolerance is more rigorous, instead. A general check over the whole Italian domain with the same logic is performed only for the interpolated-map making process, in order to be aware to delete those outliers that might cause graphical issues. Data failing the QC are flagged, and do not enter in the maps elaboration process; however, they remain in the database.

Future Real-time Anomaly Detection System

Given the high frequency and the amount of ingested data, the ability to consistently detect and flag erroneous data readings as soon as possible is of paramount importance to ensure the quality of the downstream products (live maps, extremes, hourly/daily maps, models).

For this reason, MNW is currently testing the adoption of an upgrade version with an automated anomaly detection method to be fully operative by the end of 2022: the aim is to implement a completely automated procedure to highlight anomalous observed values for each individual sensor of the weather stations in real-time.

The current best candidate for the Real-time Anomaly Detection System (RADS) is the adoption of a workflow inspired by the work proposed in refs. [22,23] that features a combination of a rule-based quality control procedure and a machine learning estimate based on the last 24 h of data from the station.

Given that no labels are available in the dataset, the approach for the machine learning is based on unsupervised methods. The aim of machine learning model is to predict values for the next data point measured by the station, based on the last 24 h of data. When the value sensed by the station is between 3σ of the previous 24 h plus or minus the estimate predicted by the machine learning model, the data point is considered nominal (not an anomaly). Otherwise, the record is flagged as an anomaly, and thus excluded from being used in the generation of downstream products.

Careful considerations are required in the choice of the machine learning estimation model, especially given the different sampling frequency of the MNW stations: the machine learning method must be able to model the wide range of operating frequencies of stations when estimating values for the next data point. The current best candidate method is based on the recent deep learning work by ref. [24]. Preliminary testing on a limited dataset shows a promising ability of this method to generate estimates at different sampling frequencies, even when encountering missing and irregular time steps between data points.

3. Products and Discussion

The following section describes main technical and scientific achievements reached using MNW data. In detail: (a) generated services with live data and interpolated maps, (b) the Data Assimilation (DA) using MNW values into the Weather Research Forecasting (WRF) meteorological model, and (c) the Weatherness project, coordinated with the university of Milan (Università degli Studi di Milano).

3.1. Services

Once a weather station proceeds through the validation phase and finally joins the database, data are freely published in real-time on the MNW web sites and app. Data are used to generate various services, such as real-time charts or daily extremes maps. Moreover, through single call or even bulk APIs, MNW data are accessible for users’ researches and studies. The plotted variables are the following: 2 m air temperature (T_2m) and relative humidity (RH_2m), dew point (Td_2m), precipitation (TOT_PREC), wind speed (WSPD) and direction (WDIR), sea level pressure (MSLP), and UV and global solar radiations (UV_RAD and GRAD).

3.1.1. Live Data

CWS data are available in real time from live maps (https://www.meteonetwork.it/rete/livemap/ (accessed on 15 May 2022)) and updated every 5 min for stations registered on the MNW database, while it is 5 to 45 min for the data imported from other networks. On live charts, it is further possible to view the daily extremes for temperature, precipitation, humidity, and wind as well. Additionally, the TOT_PREC and T_2m values greater than 5 °C in the preceding 30 min are shown together with T_2m values of the previous 24 h, in order to be compared with the actual ones. Figure 4 shows a temperature map from the MNW webpage over Europe (a) and a zoom over Italy (b).

3.1.2. Interpolated Maps

Available MNW data are used to produce maps which show weather fields at a regional, national level and on some European states both in real time (every 20 min) and with a daily update. Animations are available from midnight of the current day to the last data record. These plots are available for Italy with a possible zoom over each regional district, France, Greece, and Norway. Data passing the described quality check enter a processing chain involving geostatistical spatialization methods for representing different variables fields. Spatial data are on grids with different horizontal resolutions, depending on the geographic domain (from 1 km for regional domains to 5.1 km for national domains). The spatialization of data is calculated with an average of the two following methods: the natural neighbor interpolation and the inverse weighted distance (IWD). Generally, the choice of using a mean value of different interpolation methods generally performs better than taking each of them alone [25].

The atmospheric variables directly pass into the spatial analysis process except for temperature. For this field, the spatialization is carried out using the potential temperature data at 2 m, calculated starting from T_2m and the sensor elevation; once the potential temperature grid has been created, the temperature field is obtained by an inverse process, using the altitude by the DEM (Digital Elevation Model) of a specific domain, in order to take into account the influence of orography in generating the field of temperature. The TD_2m is derived from T_2m and RH_2m, using the formula proposed in ref. [26].

For each variable, extreme maps are derived daily, plotting the highest and lowest values (Figure 5) to include daily maximum (Hi_T_2m) and minimum (Low_T_2m) temperature, TOT_PREC, maximum (Hi_RH_2m) and minimum (Low_RH_2m) humidity, and maximum wind gust (Hi_WSPD).

The choice to use the same interpolation method for all the meteorological variables, including precipitation, which has a non-normal distribution of values, has been made to simplify the number of generated maps which, in any case, have the purpose of representing the physical field on a regional/national scale. As a future goal, the feasibility of using more suited interpolation methods for precipitation will be taken into consideration.

3.1.3. The Open-Source Policy

The open-source policy adopted by MNW has allowed to sign scientific agreements with different bodies, such as universities and Regional Environmental Protection Agencies. In fact, the network data are licensed under Creative Commons BY (CCBY) 4.0 and we have active Application Programming Interface (API) services available to users. During the last decade, MNW stipulated official agreements to supply long data series for universities, private companies, institutions, students, and researchers. The university of Politecnico di Milano, for instance, uses MNW data for real-time hydro-meteorological simulations for droughts [27] and floods forecasts [28]. Weather data, provided by MNW association via FTP transfer, help to update the meteorological input into the FEST-WB hydrological model, and give a better overview of local ground measurements in the precipitation field [29] and surface soil moisture [30].

Another example comes from the FOMD (Fondazione Osservatorio Milano Duomo) where MNW data are used to integrate the urban meteorological network to analyze climate conditions in the Milan metropolitan area [31,32].

3.2. MNW Data Assimilation into the WRF Model

Integrating citizen observations into operational systems brings a lot of challenges, but it can potentially benefit a wide variety of applications, including NWP (Numerical Weather Prediction) models [33]. Measurements from a dense network of citizen weather stations, assimilated into a NWP model operational pipeline, improved NWP skills [34,35].

The WRF (https://www.mmm.ucar.edu/weather-research-and-forecasting-model (accessed on 15 May 2022)) is a mesoscale weather forecast model, developed by the National Center for Atmospheric Research (NCAR). It has different versions and different optional packages allowing customization of the modeling chain in a very thorough way; in this case study, the version used is the WRF-ARW (Advanced Research WRF) by ref. [36].

To assimilate the MNW network values into the WRF forecasting model, additional WRF-DA packages (https://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/v4.3/users_guide_chap6.html (accessed on 15 May 2022)) were used. Compared to the standard version, the WRF-DA can assimilate the observations at t = 0 as the data measured by meteorological stations on the ground (BUOY, METAR, SHIP, and SYNOP), upper sounding, radar scans, and satellite data.

Usually, the data assimilation is carried out using the 3D-Var or 4D-Var techniques. In this case, the 3D-Var was implemented for the coupled WRF-MNW version model computations. This enormous variety of observations, whether they are made at surface or at upper levels, can be prone to manual or instrumental errors. Therefore, the WRF-DA code has an essential component of forecasting correction based on the calculation of coarse errors, correction of bias, and objective fine-tuning of observation errors, which is essential to produce input data that correspond as much as possible to the real state of the atmosphere at time zero.

The 3D-Var implanted in the WRF-MNW version model runs on a domain between 2° E and 22° E longitude and between 35° and 48° N latitude with a horizontal resolution of 5.1 km and on 38 vertical levels. It uses the GFS initialization and integrated contour data with surface observations at time zero, deriving from the MNW meteorological stations network as well as with data collected from the international synoptic network assimilated in the domain.

The model was set up to use the following physics packages in simulations:

-

microphysics: Ferrier [37]

-

long wave radiation: RRTMG [38]

-

short wave radiation RRTM [39]

-

Planetary Boundary Layer: Yonsei University Scheme [40]

-

surface layer options: MM5 Similarity Scheme [41]

-

land surface options: Unified Noah Land Surface Model [42]

The entire flow scheme of the WRF-MNW system operates on two daily runs: 00 UTC and 12 UTC for a time horizon of 48 h.

As integrating citizen observations has been seen a fundamental source of information that could increase the performance of weather forecasts [43], a future investigation will be to check the benefit given by the introduction of assimilated MNW stations into the WRF model, thereby ensuring that the forecast matches better locally observed weather.

3.3. The Weatherness Project

The Weatherness Project aims to set up some MNW stations among the entire dataset to calculate biometeorological indices of particular interest for health and medical purposes. Biometeorological indices are, in fact, very important in biometeorology and medical bioclimatology, and they are calculated with mathematical formulas that allow evaluation of different human physiological situations up to the physio-pathological expressions (from well-being to physiological discomfort for hot or humid cold). These indices are based on correlations between values expressed by various meteorological variables such as: temperature, relative humidity, wind, and atmospheric pressure. The same World Health Organization (WHO) [44] gives evidence to this issuing, underlying the monitoring of extreme weather events, especially related to the concept of heat stress.

A recent work [45] described a large number of thermal indices reported in scientific literature during the last century. Human health, well-being, comfort, and discomfort are the result of the diversified influence of many factors, one of which is the thermal state of the environment.

The indices taken into account and applied in the MNW Weatherness Project are: the thermo-hygrometric index (THI), [46], two winter indices, namely the Scharlau Winter (WS) and the New Wind Chill (NWC) [47,48] and two summer indices, the Heat Index (HI) [49] and the New Summer Simmer Index (NSSI) [50].

Currently, the Weatherness is an application which runs a subnet of Italian weather stations present in the MNW database, allowing the previous five biometeorological parameters to be shown in real time (Figure 6). With the gradual extension of the MNW meteorological observation platform, it has been possible to structure and incorporate the described indices in a complete system of derived information to be calculated in different climate–environmental modes. In addition, to generate a specific database, the value-index association is shown on real time maps with the possibility of plotting monthly graphs for each station. The index efficiency depends on quality control checks applied to the MNW meteorological observations, which are influenced by possible downstream problems of the system itself. The upgrade version, described in Section 2.5, will also certainly improve the accuracy of the Weatherness data.

The defined indices use formulas combining temperature and relative humidity, except for the NWC, which aggregates temperature and wind speed. THI roughly calculates the Thom Discomfort Index (TDI) value [51], using air temperature and relative humidity directly, thus excluding the wet bulb temperature from calculation. It leads directly to a seasonal bioclimatic classification and its extreme classes. Software, applicable to both hemispheres at middle and high latitudes, comes to estimate the real feeling of physiological discomfort for cold, which is greater when air relative humidity is close to the saturation point [52]. Another factor related to temperature and relative humidity is the wind speed: in such situations, the feeling of discomfort, especially during the winter season, occurs somewhat differently with situations of calm wind or with the presence of breeze. The summer thermal indices included in the MNW system concern the HI and NSSI. The HI is of particular interest, especially in the current phase of global warming and simultaneous aging of human population (in more economically developed areas). The HI index allows estimation of the physiological discomfort caused by meteorological conditions characterized by high temperatures and high levels of relative humidity. This index can be calculated for temperatures ≥ 27 °C and RH ≥ 40%. Finally, the NSSI is a recent thermal index included in the MNW database in its latest version. It is applied when temperature is ≥ 22 °C, and it is sensitive up to a temperature of 53 °C. It is, therefore, a thermal index aimed to describe the conditions of heat stress during the hot season. The MNW platform, in the latest update, counts 175 weather stations in the Italian studied area, selected by the MNW database in the Weatherness Project. The percentage distribution by geographical area is approximately the following: northern Italy (62.6%), central Italy (17.2%), southern Italy (13.2%), and the largest Italian islands (6.9%). Several selected observation points of the Ultra-Violet Index (UVI) have also been included as part of the MNW meteorological station system. This parameter is of particular importance for human health. A recent systematic review [53] was carried out over two decades of international research investigating awareness, comprehension, use, and health impact of the UVI index in several countries (USA, Canada, Europe, Australia, New Zealand).

4. Conclusions

The paper showed the open crowd-sourced weather system developed by the Meteonetwork association, which is a peculiar example of citizen science, born by passion in atmospheric sciences, meteorology, and climatology. One of its greatest results is the MNW automatic weather station network, gathering about 6500 stations from 42 different European countries. Out of this massive amount of weather data, almost 4800 meteorological sites archive their data at least once a day, and about 3400 of them are constantly real-time connected. Produced data and maps are freely shared online and via app, and accessible for scientific purposes as well.

A constant upgrade through the years has led to the implementation of a robust database and servers to accommodate the huge amount of data flow. Notwithstanding this, quality control of meteorological observations is not affected by the volume of ingested data, which is regularly checked and updated. The metadata about the type of siting and configuration of sensors published on the MNW web pages not only increase user confidence with data, but also ensure that contributors think more about the quality of data they are producing [54].

Downstream products such as maps, data, and all meteorological and climatological products about temperature, relative humidity, precipitation, wind, pressure, and solar and UV radiations in the last decade have been shared to many institutions for scientific purposes. Weatherness project, by the Università degli Studi di Milano in Italy, uses a subnet of MNW data in order to study the effects of climate on human health in accordance with the WHO, who nowadays pays particular attention to problems related to extreme weather events. Lastly, a data assimilation system to introduce MNW data into the WRF-ARW meteorological model in addition to the international synoptic weather networks has been developed in order to improve initial conditions of everyday forecasts over Italian territory.

From the MNW experience, crowdsourcing is now an appreciated tool for engaging the public and scientific research. If appropriate validation and quality control procedures are adopted and implemented, it has huge potential for providing an integrative valuable source of high temporal and spatial resolution real-time data, especially in regions where few observations currently exist, thereby adding value to science, technology, and society.