Reliability Assessment of Multi-Source TEC Maps over Brazil Using Ground Truth Validation

Abstract

Total Electron Content (TEC) maps allow the evaluation of the state of the ionosphere. There are many providers/sources of worldwide or regional TEC maps for the continuous monitoring of the ionosphere, which employ different GNSS monitoring networks for data acquisition, TEC calculation or interpolation methods for generating the maps, or different spatial and temporal resolutions and coverage. How reliable are TEC maps over Brazil? We employed TEC maps from four different providers for 2022–2024, in the growing phase of the current solar cycle 25. Seasonality is also taken into account. A systematic comparison of TEC maps over Brazil was performed using correlation and similarity analysis between maps of different sources. Significant differences were found. Even for the same source there are differences in the density of monitoring stations according to the region. An example of bubble signature in TEC maps is also analyzed. Ground truth validation of TEC is performed by comparing TEC point values extracted from the maps with values derived from a set of GNSS stations over Brazil. As a result, no TEC maps of these sources were deemed reliable, due to low spatial and/or temporal resolution, low monitoring station density, or inadequate interpolation scheme.

1. Introduction

The Total Electron Content (TEC) field allows the evaluation and modeling of the state of the ionosphere, which is affected by the space weather. Two-dimensional TEC maps are generated using vertical TEC values derived from sets of Global Navigation Satellite Systems (GNSS) stations. Time series of TEC maps are used to identify the formation and the evolution of ionospheric bubbles over time. There are many methodologies [1,2,3,4,5,6,7,8,9,10] being employed by scientific institutes or governmental organizations for the continuous monitoring of the ionosphere by means of TEC maps, here denoted as TEC map sources or providers. Differences between maps are expected since sources may employ different GNSS monitoring networks for data acquisition, TEC calculation methods or interpolation methods for generating the maps. In addition, TEC maps sources provide maps with different spatial and temporal resolutions. There is an abundance of scientific works using TEC maps, but they usually employ a single source, which is assumed to be the most convenient. Because of the variety of TEC map sources, understanding how consistent and accurate such maps are, especially over low-latitude regions like Brazil, seems to be important.

Considering TEC maps over Brazil, how reliable are the TEC maps of different sources/providers? This work aims to provide a systematic comparison of TEC maps from four major sources by statistical correlation and image similarity analysis. In addition, a validation with ground truth is performed using a set of GNSS stations. Thus, this work aims to verify which source of TEC maps, if any, is more suitable for ionosphere monitoring over Brazil. This assessment would be useful for research of the ionosphere and for GNSS-based applications. The four sources/providers of TEC maps employed here are the IGS (International GNSS Service, referred to as IGS for simplicity), the Brazilian INPE/EMBRACE program, referred to as EMBRACE for simplicity (INPE stands for Brazilian National Institute for Space Research, and EMBRACE, for Studying and Monitoring of the Brazilian Space Weather), the Argentinian UNLP-FCAGLP/MAGGIA, referred to as MAGGIA for simplicity (MAGGIA stands for Meteorología espacial, Atmósfera terrestre, Geodesia, Geodinámica, diseño de Instrumental y Astrometría, and UNLP, for Laboratory of the La Plata National University, while FCAGLP stands for Faculty of Astronomical and Geophysical Sciences of the La Plata National University), and the Japanese University of Nagoya/Institute for Space–Earth Environment Research (ISEE), referred to as Nagoya for simplicity. Even for the same source there are significant differences in the density of monitoring stations around the globe, or even for different regions in Brazil. TEC maps employed here encompass the years 2022 to 2024, in the current solar cycle 25 growing phase. The TEC maps of each source were selected according to their temporal and spatial availability, considering coverage of the Brazilian territory. The months March, June, September, and December were chosen for each of these years to represent the seasonal behavior of the ionosphere. In a similar way, some times of the day were selected to represent TEC variation along the 24 h.

The proposed comparison required a search for temporal and spatial gaps of the time series of maps of each source, which are due to lack of data, thus filtering the available maps. Each map is represented as a matrix that contains TEC point values for the considered [latitude × longitude] grid. In this text, such matrices are referred to as TEC map data. A filtering concerning available time periods was performed to ensure overlapping periods, followed by an interpolation to obtain a unified spatial resolution. The TEC map data of these four sources over Brazil are then compared by means of the Pearson correlation coefficient ($\rho$) and the Structural Similarity Index Measure (SSIM) values between each pair of maps and between each pair of sources. Significant differences were found, i.e., low correlations between pairs of maps.

In order to show how TEC maps could be potentially employed for detecting ionospheric bubble signatures, an example is also presented, comparing the sequence of three TEC maps to the corresponding maps of amplitude scintillation and rate of TEC index (ROTI) maps.

A foremost analysis is the validation with ground truth of the TEC maps of the four sources, in which TEC point values derived from a set of GNSS stations in Brazil are used as ground truth. It is worth noting that the point values extracted from the TEC maps are subjected to interpolation errors.

This work draws the conclusion that none of the TEC maps provided by these four sources is reliable, due to one or more of the following reasons: (i) low spatial and/or temporal resolution, (ii) low density of GNSS monitoring stations employed to generate the maps, and (iii) inadequate interpolation scheme.

The proposed comparative analysis is particularly significant in regions where ionospheric irregularities, such as equatorial plasma bubbles, are prevalent and could drastically degrade GNSS signal quality [11]. Despite numerous TEC map products, few research works have performed a comprehensive comparative investigation over a South American country using homogeneous spatiotemporal standards and ground truth validation.

2. Materials and Methods

Comparing TEC maps of different sources that have different spatial and temporal resolution is a difficult task. IGS, Nagoya, EMBRACE, and MAGGIA maps were compared for the selected months of the 2022–2024 period, which are March, June, September, and December. These four TEC map sources are described in Section 2.2. The matrices composed of the set of TEC values for the spatial grid of the maps are denoted here as TEC map data files. All these data files were time-filtered and interpolated to a unified grid to allow direct comparisons between pairs of maps of different sources for each day/time considering the Brazilian territory and surrounding countries.

The flowchart of the adopted methodology is shown in Figure 1, composed of the steps of data acquisition of the available TEC maps, preprocessing, spatiotemporal pairing, and comparative analysis. The step of data acquisition and preprocessing involves downloading from Internet sites of the TEC map sources/providers. This step also includes the storage of the corresponding TEC values for each grid point in files with a convenient format. The spatiotemporal pairing, described in Section 2.3 (“Generation of TEC Maps”), is achieved by following the procedures shown in Figure 1: (i) temporal filtering that selects periods in which maps overlap in time; (ii) sampling for the same time resolution; (iii) mask application for the Brazilian territory; (iv) interpolation to a common grid. Procedures (i) and (ii) appear in Section 2.3.1 (“Matching of Temporal Coverage and Temporal Resolution”). Procedures (iii) and (iv) appear in Section 2.3.2 (“Interpolation to a Common Grid over Brazil”). The temporal filtering procedure (i) allowed the generation of 2 different datasets related to the TEC maps, according to the chosen months or the chosen set of TEC map sources. These datasets, denoted as TFs (Time-Filtereds), are TF-120 and TF-30, which have time resolutions of 120 and 30 min, respectively, and both undergo the same remaining procedures (ii, iii, and iv). As a result, these two datasets contain TEC maps with a unified spatial and temporal resolution, enabling the comparative analysis shown in Section 2.4 (“Procedure for the Comparison of TEC Maps”). The proposed analysis is composed of the Pearson correlation and similarity calculation between pairs of maps, an example of bubble signatures in TEC maps for a particular event, and the validation of TEC point values extracted from the maps for a set of GNSS stations using values of the Gopi Seemala application [12].

2.1. TEC Maps and the Earth’s Ionosphere

Earth’s ionosphere is affected by the space weather, which is related to the Earth’s interaction with outer space. The biggest effect is due to the Sun, since Solar activity influences Earth’s magnetic and electric fields, which influence the ionosphere. The ionosphere is a layer of the atmosphere that ranges from 50 to 1000 km of altitude composed of plasma, i.e., ions and free electrons. Plasma density varies with altitude because it depends on the intensity of the solar ionizing radiation, plasma absorption by atmospheric gases through ion recombination, and electron-capture processes. Dynamic processes like diffusion, plasma drifts and neutral winds also affect the plasma density profile. In the ionosphere, there is a competition between ionization production due to the Sun’s radiation and ionization losses due to the recombination of ions and electrons. The ionospheric ionization is much larger during sunlight hours than nighttime hours when there is no more solar radiation. The ionosphere is also influenced by seasonality, geographic location, phase of the solar cycle, and magnetic storms [13,14].

Radio frequency signals between satellites and ground receivers are affected by the ionosphere, mainly by perturbations known as ionospheric scintillation, caused by irregularities of ionospheric density. The most important radio signals are from GNSS, constellations of satellites that provide precise geographic position to receivers in aircrafts, ships, vehicles, and ground stations. Once the GNSS receiver acquires a lock with at least four satellites, its position is calculated by triangulation from each estimated receiver–satellite distance. Currently operational GNSS are the American Global Positioning System (GPS), the Russian Global Navigation Satellite System (GLONASS), the Chinese BeiDou Navigation Satellite System (BDS), the European Galileo, and the Japanese Quasi-Zenith Satellite System (QZSS) [15].

Ionospheric irregularities span a wide range of spatial and temporal scales and can seriously affect the performance of satellite-based navigation systems such as GNSS. While TEC may be used to characterize large-scale ionospheric characteristics, small-scale plasma irregularities have the ability to produce signal phase and amplitude variations, known as scintillation. Typical work, for instance [16], has provided the theoretical and experimental foundation to understand the generation of irregularity, particularly at low latitudes. Contemporary research, for instance, [17,18], has employed ground- and space-based GNSS measurements to outline the multi-scale nature of the irregularities. After decades of research, accurate monitoring and prediction of such phenomena remain difficult, especially within equatorial latitudes [19]. Some studies explore the potential of using remapped TEC images to capture ionosphere bubble signatures [20].

The TEC field is estimated from dual-frequency measurements of phase delay or adding the pseudorange made by receivers of networks of ground stations. Worldwide or regional TEC maps are obtained by interpolating local TEC values derived from measurements acquired by these receivers. Mathematical models are also used to generate the TEC field and the corresponding maps, but are usually not accurate due to low temporal or spatial resolutions and lack of assimilation of data from GNSS stations.

During daylight, the equatorial ionospheric plasma is lifted due to eastward electric fields. After sunset and until about 21 LT (Local Time) this upward plasma movement is intensified, giving rise to a large vertical electron density gradient at the ionospheric F region base, which is one condition for plasma irregularities to be generated through the Rayleigh–Taylor mechanism [21].

One ionospheric anomaly is the Equatorial Ionization Anomaly (EIA), which occurs about 15° magnetic degrees north and south of the dip equator, characterized by larger background electronic density during the pre-reversal hours (18–21 LT) compared to lower latitudes [22]. The TEC field at EIA shows crests with high gradients, which are associated with the eventual occurrence of strong scintillation.

The most common of the ionospheric irregularities are the ionospheric bubbles, low-density structures (low TEC values) that are formed near the magnetic equator or at low magnetic latitudes, mostly in the equinoxes and summer of years with high solar maximum activity. These bubbles are created 1-to-2 h after sunset, expand along the magnetic north and south directions, then migrate to the magnetic south and west directions. They disappear before midnight and sometimes 1-to-2 h after midnight. Inside the equatorial plasma bubbles, small-scale ionospheric plasma irregularities are generated, causing amplitude and phase scintillation which affects RF communication in the L band, for instance GNSS navigation. Ionospheric scintillation is measured by the amplitude and phase indexes. Amplitude scintillation is given by the S4 index, calculated as the normalized standard deviation of the GNSS signal intensity for 1 min with a sampling rate of 50 Hz [23]. Phase scintillation is given by the ${\sigma }_{\phi }$ index, calculated as the standard deviation of the detrended phase in each 1-min interval [23,24]. Scintillation typically occurs in the months of September to March in Brazil.

Besides the increased frequency of bubbles in the equinoxes and summer, there is a high variability throughout the year, making it difficult to predict their occurrence. TEC gradients in a bubble can reach 30–50 TEC units (TECU) for some hundreds of kilometers [1,2]. Similar to scintillation, these bubbles occur in general between the months of September and March in Brazil, presenting high TEC gradients and affecting the ionosphere refraction index. The bubbles may cause scintillation, disturbing GNSS and telecommunication signals in amplitude and phase [25].

2.2. TEC Maps Sources

The ionosphere can be modeled by the TEC field, depicted by latitude–longitude maps. Values of TEC in these maps are obtained assuming the ionosphere as a thin layer at a defined altitude. Slant TEC (STEC) is defined as the line integral electron density ($n_e$) given by the line integral along the signal slant path $s$ from receiver to satellite, considering a column of one-meter-square cross section centered on the path. STEC is given in TEC units, defined as 1 TECU = 10¹⁶ electrons/m², as shown below:

$$STEC=\int n_{e}ds$$

(1)

Typical TEC values measured near the Earth’s surface range from about 1 to 150 TECU [26] with the actual value depending on the geographic location, local time, season, solar flux, and magnetic activity. A relative STEC can be estimated by observing carrier phase and pseudorange of the received RF signals recorded by multi-frequency GNSS receivers. After leveling STEC values derived from the carrier phase using those derived from the pseudorange, these STEC values are corrected for the ground receiver and the satellite biases to obtain the absolute STEC values. Finally, the vertical projection of these values results in the vertical TEC values (VTEC), here referred to as TEC.

Better TEC maps are derived using higher temporal and spatial resolutions, but global maps may have accuracy compromised by TEC data provided by low density of TEC monitoring stations according to the region. In the case of TEC map prediction, a long time series may be required, and the extension of the time series varies widely for different sources. TEC map prediction using machine learning requires a long time series to perform the training of a neural network (most common case). Another point is the need to have a time series that covers entire solar cycles, not only solar maximum or solar minimum. However, this study covers only the current growing phase of the solar cycle 25. Two global and two regional TEC map sources are analyzed here.

Table 1. TEC maps sources of this study.

Source	Spatial Resolution (Lat × Long)	Temporal Resolution (min)	Spatial Coverage	TEC Map Extension
IGS (https://igs.org/products/#ionosphere accessed on 10 May 2025)	2.5° × 5°	120	Global	(87.5° S to 87.5° N; 180.0° W to 180.0° E)
INPE/EMBRACE (https://www2.inpe.br/climaespacial/portal/tec-map-sobre/ accessed on 10 May 2025) and https://www2.inpe.br/climaespacial/portal/tec-map-inicio/ accessed on 10 May 2025)	2° × 2.5°	10	South America	(60.0° S to 20.0° N; 90.0° W to 30.0° W)
MAGGIA (https://www.maggia.unlp.edu.ar/productos_maggia_alta accessed on 10 May 2025)	0.5° × 0.5°	15 (until 4 September 2024) 10 (since 5 September 2024)	Central and South America, the Caribbean and Antarctic Peninsula	(80.0° S to 40.0° N; 110.0° W to 0°)
University of Nagoya (https://stdb2.isee.nagoya-u.ac.jp/GPS/GPS-TEC/index.html accessed on 10 May 2025)	0.5° × 0.5°	5	Global	(89.9° S to 89.6° N; 180.0° W to 180.0° E)

describes these four TEC map sources.

Detailed information about these four TEC map sources is available in Appendix A (Appendix A.1, Appendix A.2, Appendix A.3 and Appendix A.4).

2.3. Generation of TEC Maps

This study does not compare TEC maps of different sources directly, since there are differences in temporal and spatial resolutions, time, and area coverage that would make such comparison very subjective. The rest of this section explains the steps employed to perform such comparisons.

An initial step is to download and process data from each source in the corresponding formats (for instance, IONosphere Map EXchange Format - IONEX). The resulting processed data are TEC map matrices, one for each map, in the related original grid resolution. Each value of the matrix refers to a specific latitude and longitude. Outliers and negative TEC values were not treated, since they are very scarce, less than 1% in the worst cases. On the other hand, NaN (not a number) values are identified if not already indicated as such. The percentage of TEC NaNs in the original maps surpassed 50% according to the map, due to incomplete coverage or unavailability of data. This issue can be seen, for instance, in Figure A1 in Appendix B, which presents original TEC maps for the 2024 Solstices and Equinoxes. However, after the temporal filtering and interpolation to a unified grid, the remaining NaNs of the original maps of grid points were replaced by zero TEC values, as can be seen in the corresponding TEC maps that appear in Section 3.3. Otherwise, NaNs that were surrounded by valid TEC values were interpolated using neighboring grid points. As a result, no NaN value persisted in the interpolated/filtered grid point values of datasets TF-120 and TF-30.

Each original TEC map matrix is then processed to obtain the corresponding new TEC map matrix with unified temporal and spatial resolutions and for nearly identical areas covering Brazil. This allows for a direct comparison between matrices of TEC values of different sources for the same time. This section details the main steps that were required.

2.3.1. Matching of Temporal Coverage and Temporal Resolution

The limited availability of TEC maps from the four sources for the selected four months of March, June, September, and December of years 2022, 2023, and 2024 has resulted in the use of maps of only three of the selected months for 2022 (all except December), none in 2023, and maps of all four of the selected months for 2024. Even for the months with available maps, some days were not available. The complete description of the available maps of the four sources for days/months is available in Appendix B (

Table A1. Temporal coverage of TEC maps (DOY denotes day-of-the-year).

Period	IGS	INPE	MAGGIA	Nagoya
March/ 2022	Complete	Complete	DOYs 68, 72 and 87 to 89 incomplete	Complete
June/ 2022	Complete	Complete	DOYs 155, 175, 177, 179 and 180 incomplete	Complete
September/ 2022	Complete	Complete	DOYs 244, 257, 259 and 263 incomplete	Complete
March/ 2024	Complete	Complete	DOYs 82 to 85 incomplete	DOY 90 missing
June/ 2024	Complete	Complete	DOYs 172, 179 and 182 incomplete	DOYs 178 and 179 incomplete
September/ 2024	Complete	Complete	DOYs 259, 260 and 273 incomplete	DOYs 245 and 246 incomplete
December/ 2024	Complete	DOY 348 incomplete and 366 missing	Complete	Complete

). These months (except for June) were chosen as being more likely to present more ionospheric irregularities and, thus, more ionospheric scintillation, since better TEC maps are expected to show such irregularities that correspond to high TEC gradients.

In order to cope with the different temporal coverages between the TEC map sources, two derived datasets, denoted as TFs datasets, were generated from the different TEC map sources, according to the following filtering criteria. For each source, the data corresponds to matrices of TEC values for the map grid. Two specific times of the day (08:00 and 16:00 UT - Universal Time) were selected to check the increasing TEC values along the day and a specific period of 20:00 to 04:00 UT was chosen as being more likely to present ionospheric scintillation, which is associated with high TEC gradients and, thus, high TEC values.

• Dataset TF-120: Data intersection for 120 min resolution, considering only complete data between the four TEC map sources for the months (7) of March/2022, June/2022, September/2022, March/2024, June/2024, September/2024, and December/2024, and for the times 08:00, 16:00, 20:00, 22:00, 00:00, 02:00, and 04:00 UT (7 times of day), resulting in 183 days of valid data, totaling to 183 × 7, or 1281 maps overall for each of the four sources.
• Dataset TF-30: Data intersection for 30 min resolution, considering only complete data between the MAGGIA, Nagoya, and EMBRACE TEC map sources, which have higher temporal resolution than IGS, for the months (4) March/2024, June/2024, September/2024, and December/2024, and for the times 08:00, 16:00, 20:00, 20:30, 21:00, 21:30, 22:00, 22:30, 23:00, 23:30, 00:00, 00:30, 01:00, 01:30, 02:00, 02:30, 03:00, 03:30, and 04:00 UT (19 times of day), totaling to 106 × 19, or overall 2014 maps overall for each of the three sources.

2.3.2. Interpolation to a Common Grid over Brazil

The TEC map matrices described above are used to generate the corresponding new TEC map matrices with a unified spatial resolution of 1° × 1°. A common area covering Brazil and part of some neighboring countries was defined as 39° S to 9° N, and 78° W to 30° W, which required cropping each map to fit this area. The resulting unified TEC map matrix for each map has 49 × 49 elements.

This was achieved by mapping the grid of each original TEC map matrix of the original grid to the grid values of the unified grid. Such mapping is performed using a standard interpolation technique, interpolation by Inverse Distance Weighting (IDW), adjusting the different spatial resolutions of the different sources, some coarser and some finer, into the unified 1° × 1° resolution in latitude and longitude. Aiming to perform a fair interpolation at the borders, IGS and Nagoya data were adjusted to the original MAGGIA area coverage.

The IDW interpolation computes the value of each unified grid point, taking into account the values of its neighboring original grid points, defined as the grid points that fall inside a given radius of influence.

The IDW radius of influence allows for smooth interpolation, taking into account TEC values of neighboring grid points. Furthermore, there is no propagation of TEC values in neighboring areas with no valid grid points, nor production of image artifacts. This is the case for areas over the Amazonian jungle and areas far from the Brazilian coast, mainly the northeast coast.

IDW is a deterministic interpolation method suitable for interpolating a scattered set of input points. Points without known values can be assigned with a weighted average of the known point values. This interpolator yields the value u at a given point $x$ based on the samples ${u_i=u(x}_i)$ for $i=1,2,\dots ,N$ using the IDW function below:

$$u\left(x\right)={\displaystyle \frac{{\sum }_{i=1}^N{w}_i\left(x\right)u_i}{{\sum }_{i=1}^N{w}_i\left(x\right)}}$$

(2)

The above equation applies if $d\left(x,x_i\right)\ne 0$ for all $i$. Otherwise, $u\left(x\right)$ returns $u_i$ itself (a point with known value). In the same equation, the term $w_i(x)$ is defined by:

$$w_i(x)={\displaystyle \frac{1}{d^p(x,x_i)}}$$

(3)

In the above equation, $x$ is the point to be interpolated, $x_i$ is a known point, $d$ is the distance from every known point to the point to be interpolated, $N$ is the number of known points, and $p$ is a real positive number called power factor. In short, the value of every interpolated point is estimated from the values of all known points. An influence radius $R$ defines the neighboring points that will be considered. It can act as a smoothing factor. This is performed by modifying the function $w_k\left(x\right)$ as follows:

$$w_k(x)={\left({\displaystyle \frac{max(0,R-d(x,x_k\left)\right)}{Rd(x,x_k)}}\right)}^2$$

(4)

The distance metric employed in the IDW interpolation was the Haversine. Even considering the Earth as a perfect sphere, distances along a parallel direction shorten as latitude increases, going away from the Equator. The Haversine distance, or great circle distance is defined to calculate the distance $D(x,y)$ between two points $x$ and $y$ at the surface of a sphere, with latitudes $x1$ and $y1$ and longitudes $x2$ and $y2$, given in radians:

$$D(x,y)=2arcsin\sqrt{{sin}^2\left(\right(x1-y1)/2)+cos\left(x1\right)cos\left(y1\right){sin}^2\left(\right(x2-y2)/2)}$$

(5)

2.4. Procedure for the Comparison of TEC Maps

In this work, the metrics chosen for comparing pairs of TEC map data are the Pearson correlation coefficient, and the SSIM. As previously mentioned, the proposed comparison between TEC maps of different sources by means of correlation is performed comparing the corresponding TEC map matrices adjusted for a unified temporal and spatial resolution and for the same coverage over Brazil and part of its neighboring countries. The SSIM comparison is performed using the raster images derived from these TEC maps. The results of the comparisons using the Pearson correlation coefficient and the SSIM are shown in Section 3. The datasets TF-120 and TF-30, used in the comparisons, were described in Section 2.3.1.

The first metric employed to compare TEC maps is the Pearson correlation coefficient [27], a well-known statistical measure that evaluates the strength of a linear relationship between each pair. It is given by the ratio of the covariance of the pair of TEC map data to the product of their standard deviations, resulting in a normalized value that ranges from −1 to +1. The value of +1 corresponds to a perfect linear correlation, while −1 indicates a perfect anticorrelation, and a value of zero is inconclusive, suggesting no linear relationship. Given a pair of matrices ($X,Y$) corresponding to TEC map data, the Pearson correlation coefficient $\rho$ is given by:

$${\rho }_{X,Y}={\displaystyle \frac{cov\left(X,Y\right)}{{\sigma }_X\ast {\sigma }_Y}}$$

(6)

Above, cov is the covariance, ${\sigma }_X$ is the standard deviation of matrix $X$, and ${\sigma }_Y$ is the standard deviation of matrix $Y$. Each pair of matrices, i.e., each pair of TEC map data yields a Pearson correlation value. However, it is not suitable to calculate the joint correlation of hundreds of matrices by simply calculating the average of the corresponding correlations, since they usually do not follow a normal probability distribution function. If a joint correlation with a confidence interval is desired, as in the current work, the Fisher transformation [28,29] can be employed to obtain a Fisher correlation coefficient $z$ for each Pearson value $\rho$ using the inverse hyperbolic arctangent function, also known as Fisher’s transformation:

$$z={\displaystyle \frac{1}{2}}\mathit{ln}\left({\displaystyle \frac{1+\rho }{1-\rho }}\right)$$

(7)

The set of Fisher correlation coefficients follows a less skewed distribution, and it is less susceptible to distributions with a small number of samples than the set of Pearson correlation coefficients. The application of the Fisher transformation is then followed by its inverse transformation to obtain a joint Pearson correlation value with, for instance, a 95% confidence interval, as in this work.

The second metric employed to compare TEC maps is the SSIM [30]. The comparison using the Pearson correlation is calculated using pairs of matrices corresponding to the maps. However, the SSIM is calculated using pairs of raster images, and thus the TEC maps to be compared were rasterized. SSIM is a standard technique originally developed to assess the quality of images by comparing a degraded image to the original one. However, it was further extended as a metric to evaluate the similarity between two images, considering structural information such as luminance, contrast, and spatial relationships. The SSIM is computed as follows, for the set of pixels of raster images $\mathrm{X}$ and $\mathrm{Y}$:

$${SSIM}_{X,Y}={\displaystyle \frac{\left[\right({2\mu }_X{\mu }_Y+C_1)\ast ({2\sigma }_{XY}+C_2\left)\right]}{\left[\right({{\mu }^2}_X+{{\mu }^2}_Y+C_1)\ast ({{\sigma }^2}_X+{{\sigma }^2}_Y+C_2\left)\right]}}$$

(8)

where

• ${\mu }_X$ and ${\mu }_Y$ are the mean values of the pixels of $\mathrm{X}$ and $\mathrm{Y}$, respectively;
• ${{\sigma }^2}_X$ and ${{\sigma }^2}_Y$ are the variances of $\mathrm{X}$ and $\mathrm{Y}$, respectively;
• ${\sigma }_{XY}$ is the covariance between $\mathrm{X}$ and $\mathrm{Y}$, respectively;
• $C_1$ and $C_2$ are small constants introduced to stabilize the computation when denominators are close to zero; these constants are defined in Equation (9), where $K_1$ and $K_2$ are algorithm parameters (default values of 0.01 and 0.03, respectively), and $D$ is the dynamic range of the pixel values.

$$C_1={(K_1\ast D)}^2\mathrm{a}\mathrm{n}\mathrm{d}{C}_2={(K_2\ast R)}^2$$

(9)

SSIM values range from −1 to +1, with a value of +1 indicating a perfect similarity between the compared TEC images. In cases where either image exhibits negligible variance, as in the case of uniform or near-uniform TEC images, the SSIM code adjusts its computation by comparing the mean absolute difference normalized by the dynamic range of the data. A joint SSIM value combining many SSIM values is also obtained by applying the Fisher transform and its inverse. SSIM provides an essential complement to the Pearson correlation coefficient, focusing on structural fidelity rather than just the linear relationships of the correlation. Thus, the use of both metrics, Pearson correlation coefficient and SSIM, allows a comprehensive comparison of TEC maps.

2.5. Procedure for the Validation of TEC Maps with Local Values

The validation of TEC maps shown in Section 3.4 compares local TEC values derived from a set of GNSS stations, assumed as ground truth, with the corresponding local values extracted from the maps. Besides the statistical metrics described in the former section, Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Pearson correlation coefficient, this validation employs two complementary statistical metrics, the Taylor Skill Score (TSS) and the Kling–Gupta Efficiency (KGE). In the definitions shown below, estimated values refer to TEC values extracted from the maps, while true values refer to TEC values given by GNSS stations.

The TSS is a metric derived from the Taylor Diagram that simultaneously evaluates the correlation and relative variability between estimated and true values [31]. The TSS ranges from 0 to 1, where 1 represents perfect agreement and is defined as shown below:

$$\mathrm{T}\mathrm{S}\mathrm{S}={\displaystyle \frac{4\left(1+\rho \right)}{{({\widehat{\sigma }}_f+1/{\widehat{\sigma }}_f)}^2(1+{\rho }_0)}}$$

(10)

where

• $\rho$ is the Pearson correlation coefficient between estimated and true values;
• ${\widehat{\sigma }}_f$ represents the normalized standard deviation given by the ratio between the standard deviations of estimated and true values;
• ${\rho }_0$ is a reference correlation value, often defined as the unity.

The KGE [32] represents a multi-component objective function developed for hydrological modeling, indicating goodness-of-fit between modeled and observed values. It evaluates three distinct components between modeled and observed values: correlation, variability bias, and mean bias. The adopted KGE formulation, applied here for estimated and true values, is given by:

$$\mathrm{K}\mathrm{G}\mathrm{E}=1-\sqrt{{(\rho -1)}^2+{(\alpha -1)}^2+{(\beta -1)}^2}$$

(11)

where

• $\rho$ is the Pearson correlation coefficient between modeled and true values;
• $\alpha$ = ${\sigma }_p$/${\sigma }_o$ is the variability ratio (ratio of standard deviations);
• $\beta$ = ${\mu }_p$/${\mu }_o$ is the bias ratio (ratio of means);
• ${\sigma }_p$ and ${\mu }_p$ are the standard deviation and mean of modeled values;
• ${\sigma }_o$ and ${\mu }_o$ are the standard deviation and mean of true values.

KGE values range from unity to negative values. A perfect KGE has unity value, indicating ideal performance across all three components ($\rho$ = 1, $\alpha$ = 1, $\beta$ = 1). Decreasing, less than unity, KGE values indicate deteriorating performance. In the context of modeling, negative values suggest that the mean of observations is a better estimator than the model.

3. Results

3.1. Correlation and Similarity Results

In all presented results, “correlation” always refers to the Pearson correlation coefficient, and “similarity” refers to the SSIM. Correlations are calculated for each pair of matrices, i.e., each pair of TEC map data, and for all sources of TEC data, while SSIM is calculated analogously, but for the corresponding raster images of the TEC maps. The following sections present results using the TF-120 and TF-30 datasets. In the latter, correlations considering TEC values equal to or above the third quartile (Q3) of the considered TEC values are also shown. In each comparison, the set of resulting correlations is reduced to a single correlation value using the Fisher transformation and its inverse.

3.1.1. Correlation Results Using the TF-120 Dataset

Correlations are calculated between pairs of TEC map data with unified spatial and temporal resolution, i.e., between the corresponding matrices with TEC grid values for each time. All four sources are considered, and the time resolution is 120 min. Using the TF-120 dataset, comparisons are performed for a given source with all other sources and also for a given source with each one of the other sources. In the first case, there are 3843 comparisons between pairs of maps, while in the second case, 1/3 of that value, i.e., 1281 comparisons. The latter is broken down by month as 182 for March 2022, September 2022, March 2024 and June 2024; 175 for June 2022 and September 2024; and 203 for December 2024.

In the first case, the joint correlations for the 3843 comparisons between pairs of maps of each source with the remaining three sources using TF-120 are

• MAGGIA × All other sources:    0.80;
• IGS × All other sources:               0.79;
• EMBRACE × All other sources:  0.64;
• Nagoya × All other sources:        0.63.

In the second case, the complete set of joint correlations for each of the 1281 comparisons of each source with the other three, by month and for all months using TF-120, is available in Appendix B (

Table A2. Joint correlations of TEC map sources using TF-120 (best value is marked in blue, and the worst, in red; all values are repeated in function of the source, for clarity).

Source *	All Months	March 2022	June 2022	September 2022	March 2024	June 2024	September 2024	December 2024
EMBRACE × Nagoya	0.67	0.75	0.80	0.75	0.57	0.70	0.49	0.51
EMBRACE × MAGGIA	0.64	0.70	0.75	0.71	0.62	0.70	0.55	0.40
EMBRACE × IGS	0.62	0.67	0.71	0.69	0.58	0.69	0.54	0.42
IGS × Nagoya	0.61	0.66	0.72	0.62	0.57	0.67	0.50	0.53
IGS × EMBRACE	0.62	0.67	0.71	0.69	0.58	0.69	0.54	0.42
IGS × MAGGIA	0.95	0.93	0.97	0.96	0.96	0.98	0.92	0.80
MAGGIA × Nagoya	0.62	0.68	0.77	0.66	0.59	0.68	0.50	0.43
MAGGIA × EMBRACE	0.64	0.70	0.75	0.71	0.62	0.70	0.55	0.40
MAGGIA × IGS	0.95	0.93	0.97	0.96	0.96	0.98	0.92	0.80
Nagoya × MAGGIA	0.62	0.68	0.77	0.66	0.59	0.68	0.50	0.43
Nagoya × IGS	0.61	0.66	0.72	0.62	0.57	0.67	0.50	0.53
Nagoya × EMBRACE	0.67	0.75	0.80	0.75	0.57	0.70	0.49	0.51

* Reciprocal correlation (x and y, y and x) are equal, such correlations were repeated to enhance clarity.

).

3.1.2. Similarity Results Using the TF-120 Dataset

SSIM values are calculated between pairs of raster images derived from the TEC map data with unified spatial and temporal resolution. All four sources are considered and the time resolution is 120 min. Using the TF-120 dataset, SSIM comparisons are performed for a given source with all other sources and also for a given source with each one of the other sources. In the first case, there are 3843 comparisons between pairs of maps, while in the second case, 1/3 of that value, i.e., 1281 comparisons. The latter is broken down by month as 182 for March 2022, September 2022, March 2024 and June 2024; 175 for June 2022 and September 2024; and 203 for December 2024.

In the first case, the joint SSIM values for the 3843 comparisons between pairs of maps of each source with the remaining three sources using TF-120 are

• MAGGIA × All other sources:    0.82;
• IGS × All other sources:               0.81;
• EMBRACE × All other sources:  0.79;
• Nagoya × All other sources:        0.75.

In the second case, the complete set of joint SSIM values for each of the 1281 comparisons of each source with the other three, by month and for all months using TF-120, is available in Appendix B (

Table A3. Joint SSIM values of raster images generated from TF-120 maps (best value is marked in blue, and the worst, in red; all values are repeated in function of the source, for clarity).

Source *	All Months	March 2022	June 2022	September 2022	March 2024	June 2024	September 2024	December 2024
EMBRACE×Nagoya	0.87	0.83	0.95	0.84	0.72	0.87	0.68	0.69
EMBRACE×MAGGIA	0.84	0.83	0.92	0.81	0.78	0.84	0.73	0.64
EMBRACE×IGS	0.82	0.81	0.91	0.79	0.77	0.84	0.73	0.67
IGS×Nagoya	0.77	0.76	0.90	0.76	0.66	0.81	0.64	0.59
IGS×EMBRACE	0.82	0.81	0.91	0.79	0.77	0.84	0.73	0.67
IGS×MAGGIA	0.96	0.92	0.98	0.94	0.94	0.97	0.88	0.85
MAGGIA×Nagoya	0.80	0.79	0.92	0.78	0.68	0.81	0.64	0.57
MAGGIA×EMBRACE	0.84	0.83	0.92	0.81	0.78	0.84	0.73	0.64
MAGGIA×IGS	0.96	0.92	0.98	0.94	0.94	0.97	0.88	0.85
Nagoya×MAGGIA	0.80	0.79	0.92	0.78	0.68	0.81	0.64	0.57
Nagoya×IGS	0.77	0.76	0.90	0.76	0.66	0.81	0.64	0.59
Nagoya×EMBRACE	0.87	0.83	0.95	0.84	0.72	0.87	0.68	0.69

* Reciprocal correlation (x and y, y and x) are equal, such correlations were repeated to enhance clarity.

).

3.1.3. Correlation Results Using the TF-30 Dataset for the Q3

Correlations are calculated between pairs of TEC map data with a unified spatial and temporal resolution using the TF-30 dataset, but comparisons only apply for TEC values equal to or above the Q3 of each TEC source. All sources, except IGS, are considered and time resolution is 30 min. Thus, considering a given map of the considered TEC source, only the set of TEC grid values of that map equal to or above Q3 is used for the calculation of the correlation, which also uses the corresponding set of the grid values of the other source. In the first case, there are 4028 comparisons between pairs of maps, while in the second case, half of that value, i.e., 2014 comparisons. The latter is broken down by month as 494 for March 2024 and June 2024; 475 for September 2024; and 551 for December 2024.

In the first case, the joint correlations for the 4028 comparisons between pairs of maps of each source with the remaining two sources considering only TEC values above or equal to Q3 using TF-30 are

• EMBRACE × All other sources: 0.30;
• Nagoya × All other sources:       0.22;
• MAGGIA × All other sources:    0.17.

Also, for the first case, but for the full range of TEC values, the joint correlations for the 4028 comparisons between pairs of maps of each source with the remaining two sources using TF-30 are

• MAGGIA × All other sources:   0.61;
• EMBRACE × All other sources: 0.60;
• Nagoya × All other sources:       0.60.

In the second case, the complete set of joint correlations for each of the 2014 comparisons of each source with the other two, by month and for all months using TF-30, is available in Appendix B (

Table A4. Joint correlations of TEC map sources using TF-30, only for TEC values equal to or above Q3 (best value is marked in blue, and the worst, in red).

Source	All Months	March 2024	June 2024	September 2024	December 2024
EMBRACE×Nagoya	0.23	0.23	0.26	0.17	0.26
Nagoya×EMBRACE	0.12	0.13	0.76	0.11	0.15
EMBRACE×MAGGIA	0.36	0.36	0.46	0.33	0.30
MAGGIA×EMBRACE	0.12	0.18	0.05	0.10	0.16
Nagoya×MAGGIA	0.31	0.32	0.51	0.22	0.19
MAGGIA×Nagoya	0.22	0.19	0.40	0.13	0.13

). Please note that Q3 is different for each source, and correlations are calculated considering the first source as reference. Therefore, for instance, correlation between MAGGIA × EMBRACE is different from EMBRACE × MAGGIA, thus, yielding two values for each pair of sources.

Also, considering TEC values above or equal to Q3, the correlation values by month comprises 988 comparisons for March 2024 and June 2024, 950 for September 2024, and 1102 for December 2024 (total of 4028), shown in

Table 2. Joint correlations by month of pairs of TEC map sources using TF-30 for the full range of TEC values and for TEC values equal to or above Q3, shown as the first and second values in each cell, respectively (best value is marked in blue, and the worst, in red).

Source	All Months	March 2024	June 2024	September 2024	December 2024
EMBRACE × All	0.60/0.30	0.64/0.29	0.73/0.36	0.56/0.25	0.46/0.27
MAGGIA × All	0.61/0.17	0.67/0.18	0.72/0.21	0.59/0.10	0.43/0.14
Nagoya × All	0.60/0.22	0.62/0.23	0.72/0.31	0.54/0.15	0.47/0.17

. In addition, the same table shows joint correlations considering the full range of TEC values.

3.1.4. SSIM Results Using the TF-30 Dataset

Similarity comparisons using maps with TEC maps equal to or above Q3 would not be possible, since the SSIM calculation requires rectangular raster images corresponding to the same number of grid points. Thus, this section shows similarity results for the TF-30 dataset using maps with the full range of TEC values. As before, SSIM values are calculated between pairs of raster images derived from the TEC map data with a unified spatial and temporal resolution. All four sources are considered, and time resolution is 30 min. Using the TF-30 dataset, SSIM comparisons are performed for a given source with all other sources, and also for a given source with each one of the other sources. In the first case, there are 4028 comparisons between pairs of maps, while in the second case, half of that value, i.e., 2014 comparisons. The latter is broken down by month as 494 for March 2024 and June 2024; 475 for September 2024; and 551 for December 2024.

In the first case, considering the full range of TEC values, the joint SSIM values for the 4028 comparisons between pairs of raster images of each source with the remaining two sources using TF-30 are

• EMBRACE × All other sources: 0.71;
• MAGGIA × All other sources:   0.68;
• Nagoya × All other sources:       0.67.

In the second case, considering the full range of TEC values, the complete set of joint SSIM values for each of the 2014 comparisons between pairs of raster images of each source with the other two, by month and for all months using TF-30, is available in Appendix B (

Table A5. Joint SSIM values of raster images generated from TF-30 maps (best value is marked in blue, and the worst, in red; all values are repeated in function of the source, for clarity).

Source *	All Months	March 2024	June 2024	September 2024	December 2024
EMBRACE×Nagoya	0.75	0.67	0.87	0.63	0.64
EMBRACE×MAGGIA	0.74	0.74	0.82	0.69	0.61
MAGGIA×Nagoya	0.67	0.64	0.80	0.59	0.53
MAGGIA×EMBRACE	0.74	0.74	0.82	0.69	0.61
Nagoya×MAGGIA	0.67	0.64	0.80	0.59	0.53
Nagoya×EMBRACE	0.75	0.67	0.87	0.63	0.64

* Reciprocal correlation (x and y, y and x) are equal, such correlations were repeated to enhance clarity.

).

3.2. Case Study of Ionospheric Bubble Signatures on TEC Maps

As previously mentioned, this case study is presented as an example of how TEC maps can potentially be used to identify ionospheric bubble signatures. Due to the availability of maps for this test case, only EMBRACE and MAGGIA maps are employed, and their high temporal resolution allows the use of 10 min resolution.

Ionospheric bubbles are large ionization depleted structures that are generated at the magnetic equator. While drifting up at the magnetic equator, they extend north and south to larger latitudinal sectors, along the magnetic field lines. As TEC maps cover a wide range of latitudes and longitudes, they allow the identification of the bubbles signatures as rarefied TEC regions. The bubble zonal movement is normally eastward during magnetically quiet days, which can be observed at TEC maps using a temporal sequence of images. Although TEC reflects typically large-scale ionospheric structure predominantly, its rate of change, calculated from the ROTI, has proved to be useful in measuring medium- to small-scale irregularities. It has been recently demonstrated that greater values of ROTI reveal strong correlation with the occurrence of ionospheric bubbles [33,34].

Inside plasma bubbles, smaller scale size irregularities are generated, causing scintillation in GNSS signals. Therefore, ROTI maps and also amplitude scintillation maps (S4 maps) show regions with moderate/strong scintillation (S4 > 0.5, and ROTI > 0.14 TECU/s) that may be associated with the bubbles. In order to check if the TEC maps from different sources allow the identification of bubble signatures, the corresponding ROTI and S4 maps were generated. It is worth mentioning that these maps are more accurate, present less artifacts and provide better coverage in regions with low density of GNSS monitoring stations like Northern Brazil than the existing ROTI and S4 maps [35]. Interpolation errors were negligible in regions with denser coverage of stations based on validation tests using a set of specific stations.

A case study of ionospheric bubble signatures was chosen after a thorough search of scintillation occurrences in ROTI and S4 maps for the months of this study and the scan of the corresponding TEC maps provided by EMBRACE and MAGGIA, searching for bubbles extending to the south. A single occurrence of scintillation matching bubbles was selected, which can be observed in a sequence of EMBRACE TEC maps on 27 September 2024 at 00:50, 01:00, and 01:10 UT over the Brazilian territory. These bubbles cannot be clearly observed in the corresponding MAGGIA maps, despite their better 0.5° × 0.5° resolution in comparison with the coarser spatial resolution of 2.0° × 2.5° of the EMBRACE maps. Nagoya TEC maps were not considered due to the lack of data on this date, and IGS TEC maps were also not considered due to insufficient temporal resolution.

The value of the closest observation for the case study of the solar radio flux (F10.7 index), taken on 26 September 2024, at 23:00 UT in Kaleden, Canada, was 182.7 solar flux units (1 sfu = 10⁻²² W m⁻² Hz⁻¹). Considering the entire period adopted in this study from 1 March 2022 to 30 December 2024, the initial value of F10.7 was 104.0 sfu, the final value was 222.8, and the average value was 161.0 for 23:00 UT. Despite increasing solar activity in the period, clear signatures of ionospheric bubbles were observed in September, but not in December of 2024, considering the available maps.

Figure 2 shows the TEC, S4, and ROTI sequence of three maps for the 27 September 2024 test case of bubble occurrence. In this Figure, TEC maps are the original ones, adopting a TEC scale of 0 to 80 TECU for EMBRACE and 0 to 160 TECU for MAGGIA as shown in the vertical color bars, since the latter overestimates TEC values. The EMBRACE maps, shown in the first row of the Figure, allow observations of large TEC depleted structures, extending from the magnetic dip equator to about 20° S of dip latitude, but do not allow a clear identification of the bubbles. Larger TEC values are confined between 10° and 20° S, which is the EIA south crest latitudinal range for this period. Similar bubble signatures were presented [2] using the EMBRACE TEC maps for 15–16 February 2014, with an enhanced resolution of 0.5° × 0.5° but under demand. The second row of Figure 2 shows the corresponding MAGGIA TEC maps, which are much smoother, precluding even clearer identification of bubble signatures [20].

The third row of the same Figure shows the corresponding maps for the ionospheric scintillation index S4 that is the normalized standard deviation of the amplitude scintillation in the L1 frequency. Scintillation is caused by the ionospheric plasma irregularities generated inside the bubbles. Finally, the fourth row of Figure 2 shows the corresponding maps for ROTI. It required the L1 and L2 frequencies to be calculated, using phase observables with 1-s resolution, with ROTI values being calculated every minute.

It is worth to note that phase scintillation affects more ROTI maps than amplitude scintillation, and the ROTI were generated using a relatively dense subset of 51 GNSS stations of the Brazilian Network for Continuous Monitoring (RBMC) (https://www.ibge.gov.br/en/geosciences/geodetic-positioning/geodetic-networks/20079-brazilian-network-for-continuous-monitoring-gnss-systems.html?lang=en-GB&t=sobre accessed on 5 May 2025), provided by the Brazilian Institute of Geography and Statistics (IBGE). On the contrary, the S4 maps were generated using 30 GNSS stations from the Brazilian multi-institutional project National Institutes of Science and Technology (INCT) “GNSS Technology for Supporting Aerial Navigation” (GNSS-NavAer) network.

These S4 and ROTI maps show that areas with higher values correspond to the depleted regions of the EMBRACE TEC maps presented for the case study, providing evidence that TEC maps can be used to point out scintillation regions. This is very useful, since GNSS users can identify areas prone to the occurrence of scintillation, since the number of GNSS stations monitoring scintillation over Brazil is much lower than the number of GNSS stations monitoring TEC, mainly in the north of Brazil [36]. The ROTI and S4 scintillation maps were generated using a recent methodology that employs the Gaussian Process Regression (GPR) for interpolation and a specific set of preprocessing options, which corresponds to the use of the vertical projection of S4 values reduced by the average value above the Q3 [35,37]. This methodology was developed in the scope of the INCT GNSS-NavAer [38].

In Figure 2, it is possible to compare the depleted ionospheric zones (three maps on the first row), i.e., low TEC values, with zones that present scintillation (three maps on the third and fourth rows). The best match is observed for the EMBRACE maps. On the other hand, despite its higher resolution, MAGGIA TEC maps (second row, three maps) presented higher TEC values than the EMBRACE ones, leading to smoother images that prevent a direct comparison with the ROTI and S4 scintillation maps, even with MAGGIA maps adopting a TEC scale that has double the range of the EMBRACE scale.

3.3. Performance of TEC Maps for the 2024 Equinoxes and Solstices

This section presents TEC maps of the different sources to allow a visual and correlation/similarity comparisons for the two Equinoxes and the two Solstices of 2024. Both the TEC maps with original and unified spatial and temporal resolutions are shown (original ones appear in the Figure A1 of Appendix B).

In the Southern Hemisphere (https://www.timeanddate.com/calendar/seasons.html accessed on 20 May 2025), the Autumn and Spring 2024 Equinoxes occurred, respectively, on 20 March 2024 at 03:06 UT and 22 September 2024 at 12:43 UT, while the Winter and Summer 2024 Solstices occurred, respectively, on 20 June 2024 at 20:50 UT and 21 December 2024 at 09:20 UT. Due to the limited 2-h temporal resolution of the IGS maps, the adopted TEC maps were those with closer times on the same day. Another restriction was the unavailability of MAGGIA maps for 20 June 2024, forcing the adoption of maps from all sources for the next day. These maps, either the original or the unified/interpolated are denoted as

• AE24—near Autumn Equinox, on 20 March 2024 at 04:00 UT;
• WS24—next day of Winter Solstice, on 21 June 2024 at 00:00 UT;
• SE24—near Spring Equinox, on 22 September 2024 at 14:00 UT;
• SS24—near Summer Solstice, on 21 December 2024 at 10:00 UT.

Figure 3 shows the unified versions (same spatial and temporal resolution) of the AE24, WS24, SE24, and SS24 TEC maps, generated by EMBRACE, IGS, MAGGIA, and Nagoya. The corresponding original TEC maps of each source are available in Appendix B (Figure A1). TEC values vary according to the season of the year and the phase of the solar cycle, being also subjected to eventual variations during magnetic storms. A study of TEC seasonality is out of the scope of this study, since the focus here is to evaluate the accuracy of TEC maps of different sources. In the case of the above maps, TEC values above 50-60 TECU are only observed for SE24 as it is for 14 UT, while AE24 and WS24 are for night times, and SS24 for early morning.

The following Section 3.3.1, Section 3.3.2 and Section 3.3.3 show correlation, similarity, and Q3 results using the unified/interpolated AE24, WS24, SE24, and SS24 TEC maps taken as Equinoxes and Solstices samples for 2024, generated by EMBRACE, IGS, MAGGIA, and Nagoya.

3.3.1. Correlation Results for TEC Seasonability

Joint correlations were calculated for a given source with all other sources and also for a given source with each one of the other sources.

In the first case, the joint correlations between each map with the other three considering AE24, WS24, SE24, and SS24 TEC maps of each source, are

• MAGGIA × All other sources:   0.79;
• IGS × All other sources:              0.79;
• EMBRACE × All other sources: 0.63;
• Nagoya × All other sources:       0.52.

In the second case, the correlations for the comparisons between pairs of maps for each four-map set (AE24, WS24, SE24, SS24) of all sources are shown in Appendix B (

Table A6. Correlations for the unified/interpolated AE24, WS24, SE24, and SS24 TEC map sources (best value is marked in blue, and the worst, in red; all values are repeated in function of the source, for clarity).

Source *	All Dates	AE24	WS24	SE24	SS24
EMBRACE×Nagoya	0.58	0.59	0.52	0.44	0.73
EMBRACE × MAGGIA	0.64	0.84	0.61	0.41	0.60
EMBRACE × IGS	0.65	0.84	0.62	0.43	0.57
MAGGIA × Nagoya	0.49	0.67	0.56	0.10	0.53
MAGGIA × EMBRACE	0.64	0.84	0.61	0.41	0.60
MAGGIA × IGS	0.96	0.96	0.98	0.96	0.90
IGS × Nagoya	0.48	0.66	0.58	0.13	0.48
IGS × EMBRACE	0.65	0.84	0.62	0.43	0.57
IGS × MAGGIA	0.96	0.96	0.98	0.96	0.90
Nagoya × MAGGIA	0.49	0.67	0.56	0.10	0.53
Nagoya × IGS	0.48	0.66	0.58	0.13	0.48
Nagoya × EMBRACE	0.58	0.59	0.52	0.44	0.73

* Reciprocal correlation (x and y, y and x) are equal, such correlations were repeated to enhance clarity.

). Joint correlations for each source with the other three are also presented.

3.3.2. Similarity Results for TEC Seasonability

SSIM values were calculated for a given source with all other sources and also for a given source with each one of the other sources.

In the first case, the joint SSIM values, between each map with the other three AE24, WS24, SE24, and SS24 TEC maps of each source, are

• MAGGIA × All other sources:   0.78;
• IGS × All other sources:              0.77;
• EMBRACE × All other sources: 0.75;
• Nagoya × All other sources:       0.71.

In the second case, the SSIM values for the comparisons between pairs of maps for each four-map set (AE24, WS24, SE24, and SS24) of all sources, are shown in Appendix B (

Table A7. SSIM values for the raster images generated from the unified/interpolated AE24, WS24, SE24, and SS24 TEC maps (best value is marked in blue, and the worst, in red; all values are repeated in function of the source, for clarity).

Source *	All Dates	AE24	WS24	SE24	SS24
EMBRACE × Nagoya	0.83	0.76	0.91	0.70	0.88
EMBRACE × MAGGIA	0.73	0.67	0.76	0.80	0.67
EMBRACE × IGS	0.73	0.66	0.75	0.82	0.65
IGS × Nagoya	0.67	0.60	0.76	0.70	0.60
IGS × EMBRACE	0.73	0.66	0.75	0.82	0.65
IGS × MAGGIA	0.94	0.90	0.95	0.97	0.92
MAGGIA × Nagoya	0.68	0.61	0.76	0.69	0.62
MAGGIA × EMBRACE	0.73	0.67	0.76	0.80	0.67
MAGGIA × IGS	0.94	0.90	0.95	0.97	0.92
Nagoya × MAGGIA	0.68	0.61	0.76	0.69	0.62
Nagoya × IGS	0.67	0.60	0.76	0.70	0.60
Nagoya × EMBRACE	0.83	0.76	0.91	0.70	0.88

* Reciprocal correlation (x and y, y and x) are equal, such correlations were repeated to enhance clarity.

). Joint SSIM values for each source with the other three are also presented.

3.3.3. Q3 Correlation Results for TEC Seasonability

Joint correlations using TEC values equal to or above the Q3 were calculated for a given source with all other sources and also for a given source with each one of the other sources.

In the first case, the joint correlations between each map with the other three, considering AE24, WS24, SE24, and SS24 TEC maps of each source and only for TEC values equal to or above Q3 are

• Nagoya × All other sources:       0.43;
• MAGGIA × All other sources:    0.31;
• EMBRACE × All other sources: 0.30;
• IGS × All other sources:              0.21.

In the second case, all correlations for each set of four comparisons of each source with any of the other three are shown in Appendix B (

Table A8. Correlations for the unified/interpolated AE24, WS24, SE24, and SS24 TEC map sources considering TEC values equal to or above Q3 (best value is marked in blue, and the worst, in red).

Source	All Dates	AE24	WS24	SE24	SS24
EMBRACE × Nagoya	0.09	−0.14	0.20	0.07	−0.05
Nagoya × EMBRACE	0.32	0.29	−0.01	0.15	0.71
EMBRACE × MAGGIA	0.41	0.54	0.30	0.04	0.65
MAGGIA × EMBRACE	−0.09	0.03	−0.27	−0.09	−0.01
IGS × EMBRACE	−0.18	0.18	−0.32	−0.27	−0.29
EMBRACE × IGS	0.43	0.56	0.31	0.09	0.67
IGS × MAGGIA	0.72	0.43	0.97	0.74	0.15
MAGGIA × IGS	0.76	0.40	0.90	0.72	0.84
Nagoya × IGS	0.45	0.53	0.64	0.04	0.50
IGS × Nagoya	−0.10	0.29	0.29	−0.29	−0.59
Nagoya × MAGGIA	0.51	0.62	0.62	0.01	0.65
MAGGIA × Nagoya	0.04	0.12	0.34	−0.01	−0.29

).

In addition, for TEC values equal to or above Q3, joint correlations considering AE24, WS24, SE24, and SS24 TEC maps of all sources are shown in

Table 3. Joint correlations for the unified/interpolated AE24, WS24, SE24, and SS24 TEC maps, considering the full range of TEC values or values equal to or above Q3, shown as the first and second values in each cell, respectively (best value is marked in blue, and the worst, in red).

Source	All Months	AE24	WS24	SE24	SS24
EMBRACE × All	0.63/0.30	0.84/0.56	0.52/0.20	0.44/0.07	0.73/−0.05
IGS × All	0.79/0.21	0.84/0.18	0.98/0.97	0.43/−0.27	0.48/0.59
MAGGIA × All	0.79/0.31	0.96/0.40	0.98/0.90	0.10/−0.01	0.60/−0.01
Nagoya × All	0.52/0.43	0.66/0.53	0.58/0.64	0.44/0.15	0.48/0.50

. The same table shows correlations considering the full range of TEC values.

3.4. Validation of TEC Maps with Ground Truth

The validation of the TEC maps of the four sources is performed using the ground truth provided by the Gopi Seemala application, which generates local TEC time series for specific GPS monitoring stations. GPS raw data is formatted into Receiver INdependent EXchange Format (RINEX) 2 and RINEX 3 files that serve as input to the GPS-TEC analysis program (https://seemala.blogspot.com/2024/04/gps-tec-analysis-program-version-35.html accessed on 10 May 2025) [12]. In this section, it was assumed that local TEC measurements such as those of Gopi could be taken as reference as being more accurate than point values extracted from any TEC maps, since the latter embeds interpolation errors. Among other possible packages for TEC extraction like ICPT_TEC or Ionolab_TEC, the choice of the Gopi Seemala application was based on previous studies related to the ionosphere over Brazil, which included validation with independent observations such as ionosondes, all-sky imagers, and scintillation monitors [20,37].

In order to perform a comparison with local values extracted from the different sources of TEC maps, the Gopi Seemala application was employed to generate TEC time series from the RINEX 3 data (https://www.ibge.gov.br/geociencias/informacoes-sobre-posicionamento-geodesico/rede-geodesica/16258-rede-brasileira-de-monitoramento-continuo-dos-sistemas-gnss-rbmc.html?=&t=dados-diarios-e-situacao-operacional accessed on 5 May 2025) of RBMC GNSS stations in Brazil, assumed as local reference values. A set of error and accuracy metrics was calculated for the corresponding TEC values extracted from the TEC maps of each source: MAE, RMSE, Pearson correlation, TSS, and KGE.

Figure 4 shows the RBMC GNSS stations over Brazil, with a total of 147 that were operational as of 2024, denoting by red triangles the 27 that were employed in the validation, and by blue dots, the remaining ones (geographical coordinates of these stations are available in a repository: “Geographical_coordinates_of_the_27_selected_GNSS_stations.pdf” document in Zenodo at https://doi.org/10.5281/zenodo.15453941 accessed on 10 October 2025). The same figure shows ellipses that delimitate a cluster of stations over the Amazonia and over the Brazilian Southeast, each one with nine RBMC stations.

The validation was performed considering each pair of TEC values given by the Gopi Seemala reference value and the corresponding value extracted from the map, for the considered days and times of day. Thus, the error and performance metrics were calculated using all these pairs of values for each source. Two different validation cases were considered:

(A)

Validation of EMBRACE and MAGGIA TEC maps using TEC data from a reduced set of seven GNSS stations for the example of ionospheric bubble signature in the three TEC maps shown in Section 3.2 (27 September 2024 at 00:50, 01:00, and 01:10 UT). As already mentioned, Nagoya and IGS maps were not available for this case of study. The seven employed RBMC stations are BRAZ, CUIB, NAUS, POAL, SALU, SAVO, and SJSP, which are distributed over Brazil. The resulting metrics appear in

Table 4. Overall values for the set of metrics calculated for the EMBRACE and MAGGIA TEC values corresponding to the seven chosen GNSS stations, using Gopi Seemala values as ground truth, for 27 September 2024 at 00:50, 01:00, and 01:10 UT (MAE and RMSE values in TECU).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE	18.48	20.91	0.90	0.64	0.48
MAGGIA	23.11	25.60	0.97	0.96	0.49

and in Appendix B (

Table A9. TEC values for EMBRACE, MAGGIA, and Gopi Seemala (±standard deviation) corresponding to the seven chosen GNSS stations, for 27 September 2024 at 00:50, 01:00, and 01:10 UT (values in TECU).

Source	00:50 UT	01:00 UT	01:10 UT
Gopi (BRAZ)	91.93 ± 8.91	92.28 ± 7.67	93.38 ± 9.55
EMBRACE (BRAZ)	72.52	71.86	67.75
MAGGIA (BRAZ)	138.74	133.04	129.11
Gopi (CUIB)	73.90 ± 11.18	73.87 ± 10.38	74.22 ± 10.51
EMBRACE (CUIB)	36.89	43.34	42.93
MAGGIA (CUIB)	75.94	111.90	85.23
Gopi (NAUS)	19.83 ± 2.12	19.68 ± 1.55	19.47 ± 2.55
EMBRACE (NAUS)	28.84	32.11	34.07
MAGGIA (NAUS)	40.93	39.25	44.92
Gopi (POAL)	14.77 ± 4.29	15.80 ± 3.82	16.92 ± 3.35
EMBRACE (POAL)	33.90	33.01	34.13
MAGGIA (POAL)	29.80	27.86	26.85
Gopi (SALU)	21.98 ± 6.73	20.75 ± 6.62	19.45 ± 7.05
EMBRACE (SALU)	39.31	41.23	34.59
MAGGIA (SALU)	38.69	42.25	43.50
Gopi (SAVO)	101.46 ± 7.79	98.65 ± 8.53	94.63 ± 5.26
EMBRACE (SAVO)	69.32	73.81	73.59
MAGGIA (SAVO)	121.90	133.17	127.19
Gopi (SJSP)	30.89 ± 19.69	30.06 ± 11.62	29.80 ± 7.92
EMBRACE (SJSP)	31.64	30.80	28.02
MAGGIA (SJSP)	51.12	49.78	47.96

);

(B)

Validation of TEC maps of all four sources using TEC data from the enlarged set of 27 GNSS stations for all days of December 2024. The Amazonian cluster is composed of the following nine RBMC stations: AMTE, BELE, BOAV, NAUS, PAAR, POVE, RIOB, ROJI, and SAGA. The nine RBMC stations of the Brazilian Southeastern cluster are CHPI, MGIN, PPTE, SJRP, SJSP, SPBO, SPFR, SPTU, and UFPR. The remaining nine RBMC stations also employed in the validation are BRAZ, BRFT, CUIB, IMPZ, MSGR, POAL, SALU, SAVO, and TOPL. This analysis is detailed in the next section.

The more extensive validation (B) was performed for all times of the day (except for otherwise indicated) for all days of December 2024 and for all the 27 stations. The resulting error and accuracy metrics for the considered sources are shown in

Table 5. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for December 2024 (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE	9.83	13.85	0.84	0.91	0.77
IGS	16.34	18.18	0.93	0.96	0.55
MAGGIA	21.13	23.61	0.83	0.88	0.42
Nagoya	17.56	25.57	0.51	0.71	0.41

. In addition, considering the corresponding Gopi Seemala TEC values, it was possible to define three ranges of values: (i) Q-lower, containing all values below the first quartile, including it, (ii) Q-inter, with all interquartile values, and (iii) Q-upper, with all values above the Q3, including it. The specific metrics for these three ranges of values appear in

Table 6. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for December 2024, discriminated by quartiles (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE (Q-lower)	6.57	8.58	0.37	0.34	−0.64
EMBRACE (Q-upper)	8.66	13.33	0.28	0.35	−0.47
EMBRACE (Q-inter)	11.59	15.14	0.66	0.68	0.31
IGS(Q-lower)	16.24	17.49	0.49	0.42	−1.16
IGS(Q-upper)	13.85	15.12	0.52	0.73	0.42
IGS(Q-inter)	17.21	19.07	0.81	0.86	0.43
MAGGIA(Q-lower)	16.66	18.15	0.37	0.28	−1.52
MAGGIA(Q-upper)	23.18	25.62	0.25	0.22	−1.22
MAGGIA(Q-inter)	21.87	24.06	0.67	0.65	0.09
Nagoya(Q-lower)	8.40	10.36	0.25	0.20	−1.34
Nagoya(Q-upper)	26.27	37.88	0.02	0.06	−3.61
Nagoya(Q-inter)	17.59	22.84	0.38	0.42	−0.24

.

Considering the TEC values extracted from the maps of each source and the Gopi Seemala values taken as references for the two nine-station clusters (Amazonian and Southeastern), and for all times of the day and all days of December 2024, the resulting metrics are shown in

Table 7. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for December 2024, discriminated by the two clusters of stations (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE	Cluster
EMBRACE	6.40	10.60	0.91	0.95	0.88	Amazonian
	14.56	17.61	0.89	0.92	0.53	Southeastern
IGS	13.25	14.33	0.97	0.99	0.64	Amazonian
	21.79	23.14	0.92	0.96	0.35	Southeastern
MAGGIA	18.60	21.12	0.87	0.89	0.51	Amazonian
	24.26	26.39	0.79	0.85	0.28	Southeastern
Nagoya	19.84	30.26	0.49	0.74	0.32	Amazonian
	16.39	20.86	0.70	0.78	0.46	Southeastern

. Furthermore, similar results appear in

Table A10. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for December 2024, discriminated by the two clusters of stations and by quartiles (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE	Cluster
EMBRACE (Q-lower)	4.48	5.63	0.33	0.36	−0.44	Amazonian
EMBRACE (Q-upper)	6.06	11.58	0.41	0.34	−0.58	Amazonian
EMBRACE (Q-inter)	7.37	11.43	0.81	0.86	0.67	Amazonian
EMBRACE (Q-lower)	9.83	11.79	0.58	0.45	−0.59	Southeastern
EMBRACE (Q-upper)	14.59	16.26	0.42	0.57	0.12	Southeastern
EMBRACE (Q-inter)	16.01	18.88	0.72	0.69	0.19	Southeastern
IGS (Q-lower)	13.58	14.40	0.36	0.37	−1.10	Amazonian
IGS (Q-upper)	12.65	13.57	0.67	0.78	0.52	Amazonian
IGS (Q-inter)	13.30	14.41	0.94	0.96	0.62	Amazonian
IGS (Q-lower)	21.18	22.09	0.68	0.57	−1.32	Southeastern
IGS (Q-upper)	19.76	20.61	0.48	0.73	0.37	Southeastern
IGS (Q-inter)	22.25	23.43	0.80	0.84	0.26	Southeastern
MAGGIA (Q-lower)	14.22	15.39	0.30	0.25	−1.59	Amazonian
MAGGIA (Q-upper)	23.96	26.68	0.16	0.13	−2.08	Amazonian
MAGGIA (Q-inter)	17.91	19.94	0.80	0.78	0.36	Amazonian
MAGGIA (Q-lower)	20.05	21.19	0.48	0.37	−1.41	Southeastern
MAGGIA (Q-upper)	25.06	27.15	0.18	0.20	−1.32	Southeastern
MAGGIA (Q-inter)	25.03	26.69	0.61	0.55	−0.17	Southeastern
Nagoya (Q-lower)	6.86	8.13	0.23	0.25	−1.03	Amazonian
Nagoya (Q-upper)	35.06	47.23	0.04	0.04	−4.84	Amazonian
Nagoya (Q-inter)	18.73	26.11	0.44	0.60	0.13	Amazonian
Nagoya (Q-lower)	10.60	12.86	0.45	0.27	−1.16	Southeastern
Nagoya (Q-upper)	18.12	23.55	0.20	0.12	−2.32	Southeastern
Nagoya (Q-inter)	17.79	21.50	0.43	0.41	−0.34	Southeastern

, but shown by quartiles. Surprisingly, even having a low density of GNSS stations, performance of the TEC maps for the Amazonian cluster was better than for the Southeastern cluster. This issue can be explained by the usually lower TEC values over the Amazonia, due to the effect of the EIA. As commented before, lower TEC values tend to improve the adopted metrics.

A similar validation was performed considering the TEC values extracted from maps of each source and the Gopi Seemala values taken as references for the 14 GNSS stations employed by IGS over Brazil, which are part of the RBMC, for all times of the day and all days of December 2024. The resulting metrics are shown in

Table 8. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 14 IGS stations, using Gopi Seemala values as ground truth, for December 2024 (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE	9.30	13.44	0.84	0.91	0.80
IGS	16.24	18.01	0.93	0.97	0.55
MAGGIA	20.90	23.44	0.84	0.88	0.43
Nagoya	16.95	25.13	0.52	0.73	0.44

.

Yet another validation appears in

Table 9. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for 18 UT of all days of December 2024 (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE	9.96	14.30	0.20	0.55	0.12
IGS	17.06	18.89	0.47	0.65	0.34
MAGGIA	25.05	26.95	0.29	0.42	−0.25
Nagoya	25.56	36.67	−0.09	0.12	−1.82

, which is similar to the validation that resulted in Table 5, considering all Gopi Seemala TEC values for all days of December 2024, and all the 27 stations but only for 18 UT, which is typically the time of day presenting high TEC values. Furthermore, metrics were similarly calculated for 18 UT, but discriminating by the three quartiles, as shown in

Table A11. Overall values for the set of metrics calculated using TEC values extracted from the maps of the four sources for the 27 selected GNSS stations, using Gopi Seemala values as ground truth, for December 2024, discriminated by quartiles (MAE and RMSE values in TECU; best value is marked in blue, and the worst, in red).

Source	MAE	RMSE	$\mathit{\rho }$	TSS	KGE
EMBRACE (Q-lower)	16.05	18.39	0.05	0.28	−0.63
EMBRACE (Q-upper)	5.10	6.50	0.37	0.53	0.09
EMBRACE (Q-inter)	9.20	14.41	0.06	0.10	−2.58
IGS (Q-lower)	25.10	25.97	0.22	0.60	0.07
IGS (Q-upper)	10.66	11.83	0.46	0.65	0.30
IGS (Q-inter)	16.07	17.04	0.25	0.45	−0.12
MAGGIA (Q-lower)	30.09	31.41	−0.03	0.10	−2.35
MAGGIA (Q-upper)	22.12	25.18	0.17	0.08	−3.20
MAGGIA (Q-inter)	23.83	24.99	0.14	0.13	−2.08
Nagoya (Q-lower)	24.25	29.84	−0.03	0.05	−4.03
Nagoya (Q-upper)	31.60	46.00	0.03	0.02	−7.06
Nagoya (Q-inter)	23.09	34.32	−0.09	0.02	−8.21

(similar to Table 6, but for 18 UT only). TEC values are more intense in Brazil at 18-19 UT.

Aiming to assess the performance of stations according to their geographical position and for the different sources, another validation was performed, considering all times of the day and all days of December 2024, per station. Such calculation rendered a 27-element distribution of each metric per source.

Table 10. Mean values, standard deviation, and coefficient of variation for the distribution of each metric along the 27 selected GNSS stations, corresponding to the TEC values extracted from the maps of the four sources, using Gopi Seemala values as ground truth, for December 2024 (MAE, ${\sigma }_{MAE}$, RMSE and ${\sigma }_{RMSE}$ values in TECU; best value is marked in blue, and the worst, in red).

Source	$\overline{\mathit{M}\mathit{A}\mathit{E}}$	${\mathit{\sigma }}_{\mathit{M}\mathit{A}\mathit{E}}$	${\mathit{C}\mathit{V}}_{\mathit{M}\mathit{A}\mathit{E}}$	$\overline{\mathit{R}\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{\sigma }}_{\mathit{R}\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{C}\mathit{V}}_{\mathit{R}\mathit{M}\mathit{S}\mathit{E}}$	$\overline{\mathit{C}\mathit{O}\mathit{R}\mathit{R}}$	${\mathit{\sigma }}_{\mathit{C}\mathit{O}\mathit{R}\mathit{R}}$	${\mathit{C}\mathit{V}}_{\mathit{C}\mathit{O}\mathit{R}\mathit{R}}$	$\overline{\mathit{T}\mathit{S}\mathit{S}}$	${\mathit{\sigma }}_{\mathit{T}\mathit{S}\mathit{S}}$	$\overline{\mathit{K}\mathit{G}\mathit{E}}$	${\mathit{\sigma }}_{\mathit{K}\mathit{G}\mathit{E}}$
EMBRACE	9.98	4.24	0.42	13.48	3.91	0.29	0.90	0.04	0.05	0.93	0.03	0.72	0.18
IGS	16.52	5.09	0.31	17.67	5.16	0.29	0.95	0.03	0.04	0.97	0.02	0.53	0.18
MAGGIA	21.27	3.42	0.16	23.54	3.52	0.15	0.84	0.06	0.07	0.88	0.04	0.40	0.14
Nagoya	17.57	4.55	0.26	24.90	5.59	0.22	0.61	0.15	0.25	0.76	0.08	0.40	0.16

shows the mean values and standard deviation for each metric, and for each of the four sources. The complete set of metrics for each station and for each source is also available (The complete set of metrics for each station and for each source is available in document “test_case_06-error_estimation_by_network_and_station.csv” in Zenodo at https://doi.org/10.5281/zenodo.15453941 accessed on 5 October 2025). It can be observed that standard deviations are relatively low for MAE, RMSE, and KGE and very low for the correlation and TSS, demonstrating an even performance of the 27 stations for each source.

Coefficients of variation (standard deviation divided by the mean) were also calculated for each case, demonstrating a low variability in performance (error and accuracy metrics) along the stations for each source.

4. Discussion

In Section 3.1.1, joint correlation analysis was performed for each pair of TEC maps and each pair of sources. Comparisons of a source against all other ones for the entire TEC range using the TF-120 dataset of 120 min resolution show that MAGGIA and IGS maps are better correlated to the other maps (around 80%), followed by EMBRACE and Nagoya (around 60%). Concerning image similarity, shown in Section 3.1.2, the same ranking applies, but differences are smoother, with joint SSIM values of around 80%.

In Section 3.1.3, joint correlation analysis was performed for each pair of TEC maps and each pair of sources using the TF-30 dataset of 30 min resolution, which excluded IGS due to its worst 120 min resolution. Comparisons were broken down into two groups: using only TEC values equal to/above the Q3 or using the full range of TEC values. Joint correlation results for the full range were very similar for EMBRACE, MAGGIA, and Nagoya (around 60%), but considering TEC values equal to or above Q3, correlation was better for EMBRACE (30%) than for Nagoya (22%) and MAGGIA (17%). Joint correlations using the full TEC range tend to be higher since the predominance of low TEC values tightens the range of values. Comparisons involving Q3 are more important, since for high values of Q3, the dynamics of the ionosphere is associated with high TEC gradients. However, in this case, correlations tend to be lower due to the interpolation methods of maps of different sources, which cause a mismatch of map grid points with high TEC values between these maps. In addition, grid points with such high TEC values are scarce. Similarity comparisons were performed for TF-30 considering the full range of TEC values, resulting in values of joint SSIM of around 70% (EMBRACE value was slightly better).

Section 3.2 presented an example of ionospheric bubble signature in a TEC map, which highlights the necessity of TEC maps with better accuracy and spatial/temporal resolution to that end. The rationale was that ionospheric bubbles, which are depleted ionization regions of the ionosphere, are associated with ionospheric irregularities that present high TEC gradients detectable in ROTI maps, which may cause scintillation. Therefore, it can be expected to be a correspondence between TEC maps and ROTI or amplitude scintillation maps for such regions. The example refers to an occurrence of ionospheric scintillation that occurred on 27 September 2024, detected in a sequence of three amplitude scintillation maps, at 00:50, 01:00, and 01:10 UT, as well as the ROTI maps for the same date and times. The analysis of the corresponding TEC maps shows the matching patterns that characterize the bubbles.

Section 3.3 shows the performance of the different TEC maps for the 2024 Equinoxes and Solstices, considering the four corresponding maps denoted as AE24, WS24, SE24, and SS24. Joint correlation results were similar to those of TF-120: 79% for MAGGIA and IGS, while EMBRACE and Nagoya were 63% and 52%, respectively. On the other hand, joint SSIM values were very close (around 75%), with a slight disadvantage for EMBRACE and Nagoya, as was the case for TF-120. However, as seen in Figure 3, these maps for MAGGIA and IGS are quite similar, and the results in Table A6 show corresponding values of 90% or above for correlation, while similarity results are around 90%.

Finally, a validation of TEC map local values was performed using TEC data provided by the Gopi Seemala application for a set of selected GNSS stations over Brazil, as shown in Section 3.4. Gopi Seemala values are assumed as reference values., i.e., as ground truth. The employed metrics are MAE, RMSE, Pearson correlation, TSS, and KGE, applied to each set of local values extracted from each TEC map source for specific times/dates. In the example of the bubble signature, the performance of EMBRACE and MAGGIA maps were compared for 27 September 2024 at 00:50, 01:00, and 01:10 UT). MAGGIA presented the worst MAE and RMSE values but better correlation and TSS values (KGE values were almost identical). MAGGIA values correlated better than EMBRACE with Gopi TEC reference values but systematically overestimated these values. As Table A9 details, for most of the GNSS stations, MAGGIA overestimated TEC values by 50% or more, while EMBRACE overestimated less or even underestimated. In the case of the SJSP, EMBRACE yielded an almost exact value.

A more extensive validation was performed using 27 GNSS stations over Brazil, also assuming Gopi Seemala values as the ground truth, for all times of the day and all days of December 2024. In general, the performance of EMBRACE was slightly better, followed by IGS, MAGGIA, and Nagoya (IGS presented the best correlation and TSS values). The same analysis, but discriminating quartiles, presented similar results, mainly for the interquartile range. As expected, correlation, TSS and KGE values were quite low for the upper and lower quartiles. This is due, respectively, to the possibly high mismatch of grid points with high TEC values in the maps for the corresponding station values and for some variability that is inherent to low TEC values. Two subsets of the 27-station set were defined, each one with nine GNSS stations: the Amazonian and the Southeastern clusters. As already commented, the performance of the TEC maps for the Amazonian cluster of stations was better than for the Southeastern, despite the region of the first cluster having a lower density of GNSS stations. Such unexpected results could be explained by the lower TEC values over Amazonia, which tend to improve the values of the metrics.

Additionally, considering validation for only the 14 GNSS stations employed by the IGS provider, the metrics were similar to those obtained with the set of 27 GNSS stations, and also presented a similar ranking for the four sources. Another experiment, considering the full set of 27 GNSS stations for all days of December 2024, but only for 18 UT resulted in another similar ranking, but correlation, TSS and KGE values were low as expected, since TEC values at 18 UT are high. Furthermore, only a single time of day is considered, implying a reduced set of 27 TEC values, instead of 3888 values, for instance, in the case of all times of the day considering the 10 min resolution of EMBRACE.

A final assessment considered validation for all times of the day and all days of December 2024 but discriminated by each of the 27 GNSS stations. This scheme resulted in 27 distributions for each metric, each one with a mean value, a standard deviation, and a coefficient of variation. The values of the latter two parameters demonstrated a low variability among the stations concerning these error and accuracy metrics.

5. Conclusions

This work proposed a comparison of TEC maps of different sources: IGS, EMBRACE, MAGGIA, and Nagoya. As already mentioned in the previous section, such comparison is challenging, due to the different spatial and temporal resolutions, incomplete coverage of data for some regions or times, and different density of GNSS stations over Brazil. Moreover, the direct comparison between pairs of maps, performed by means of different metrics, presented results that cannot be interpreted directly, since there is not an absolute TEC map to reference. It can be assumed that the source/provider of TEC maps that was more correlated with all the others is the best, but such a conclusion is different for the full range of TEC values and for the values related to the Q3, which are more relevant when such values are high.

A validation of these TEC maps was performed using a time series of Gopi Seemala TEC point values using a set of 27 specific GNSS stations geographically distributed over Brazil. It was assumed that such comparison is a validation, considering that the Gopi Seemala TEC values can be taken as ground truth. This assumption presumes that Gopi Seemala algorithm for calculating TEC is correct, and that carrier/signal measurements are not affected by noise or other errors.

The validation involved different experiments, which included the validation of GNSS stations of an Amazonian or a Southeastern cluster, validation considering quartiles, and validation only for 18 UT time (likely presenting TEC peak values). Moreover, statistics of the metrics according to the stations show low variability in the performance of the TEC maps of different sources along the stations. All validation resulted in a similar ranking for the four sources considered, which shows the robustness of the analysis.

This study compared systematically the four sources of TEC maps, IGS, EMBRACE, MAGGIA, and Nagoya, over Brazil during the rising phase of solar cycle 25 (2022–2024). The correlations results between pairs of maps of different sources and the validation using Gopi Seemala data were presented. These results allow the identification of substantial divergences between TEC maps of these sources, which are probably due to GNSS monitoring station coverage, TEC calculation, and map interpolation method. Validation errors were estimated using the MAE, RMSE, $\rho$, TSS, and KGE metrics. Our results can be summarized as follows:

• IGS maps have global coverage and perform reasonably well despite having a low regional density of GNSS stations and low spatial and temporal resolutions, which implied coarse interpolation, and thus smoother maps. A higher temporal resolution would enhance the quality of these maps, but it would be unfeasible due to the processing demands of global maps;
• EMBRACE maps presented lower validation errors (MAE and RMSE), but its standard available spatial resolution could be improved, as demonstrated in the case study of bubble signature presented here. Another point is its map generation that includes interpolation, but not extrapolation of TEC values, and thus many NaNs are left, according to the map. These NaNs occur despite using a dense network of GNSS stations;
• MAGGIA maps have the better spatial resolution (equal to Nagoya), but they consistently overestimated TEC, implying higher validation errors, despite also using a dense network of GNSS stations. Its maps did not allow to clearly detect the bubble signature in the example presented here;
• Nagoya maps presented validation errors similar to IGS, but the worst values for correlation, TSS, and KGE, even having a global coverage and better spatial and temporal resolutions. Its maps presented a higher amount of NaNs, probably because they were conceived for the very dense network of GNSS stations over Japan.

As shown from the obtained results, this work highlights that all analyzed TEC maps over Brazil could be more reliable. The task of drawing conclusions is subjected to the assumption of possible causes of inaccuracies. According to the considered source, it can be concluded that spatial/time resolution, map interpolation method, or density of GNSS monitoring network should be improved to provide better TEC maps. As the number of GNSS stations over Brazil is increasing, future work could involve performing an extensive validation of these TEC data sources using more GNSS stations over Brazil and closer regions of neighboring countries. Better TEC maps would enable more reliable studies of the ionosphere, such as the identification of ionospheric bubbles of the test case presented here. Furthermore, better TEC maps would also improve the evaluation of impacts in GNSS-based services such as positioning, navigation, and ionospheric modeling.