A Multi-Scale Fusion Deep Learning Approach for Wind Field Retrieval Based on Geostationary Satellite Imagery

Abstract

Wind field retrieval, a crucial component of weather forecasting, has been significantly enhanced by recent advances in deep learning. However, existing approaches that are primarily focused on wind speed retrieval are limited by their inability to achieve real-time, full-coverage retrievals at large scales. To address this problem, we propose a novel multi-scale fusion retrieval (MFR) method, leveraging geostationary observation satellites. At the mesoscale, MFR incorporates a cloud-to-wind transformer model, which employs local self-attention mechanisms to extract detailed wind field features. At large scales, MFR incorporates a multi-encoder coordinate U-net model, which incorporates multiple encoders and utilises coordinate information to fuse meso- to large-scale features, enabling accurate and regionally complete wind field retrievals, while reducing the computational resources required. The MFR method was validated using Level 1 data from the Himawari-8 satellite, covering a geographic range of 0–60°N and 100–160°E, at a resolution of 0.25°. Wind field retrieval was accomplished within seconds using a single graphics processing unit. The mean absolute error of wind speed obtained by the MFR was 0.97 m/s, surpassing the accuracy of the CFOSAT and HY-2B Level 2B wind field products. The mean absolute error for wind direction achieved by the MFR was 23.31°, outperforming CFOSAT Level 2B products and aligning closely with HY-2B Level 2B products. The MFR represents a pioneering approach for generating initial fields for large-scale grid forecasting models.

1. Introduction

Wind is a fundamental component of atmospheric dynamics, and is critically important for the global climate system and human activities [1]. Variations in wind patterns correlate directly with the frequency and intensity of extreme weather phenomena, such as storms, typhoons, and tornadoes, presenting threats to human lives and property, as well as to infrastructure and ecological stability [2]. Conversely, wind energy is a sustainable and clean energy source that has undergone rapid global development and widespread adoption. The distribution and dynamics of wind fields play pivotal roles in determining the potential and efficiency of wind energy resources, providing essential guidance for the planning, construction, and operation of wind farms [3,4]. Consequently, wind field research is valuable. Through advanced wind field retrieval techniques, a more accurate understanding of wind dynamics can be achieved, providing essential initial data for forecasting models and thereby improving the accuracy and timeliness of weather predictions.

In recent decades, numerous methodologies for wind field retrieval have emerged. Traditional approaches predominantly rely on empirical relationships, mathematical models [5,6], and meteorological principles to estimate wind field states from observational data. Waveform matching [7] involves comparing waveform characteristics observed in data with those in established models or reference datasets to identify the most compatible wind field model. Although this method offers a direct estimation of the wind field and is adaptable to various types of observational data, it typically requires a substantial volume of observational data and reference libraries to attain precise outcomes. It may also encounter challenges in effectively mitigating noise within observations [8]. The geophysical model function (GMF) method [9] uses geophysical model functions to interpret remote sensing data and interactions among the atmosphere, oceans, and Earth’s surface. Although it is frequently employed in the interpretation and analysis of remote sensing observational data, this method is challenged by its model complexity and dependence on the quality of observational data [10]. The Cloud Motion Winds method [11] involves extracting wind speed and direction data by monitoring cloud movements. This method offers wide coverage and real-time capability, but currently faces limitations in accuracy and is dependent on cloud cover [12,13].

In recent years, the development of neural networks and deep learning technologies has brought new perspectives to wind field retrieval, compensating for the shortcomings of traditional methods. For example, Li et al. proposed a wind speed retrieval model near the coastline based on an artificial neural network [14]. The model utilised global navigation satellite system reflectometry (GNSS-R) from Cyclone GNSS (CYGNSS) Level 1B data for retrieval, resulting in a 24.4% improvement in wind speed root mean square error (RMSE) compared to CYGNSS Level 2 data. However, when using a fully connected network for wind speed retrieval, overfitting may occur due to the non-uniform distribution of winds, especially at high wind speeds. To mitigate this issue, Guo et al. introduced an end-to-end convolutional neural network (CNN) model [15], which improved feature extraction, resulting in an RMSE of 1.53 m/s for wind speed retrieval in the range of 0–25 m/s. However, the inherent translation equivariance in CNNs introduces imprecision in feature extraction from Doppler-derived maps. To address this challenge, Zhao et al. proposed a Doppler-derived map model based on the Transformer architecture [15], achieving an improved wind speed retrieval RMSE of 1.43 m/s. These deep learning methods have significantly improved the accuracy of wind speed retrieval compared to traditional methods, avoiding the construction of complex physical models and systematic biases.

Over the past two years, significant progress has been made in the field of meteorological forecasting through deep learning, with several global meteorological forecast models being successively proposed [16,17,18]. These deep learning models have achieved comparable, if not superior, forecast accuracy compared to numerical weather prediction (NWP) systems, such as the European Centre for Medium-Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS). Additionally, they significantly reduce the computational resources required and substantially improve the forecast speed [19]. However, existing deep learning models that are capable of conducting large-scale forecasts still depend on initial fields supplied by NWP systems, due to the limitations of current retrieval methods, which cannot simultaneously retrieve both wind speed and direction, nor can they acquire real-time, large-scale wind field states. As a result, they fail to deliver real-time, accurate large-scale initial wind fields. This constraint hinders the ability of deep learning methods to accomplish an end-to-end forecasting process based solely on observational data.

Driven by the limitations of current wind field retrieval techniques and the demand for large-scale grid wind field retrievals, and inspired by the correlation between clouds and wind [20,21], we propose a novel multi-scale fusion retrieval (MFR) method. MFR is a deep learning approach that uses cloud data from geostationary observation satellites to retrieve wind fields at multiple scales. Direct retrieval of wind fields under clear-sky or high-level cloud conditions is challenging; however, geostationary satellites provide multi-channel data that encompass various meteorological features [22], making it possible to retrieve wind fields in cloud-free conditions. Furthermore, existing studies [23,24] have demonstrated a correlation between upper-level and near-surface wind fields. Even when the Earth’s surface is obscured by high-level clouds, it is still possible to retrieve near-surface wind fields from high-level clouds. To capture the detailed features of wind fields, we developed the cloud-to-wind transformer (C2W-Former) model, which uses a windows self-attention mechanism to extract features from high-resolution mesoscale cloud images. The Multi-encoder Coordinate U-net (M-CoordUnet) model was designed to incorporate the geographical characteristics of large-scale wind fields and avoid inconsistencies among different mesoscale retrieval results by integrating coordinate information and large-scale cloud features. This multi-scale fusion approach reduces the high computational resource requirements of directly modelling high-resolution cloud data at large scales, and preserves observational information by avoiding downsampling. Furthermore, it enables the rapid generation of large-scale grid wind fields, opening up a new direction for deep learning-based initial field generation for forecasting models.

The remainder of this article is organised as follows. Section 2 introduces the study area and the relevant data used. Section 3 outlines the two-stage retrieval method, including the C2W-Former and M-coordUnet deep learning models, as well as the details of sample construction. The evaluation metrics, training details, and experimental results are provided in Section 4. Section 5 presents case studies to analyze the robustness of the proposed model under extreme weather conditions. Finally, the conclusions and future research directions are presented in Section 6.

2. Materials

2.1. Study Area

In this study, we focused on the spatial range encompassing the Western North Pacific and the eastern Eurasian continent, covering the geographical area from 0°N to 60°N and from 100°E to 160°E.

In this region, tropical cyclones are frequent occurrences. In 2022 alone, there were 25 named storms recorded in this area. Among these tropical storms, ten developed into typhoons, three of which intensified into super typhoons. Super typhoons Hinnamnor and Nanmadol each caused USD 1 billion in damages (https://en.wikipedia.org/wiki/2022_Pacific_typhoon_season (accessed on 23 December 2024)). These phenomena are clearly illustrated in a vector diagram of ERA5 data for this region (Figure 1).

2.2. Data

This study used two primary datasets: Himawari-8 satellite data and ERA5 reanalysis data. Additionally, forecast data from ECMWF IFS, wind field products obtained from two scatterometer satellites and 340 weather stations data were used for comparison with our results.

The Himawari-8 satellite (https://www.eorc.jaxa.jp/ptree/index.html (accessed on 23 December 2024)), operated by the Japan Meteorological Agency, is an advanced meteorological satellite with a primary mission to monitor various domains, such as the atmosphere, climate, oceans, and environment. Equipped with the Advanced Himawari Imager (AHI), the satellite covers multiple spectral channels, including visible light and infrared channels, allowing it to collect visible light and infrared data. The Japan Aerospace Exploration Agency provides Level 1 grid data for AHI with a spatial resolution of 0.02° × 0.02°, covering East Asia, the Western Pacific, Australia, and surrounding waters. During preprocessing, missing values or outliers (<1%) identified by Himawari-8 L1 production were removed to maintain data integrity. In addition, the data may contain some noise, but relevant studies [25,26,27] indicate that deep learning models can adapt to noise and adjust accordingly. Therefore, we have not performed any special identification or processing of the noise. This study specifically selected channels 7 and 13 from the Level 1 data, as they provide pertinent cloud-related information [28]. Notably, although the original Level 1 data had a high temporal resolution of 10 min, we opted for a 6 h temporal resolution in this study, focusing on data at 0:00, 6:00, 12:00, and 18:00 daily for model construction and analysis, because large-scale wind field variation is not pronounced over short periods, and our analysis revealed that a 6 h interval is suitable for capturing meaningful patterns, striking a balance between data granularity and computational efficiency. Relevant studies [29,30,31] indicate that radar echo data also have great potential in wind field retrieval, but its wind field coverage is relatively limited and it is difficult to meet the forecast needs of a large area and the entire region. Therefore, we chose the Himawari-8 geostationary satellite to accurately retrieve the wind field through large-scale, high-resolution cloud image data.

The fifth-generation atmospheric reanalysis dataset released by ECMWF, ERA5 [32], provides hourly global weather records from 1940 to the present, covering a wide range of meteorological variables. Through numerous validation studies [33,34], ERA5 has demonstrated high accuracy and reliability under different climate and weather conditions. This study used the 10 m wind components (U and V) provided in ERA5 single-level data (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form (accessed on 23 December 2024)), with a spatial resolution of 0.25° × 0.25°, as the targets for wind field retrieval.

ECMWF IFS is widely recognised as one of the most advanced NWP systems currently available [17]. Powered by high-performance computing, it conducts numerical computations based on specified initial and boundary conditions to forecast the future evolution of wind fields over a designated timeframe. IFS provides wind field forecast data (https://www.ecmwf.int/en/forecasts/datasets/set-i#I-i-a_fc (accessed on 23 December 2024), https://apps.ecmwf.int/shopping-cart/orders/new/subset/162 (accessed on 23 December 2024)) twice daily, covering the initial forecast time and the subsequent 10 days, with global coverage and a spatial resolution of 0.1°. In this study, a bilinear interpolation was applied to downscale the spatial resolution of the IFS data from their original resolution of 0.1° to 0.25°. In this study, IFS forecast outputs for the start of the forecast period were used as initial fields for comparison with the wind field retrieval results obtained through deep learning models.

Level 2B wind field data from two scatterometer satellites, HY-2B (https://osdds.nsoas.org.cn/oceanOne?type=1&id=1673143940646674433 (accessed on 23 December 2024)) and CFOSAT (https://osdds.nsoas.org.cn/oceanOne?type=1&id=1673144819122675714 (accessed on 23 December 2024)), were used to evaluate the accuracy and coverage of the wind field retrievals through comparison with existing satellite wind field products. Due to the non-grid format of wind field products from these two satellites, a spatiotemporally weighted fusion algorithm [35,36] was used to project them onto ERA5 grid points. This approach integrated temporal and spatial interpolation techniques, substituting temporal data for missing spatial data and vice versa. This strategy maximised the utilisation of satellite observation data, as follows:

$$\begin{array}{c}\hfill u_n={\displaystyle \frac{{\mathsf{\Sigma }}_{k=1}^N{w}_k{u}_k}{{\mathsf{\Sigma }}_{k=1}^N{w}_k}}\end{array}$$

(1)

$$\begin{array}{c}\hfill w_k={\displaystyle \frac{2-\left[\frac{{(x_k-x_0)}^2+{(y_k-y_0)}^2}{R^2}+\frac{{(t_k-t_0)}^2}{T^2}\right]}{2+\left[\frac{{(x_k-x_0)}^2+{(y_k-y_0)}^2}{R^2}+\frac{{(t_k-t_0)}^2}{T^2}\right]}}\end{array}$$

(2)

where $u_n$ is the interpolated estimate obtained as a linear combination of N observed values within a spatiotemporal influence radius; subscript k denotes the observation data point; $u_k$ is the observed value at point k; $w_k$ is the weight of observation point k, determined by the standardised distances in time and space from the data points to the grid interpolation point; $(x_k,y_k,t_k)$ are the spatiotemporal coordinates of observation point k, with subscript 0 representing the point to be interpolated; R is the distance influence radius; and T is the time influence radius. In this study, R = 0.5° and T = 6 h.

In addition, we used observational data from weather stations furnished by the National Centers for Environmental Prediction (www.ncei.noaa.gov/products/land-based-station/integrated-surface-database (accessed on 23 December 2024)) (NCEP) and identified 340 stations within the study domain with valid data for 2021, allowing us to assess the error characteristics between the MFR results and in situ observations.

We used Himawari-8 satellite and ERA5 data spanning 2017–2022, with a temporal resolution of 6 h. The dataset was split into training (2017–2020), validation (2022), and test (2021) subsets using a partitioning method similar to that used for Pangu-Weather [17]. Data from IFS, HY-2B, and CFOSAT were sourced from 2021.

During the data preprocessing phase, several critical steps were undertaken to ensure the integrity and consistency of the datasets. Anomalous data points (missing values and outliers) indicated by Himawari-8 L1 production, accounting for less than 1% of the sample, were identified and removed. To address differences in feature scales across datasets, normalization was applied to standardise all variables, which is essential for effective model training. Furthermore, interpolation techniques were utilised to resolve spatial and temporal resolution discrepancies between datasets, ensuring consistent data alignment for model inputs.

3. Methods

3.1. MFR Method

To mitigate the issue of periodicity in wind direction, we retrieved the U and V components of the wind field. The observational data used in this study were derived from the Himawari-8 satellite, with the objective of retrieving ERA5 wind field data. Because both datasets were in grid format, at any given moment, they could be considered as H × W grids, without considering the feature dimensions. The spatial resolutions of the Himawari-8 satellite and ERA5 data were 0.02° and 0.25°, respectively. Therefore, at any given time within the entire study area (0–60°N, 100–160°E), they could be considered to be grid data with resolutions of 3000 × 3000 and 240 × 240, respectively, with the right boundary discarded for ease of processing. Although the Himawari satellite observational data offer higher spatial resolution, providing richer details, they also require more computational resources for processing. Conducting the retrieval model training across the entire study area directly would require significant computational resources due to the large amount of input data. However, downsampling the Himawari-8 satellite observational data to a spatial resolution of 0.25° before conducting the retrieval model training across the entire study area could lead to a substantial loss of mesoscale observational information during the downsampling process. Therefore, the key issue in the wind field retrieval in this study was extracting and using wind field features from these high-resolution observations more effectively, while maintaining reasonable consumption of computing resources.

The region of interest is on the east coast of Asia where the formation of boundary layer (BL) clouds is at a minimum [37]. Under clear-air conditions, although the area above the Earth’s surface is nearly cloud-free, the Himawari-8 satellite’s long-wave infrared channel (Band 13) can still observe surface temperature (including sea surface temperature (SST) [38] and land surface temperature (LST) [39]). Studies [40,41] have demonstrated a dynamic relationship between SST and near-surface wind fields; therefore, SST/LST features captured by Band 13 can provide implicit information for wind field retrieval.

Under active weather conditions, the Earth’s surface is typically obscured by clouds distributed across various layers of the troposphere [42]. However, the evolution of tropospheric cloud systems is closely related to wind fields within the same layer. Additionally, there is a certain connection between tropospheric and near-surface wind fields. Although explicit physical equations describing this relationship are currently lacking, deep learning models, with their strong nonlinear mapping capabilities (universal approximation ability [43,44,45]), can capture and learn these implicit relations through large-scale data training [46].

This study integrates multi-scale cloud information and employs a model trained on historical data to infer and reconstruct wind fields under various cloud cover conditions. This approach enables the model to utilise surrounding and historical wind field characteristics (for example, areas without clouds may have been covered in historical data) to accurately predict and reconstruct wind fields in regions that are cloud-free or covered only by high-level clouds.

Figure 2 shows the overall architecture of the multi-scale fusion retrieval method proposed in this study. First, cloud data at time t were obtained from the Himawari-8 satellite (bands 7 and 13) with a resolution of 0.02°. In the mesoscale stage, we used a sliding window sampling method (see Section 3.2.1) to decompose the data into mesoscale cloud data, which were then fed into the proposed C2W-Former model (see Section 3.2.3) to generate the corresponding mesoscale wind field UV components. Subsequently, we applied a mesoscale integration method to obtain the preliminary large-scale UV components at a resolution of 0.25° at time t. This approach enabled us to process high-resolution data in relatively smaller regions, reducing the computational burden while preserving complete mesoscale observational information. Additionally, it provided the model with sufficient samples, avoiding the problem of overfitting in transformer models [47]. In the large-scale stage, we first downsampled the original Himawari cloud data from 0.02° to 0.25° using bilinear interpolation, and then combined them with the initial large-scale UV components (see Section 3.3.1). Finally, we used the M-CoordUnet model (see Section 3.3.2) to generate the final large-scale wind field UV components at time t. The fusion approach combined the strengths of both scales, leveraging both detailed mesoscale information and a large-scale global perspective to obtain more accurate wind field retrievals.

3.2. Mesoscale Stage

In this section, we present the details of the mesoscale stage, encompassing the sliding window sampling method used in the preprocessing phase, the C2W-Former model, and the mesoscale integration method used in the postprocessing phase.

3.2.1. Sliding Window Sampling Method

Spatial sliding window sampling is a commonly used method for partitioning an entire image into smaller sub-images. In this study, we used this method to divide large-scale data into several mesoscale data points, thereby constructing mesoscale retrieval samples. If the original image size is $H\times W$, with a window size (size of the cropped sub-image) of $h\times w$, and horizontal and vertical sampling strides of $S_w$ and $S_h$, respectively, then after sliding window sampling, N sub-images of size $h\times w$ can be obtained, where N is determined as shown in Equation (3). The sampling process is shown in Figure 3. The details of the spatial sliding window sampling applied in this study are shown in

Table 1. Configuration of the sliding window sampling method.

Data Type	$\mathit{H}\times \mathit{W}$	$\mathit{h}\times \mathit{w}$	${\mathit{S}}_{\mathit{h}}\times {\mathit{S}}_{\mathit{w}}$	$\mathit{Lat}\times \mathit{Lon}$
Himawari-8	$3000\times 3000$	$600\times 600$	$300\times 300$	${12}^{\circ }\times {12}^{\circ }$
ERA5	$240\times 240$	$48\times 48$	$24\times 24$	${12}^{\circ }\times {12}^{\circ }$

. The actual spatial size corresponding to small images obtained after sliding window sampling for the two types of data was 12° × 12°, which represents the mesoscale sample data.

$$\begin{array}{c}\hfill N=\left(⌊{\displaystyle \frac{H-h}{s_h}}⌋+1\right)\times \left(⌊{\displaystyle \frac{W-w}{s_w}}⌋+1\right)\end{array}$$

(3)

3.2.2. Mesoscale Sample Construction

In the mesoscale stage, the construction of mesoscale cloud-to-wind retrieval samples necessitated the use of the original Himawari-8 satellite Level 1 observational data and ERA5 wind field data. First, the study region was clipped from the datasets. Then, Himawari-8 channels 7 and 13 and ERA5 10 m U and V wind components were selected. Spatial sliding window sampling was applied to these datasets. Corresponding mesoscale Himawari satellite and ERA5 data were paired in space and time to create samples, with Himawari satellite data used as the input and ERA5 data used as the learning target.

During sample construction, any samples with invalid values were discarded. Finally, Z-score standardisation [48] was applied to the inputs and outputs of all samples using the mean and standard deviation of each feature of the Himawari satellite and ERA5 data. This standardisation process accelerated model training and enhanced model performance [49]. The details of the datasets are presented in

Table 2. Design of the dataset.

Dataset	Time Range (Year)	Mesoscale Samples	Large-Scale Samples
Training	2017–2020	468,068	5778
Validation	2022	111,855	1456
Testing	2021	1381	1381

. Notably, to facilitate direct comparison with multi-scale fusion and other products, we evaluated mesoscale predictions at the large scale. Specifically, we used the mesoscale-integrated, post-processed preliminary large-scale UV components as our mesoscale test samples.

3.2.3. C2W-Former Model

During the mesoscale retrieval phase, it is crucial to extract wind field characteristics from high-resolution mesoscale observational cloud imagery. To address this challenge, we introduced the C2W-Former model, using the transformer architecture. Compared to Convolutional Neural Networks (CNNs), which have limitations in capturing global context [50], Transformers excel at modeling long-range dependencies due to their flexible receptive fields [51]. Additionally, unlike Recurrent Neural Networks (RNNs), which face issues such as vanishing gradients when processing sequential data [52], Transformers support parallel processing, thereby enhancing computational efficiency. Transformers have demonstrated remarkable feature extraction capability in wind field retrieval and forecasting tasks [17,53,54], particularly due to their ability to capture long-range spatial dependencies through self-attention mechanisms.The C2W-Former model adopted the windows multi-head self-attention (W-MSA) mechanism from the Swin Transformer [55], which not only reduces computational complexity but also efficiently captures interaction dynamics among local features. This hierarchical design allows the model to combine local and global information, ensuring precise retrieval of mesoscale wind field characteristics while maintaining computational efficiency.

Figure 4 shows the architecture of the C2W-Former model. The input was a mesoscale cloud image with dimensions of 2 × 600 × 600, where 600 × 600 represents the image length and width (i.e., the number of observation points in the longitude and latitude directions), and 2 denotes the two channels (7 and 13) of the AHI Level 1 data. First, the input underwent patch partition to divide it into non-overlapping patches, which was similar to the Vision Transformer (ViT) approach. In our implementation, we used a patch size of 3 × 3 and transformed the original input into 200 × 200 patches, each of which was embedded into a latent space vector of length C = 96 in this study. Subsequently, we applied stage 1 Swin Transformer blocks (Figure 5), which did not alter the input size. To obtain the desired output size, we cropped four patches from the edges before stage 2, reducing the input size from 200 × 200 to 192 × 192. Then, we applied a patch merging process, stacking four adjacent patches in the channel dimension to form a new patch, increasing the actual spatial size of each patch and making the model capable of learning at different hierarchical spatial levels. The numbers of Swin Transformer blocks in the three stages were 2, 2, and 6, respectively, and the output sizes of each stage were 200 × 200, 96 × 96, and 48 × 48, respectively, with a consistent feature dimension of 96 throughout. Next, we obtained intermediate feature maps, which were decoded using two convolutional layers. In contrast to the commonly used fully connected decoding method in the transformer, convolutional layers can compress feature dimensions, while also considering neighbouring patch features. Finally, we obtained the corresponding mesoscale UV as a 2 × 48 × 48 output through a 1 × 1 convolutional layer.

Figure 5 shows the architecture of the Swin Transformer block, which was constructed by replacing the standard MSA module in a transformer block with a module based on shifted windows, while the other layers were kept unchanged. As shown in Figure 5a, the Swin Transformer block consisted of a shifted window-based MSA module, followed by a two-layer multilayer perceptron (MLP) with Gaussian error linear unit nonlinearity between the layers. Additionally, a LayerNorm layer was applied before each MSA module and each MLP, and a residual connection was applied after each module. The Swin Transformer blocks were computed as follows:

$$\begin{array}{cc}\hfill & {\widehat{\mathbf{x}}}^l=\text{W-MSA}\left(\mathrm{LN}\left({\mathbf{x}}^{l-1}\right)\right)+{\mathbf{x}}^{l-1},\hfill \\ \hfill & {\mathbf{x}}^l=\mathrm{MLP}\left(\mathrm{LN}\left({\widehat{\mathbf{x}}}^l\right)\right)+{\widehat{\mathbf{x}}}^l,\hfill \\ \hfill & {\widehat{\mathbf{x}}}^{l+1}=\text{SW-MSA}\left(\mathrm{LN}\left({\mathbf{x}}^l\right)\right)+{\mathbf{x}}^l,\hfill \\ \hfill & {\mathbf{x}}^{l+1}=\mathrm{MLP}\left(\mathrm{LN}\left({\widehat{\mathbf{x}}}^{l+1}\right)\right)+{\widehat{\mathbf{x}}}^{l+1},\hfill \end{array}$$

(4)

where ${\widehat{\mathbf{x}}}^l$ and ${\mathbf{x}}^l$ are the output features of the (S)W-MSA module and the MLP module for block l, respectively, and W-MSA and SW-MSA are the W-MSA using regular and shifted window partitioning configurations, respectively. Figure 5b shows the W-MSA and SW-MSA, where SW-MSA is a variant of W-MSA with all windows shifted in size by one half-window to the right and downward.

The W-MSA significantly reduced the computational complexity. For an image with M × M patches within a window, the computational complexities of the W-MSA and traditional MSA were compared using Equations (5) and (6), respectively, as follows:

$$\mathsf{\Omega }\left(MSA\right)=4HW{C}^2+2{\left(HW\right)}^{2}C$$

(5)

$$\mathsf{\Omega }\left(W\text{-}MSA\right)=4HW{C}^2+2M^{2}HWC$$

(6)

where the former grows quadratically with the number of patches $HW$, and the latter remains linear when M is fixed (default value = 8). Global self-attention computation is generally prohibitively expensive for large $HW$, whereas window-based self-attention is scalable.

3.2.4. Mesoscale Integration Method

Mesoscale integration is the inverse process of sliding window sampling. Because the sliding window step size used in this study was smaller than the window size, the sampled mesoscale data that were obtained overlapped. When it is necessary to restore inverted mesoscale wind field data to large-scale data, a specific method is required to fuse the overlapping parts. The weighted fusion algorithm was used in this study, which ensured the continuity of the fused images. The following section describes the application of the weighted fusion algorithm for mesoscale integration.

Suppose there are two mesoscale images A and B, to be fused, both of size $m\times n$. These two images must be merged from left to right, with an overlap size of $m\times d$. Image A consists of two parts: $A_x$ and $A_y$, representing the overlapping and non-overlapping parts, respectively. Similarly, image B can be divided into $B_x$ and $B_y$. Then, an index vector I of length d is defined as $I=[1,2,\dots ,d]$. A weight vector $\overrightarrow{w}=[w_1,w_2,\dots ,w_d]$ is defined, where $w_i,i\in (1,2,\dots ,d)$ is calculated as follows:

$$x_i=({\displaystyle \frac{l_i}{d-1}}-0.5)$$

(7)

$$w_i={\displaystyle \frac{1}{1+e^{-k{x}_i}}}$$

(8)

where k is a hyperparameter controlling the degree of variation in weights, which was set to 0.5 in this study. The weight vector $\overrightarrow{w}$ is replicated m times to obtain the weight matrix W. Let C represent the merged image, with the overlapping part denoted as $C_x$. Then, $C_x$ can be obtained as follows:

$$C_x=(1-W)\circ A_x+W\circ B_x$$

(9)

where ∘ denotes the Hadamard product. Then, horizontally concatenating $A_y$, $C_x$, and $B_y$ yields the weighted fusion image C. Repeating this process enables mesoscale data to be integrated and concatenated into large-scale data, achieving a resolution transformation from $2\times 48\times 48$ per window to a continuous large-scale resolution of $2\times 240\times 240$, which represents the preliminary large-scale wind field.

3.3. Large-Scale Stage

In this section, we consider the large-scale stage, primarily covering the construction of large-scale samples and the M-CoordUnet model.

3.3.1. Large-Scale Sample Construction

After completing the mesoscale stage, the preliminary wind field at a resolution of 0.25° was obtained through the mesoscale integration process described above, with a resulting output dimension of $2\times 240\times 240$, which provided real-time, full-coverage wind fields but still lacked large-scale global information. Although we applied the mesoscale integration method to fuse different mesoscale results, this approach could not fully resolve the issue of inconsistent results across different mesoscale retrievals, leading to blurred or discontinuous boundaries in certain large-scale events (e.g., convection or storms), as shown in Figure 6. There have been studies that addressed similar ambiguity issues using blind source separation (BSS) methods [56], which provides valuable insights for tackling such challenges. However, we simply use the correction of M-CoordUnet on a large scale to mitigate this ambiguity.

To address this limitation, we designed the large-scale stage. In this stage, we incorporated large-scale cloud data to obtain global information at a larger scale. However, the original Himawari satellite cloud observation data had high resolution, which would require excessive computational resources if they were used directly as a model input. Moreover, our aim was to extract large-scale information, rather than detailed features, from the data. Therefore, we applied a bilinear interpolation to downsample the data to a resolution of 0.25°, resulting in a downsampled cloud image of size $2\times 240\times 240$, consistent with the output dimensions of the mesoscale stage. Consequently, we applied both downsampled cloud data and preliminary UV as inputs to the large-scale model, using the former to obtain large-scale contour information and the latter to obtain detailed mesoscale information. We continued to use ERA5 UV data as the target, with the same invalid data handling and normalisation strategies as in the mesoscale stage.

3.3.2. M-CoordUnet Model

When designing the model for the large-scale stage, we primarily considered the following key issues. First, the input observational cloud data and preliminary UV data are multi-modal, with the cloud data having a dimension of $2\times 240\times 240$ and the preliminary UV data having a dimension of $2\times 240\times 240$. These were concatenated along the channel dimension, resulting in an input data dimension of $4\times 240\times 240$. Using a single encoder for feature extraction could lead to incomplete feature extraction due to the limited capacity of the encoder. Second, large-scale wind fields possess strong regional characteristics [57], but our input data lacked explicit regional information. Third, the number of large-scale samples was limited, and we needed to maintain a lightweight model to avoid overfitting. To address these issues, we designed the M-CoordUnet model, which adopted a multi-encoder design (Figure 7) that used separate encoders to extract features from different variables, thereby avoiding the potential limitations of a single encoder. Furthermore, inspired by the CoordConv-Unet model [58], we incorporated normalised latitude and longitude coordinates into each encoder block, as two new channels to the input. These explicit spatial features facilitated more effective learning and capturing of spatial transformations. Once the features from each variable were separately extracted, they were aggregated by the M-CoordUnet model using the centre block, and then sent to the decoder for decoding. Each decoder block received the output from the previous block and the corresponding layer’s encoder block, enabling it to consider both higher-level general features and layer-specific detailed features, thereby facilitating a more accurate reconstruction of the target [59]. Finally, the output was obtained through a 1 × 1 convolutional layer, yielding large-scale UV data with a dimension of $2\times 240\times 240$.

4. Results

To evaluate the performance and versatility of the proposed MFR method compared to the OSR method, scatterometer satellite wind field products, and IFS initial fields, the numerical results for different wind field characteristics (U and V wind field components, wind speed, and wind direction) were compared.

4.1. Model Evaluation

To demonstrate the effectiveness of the MFR method proposed in this study, we also developed a one-stage retrieval (OSR) method for comparison. In the OSR method, we selected similar cloud data from channels 7 and 13 of the Himawari satellite observations. Subsequently, we directly downsampled the spatial resolution of the observation data from the original 0.02° to 0.25° using a bilinear interpolation method, matching the resolution of ERA5 wind field data. The downsampled observation data were then used as the input for the model, with the U and V components of ERA5 wind field data used as targets for model training. We experimented with the OSR method using both the Swin-Unet model [60] and the CoordConv-Unet model [58].

To better demonstrate the wind field retrieval performance in this study, evaluations were conducted separately for the U and V wind field components, as well as for wind speed and wind direction. The mean absolute error (MAE) and RMSE, which were used to assess scalar data such as wind speed and U and V wind field components, were calculated as follows:

$$\begin{array}{c}\hfill MAE={\displaystyle \frac{1}{L}}{\mathsf{\Sigma }}_{k=1}^K{\mathsf{\Sigma }}_{i=1}^I{\mathsf{\Sigma }}_{h=1}^H{\mathsf{\Sigma }}_{w=1}^W|y_{k,i,h,w}-{\widehat{y}}_{k,i,h,w}|\end{array}$$

(10)

$$\begin{array}{c}\hfill RMSE=\sqrt{{\displaystyle \frac{1}{L}}{\mathsf{\Sigma }}_{k=1}^K{\mathsf{\Sigma }}_{i=1}^I{\mathsf{\Sigma }}_{h=1}^H{\mathsf{\Sigma }}_{w=1}^W{(y_{k,i,h,w}-{\widehat{y}}_{k,i,h,w})}^2}\end{array}$$

(11)

where y is the true value, which in this experiment corresponds to the ERA5 wind field, and $\widehat{y}$ is the model prediction, representing the retrieved wind field in this experiment. K is the total number of samples, I is the number of features, and H and W are the height and width of the wind field image, respectively. In this experiment, I was 2 (U and V components), and H and W were 240, resulting in a total of $L=K\times I\times H\times W$ elements.

However, for wind direction, which exhibits periodicity, using MAE or RMSE may lead to issues at discontinuous points. For example, an error between 10° and 350° should be treated as 20° rather than 340°. Therefore, for wind direction evaluation, we used their deviations (${\mathit{MAE}}_d$ and ${\mathit{RMSE}}_d$, respectively), which were calculated as follows:

$$\begin{array}{c}\hfill E=\left\{\begin{array}{c}\widehat{y}-y,-{180}^{\circ }\le \widehat{y}-y\le {180}^{\circ }\hfill \\ \widehat{y}-y+{360}^{\circ },\widehat{y}-y\le -{180}^{\circ }\hfill \\ \widehat{y}-y-{360}^{\circ },\widehat{y}-y\ge {180}^{\circ }\hfill \end{array}\right.\end{array}$$

(12)

$$\begin{array}{c}\hfill MA{E}_d={\displaystyle \frac{1}{L}}{\mathsf{\Sigma }}_{k=1}^K{\mathsf{\Sigma }}_{i=1}^I{\mathsf{\Sigma }}_{h=1}^H{\mathsf{\Sigma }}_{w=1}^W\left|E_{k,i,h,w}\right|\end{array}$$

(13)

$$\begin{array}{c}\hfill RMS{E}_d=\sqrt{{\displaystyle \frac{1}{L}}{\mathsf{\Sigma }}_{k=1}^K{\mathsf{\Sigma }}_{i=1}^I{\mathsf{\Sigma }}_{h=1}^H{\mathsf{\Sigma }}_{w=1}^W{\left(E_{k,i,h,w}\right)}^2}\end{array}$$

(14)

4.2. Training Details

The C2W-Former and M-CoordUnet models were operated using PyTorch [61]. Both models were trained on the training set on a single NVIDIA Tesla A40 48G graphics processing unit (GPU), with C2W-Former training lasting for approximately 7 days and M-CoordUnet training lasting for approximately 3 h. The loss functions used by both models were MAE + MSE, with the AdamW optimiser [62]. The initial learning rate was set to 5 × ${10}^{-4}$, and the cosine learning rate adjustment strategy [63] was adopted, where the learning rate follows a cosine function, oscillating between a maximum of 5 × ${10}^{-4}$ and a minimum of 5 × ${10}^{-6}$ within each cycle. The batch size was set to 16.

The C2W-Former model was trained for a total of 30 epochs. Based on the validation set results, the model from the 22nd epoch was selected as the optimal model. The M-CoordUnet model was trained for a total of 70 epochs, with the model from the 28th epoch chosen as the optimal model based on the validation set performance. The difference in the number of training epochs between the two models primarily arose from the difference in sample size. Because training samples used during the mesoscale retrieval phase were obtained by sampling from large-scale whole images, each yielding 81 mesoscale sub-images (nine in each of the horizontal and vertical directions), the difference in sample sizes between the two stages was approximately 81-fold.

4.3. Experimental Results

We compared the wind field features retrieved by different methods from the test dataset and existing wind field products with the ERA5 wind field. For convenience, we denoted the large-scale preliminary UV results obtained from the mesoscale stage of the MFR method as MFR (S1), and the final results obtained from the large-scale stage of the MFR method as MFR (S2).

Table 3. Error analysis of the MFR and one-stage retrieval (OSR) wind features in comparison to the ERA5 wind field for land and sea regions. Bold and underscore indicate optimal and suboptimal performance, respectively.

	U		V		Wind Speed		Wind Direction
Model	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
MFR (S2)	1.05	1.48	1.06	1.49	0.97	1.35	23.31	38.41
MFR (S1)	1.25	1.74	1.27	1.74	1.23	1.72	28.03	44.32
OSR (CoordConv-Unet)	1.48	2.07	1.42	1.98	1.33	1.83	30.86	47.23
OSR (Swin-Unet)	1.55	2.15	1.48	2.05	1.40	1.89	33.05	49.60

and Figure 8 present the retrieval results of the MFR and OSR methods, where the U and V wind field components and wind speed were calculated using MAE/RMSE (m/s), and the wind direction was calculated using ${\mathrm{MAE}}_d$ (°) and ${\mathrm{RSME}}_d$ (°). In MFR (S2), minimal errors and maximum correlations were observed in the UV, wind speed, and wind direction (bold, Table 3), with MFR (S1) exhibiting the second-lowest errors (underscore, Table 3). The wind field retrievals significantly improved across various features from MFR (S1) to MFR (S2), confirming the effectiveness of multi-scale fusion. Furthermore, the MFR method outperformed the OSR method in all aspects, demonstrating its efficiency and superiority in wind field retrieval tasks. There were two main advantages of the MFR. First, dividing the entire large area into mesoscale regions and constructing retrieval models both considerably reduced the computational resources required for direct modelling at large-scales, and maintained the high resolution of the observational data, facilitating the generation of finer wind field retrieval information. Second, after completing mesoscale retrieval, wind field features were further extracted from large-scale cloud data, enhancing the geographical spatial perception of the model by incorporating coordinate information, and combining them with mesoscale wind field features to achieve multi-scale feature integration. The main limitation of the OSR method is that it extracted cloud features only from large scales, thereby losing significant mesoscale observational details.

Next, we compared wind field retrievals from the MFR with interpolated wind field data from the HY-2B and CFOSAT satellite products. Because both scatterometer satellites are dedicated to oceanic observations, providing data solely on ocean surface winds, this comparison focused exclusively on marine areas within the study region.

Table 4. Error analysis of the MFR and satellite products over the sea region. Bold and underscore indicate optimal and suboptimal performance, respectively.

	U		V		Wind Speed		Wind Direction
Model/Satellite	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
MFR (S2)	1.21	1.68	1.22	1.69	1.11	1.52	18.73	31.92
HY-2B	1.35	2.01	1.41	2.10	1.20	1.74	18.32	31.19
CFOSAT	1.54	2.24	1.61	2.29	1.36	2.00	23.04	38.03

presents errors with respect to the ERA5 wind fields. When evaluated based on the U and V components, the MFR wind field outperformed those of HY-2B and CFOSAT. Meanwhile, when considering wind speed and direction, the MFR results exhibited similarity to HY-2B, with the MFR performing better for wind speed, and HY-2B performing slightly better for wind direction. Both MFR and HY-2B significantly outperformed CFOSAT. These results show that the MFR wind field accuracy rivalled or even surpassed some existing operational satellite wind field products. In terms of regional coverage, MFR achieved complete coverage of both land and sea in the target area, whereas existing scatterometer wind field products only partially covered marine regions.

Table 5. Error analysis of the MFR and IFS wind initial field products over land and sea regions (00:00 and 12:00 daily). Bold indicate optimal performance.

	U		V		Wind Speed		Wind Direction
Model	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
MFR (S2)	1.04	1.47	1.05	1.48	0.96	1.34	23.81	39.19
IFS	0.62	0.89	0.64	0.91	0.58	0.83	16.03	30.41

presents comparisons of the wind fields retrieved by MFR (S2) and the initial wind fields from IFS [64] conducted daily throughout 2021 at 00:00 and 12:00 UTC, covering both land and sea regions. Figure 9 shows scatterplots of each model (IFS, MFR, and OSR) versus ERA5 at 00:00 on 19 April 2021, during the passage of Super Typhoon Surigae. Combining these results shows that IFS initial wind fields outperformed MFR across all features. This gap was primarily attributed to differences in the volume of observational data utilised. IFS uses data from multiple meteorological observation satellites, buoys, and observation stations, and processes the information on supercomputers, whereas MFR utilises data from a single satellite and a single GPU. This discrepancy highlights the significance of our findings and underscores the importance of exploring the potential of using multifaceted observational data in wind field retrieval.

Table 6. Error analysis of the MFR and one-stage retrieval (OSR) models at different wind speed intervals. Bold and underscore indicate optimal and suboptimal performance, respectively.

Method	3–10 m/s	10–20 m/s	>20 m/s
MFR (S2)	1.403	2.228	4.721
MFR (S1)	1.746	3.338	7.264
OSR (CoordConv-Unet)	1.912	3.322	6.137
OSR (Swin-Unet)	1.982	3.317	7.028

presents the RMSE (m/s) performance of different models across varying wind speed ranges, including 3–10 m/s, 10–20 m/s, and >20 m/s. The results show that MFR(S2) achieves the lowest RMSE values in all wind speed categories (bold, Table 6), demonstrating its superior accuracy in wind field retrieval. In particular, for wind speeds above 20 m/s, there is a significant improvement from MFR(S1) to MFR(S2), highlighting the considerable benefit of multi-scale feature fusion in capturing wind field dynamics in high-wind conditions. These findings further validate the effectiveness of the MFR method in accurately modeling wind fields across diverse scenarios.

Figure 10 presents a comparative analysis of ERA5, IFS, and MFR against observational data from meteorological stations. ERA5 and IFS exhibited an extremely high level of agreement across all metrics. Although the MFR outcomes slightly lagged behind those of ERA5 and IFS, with the exception of a slightly larger deviation in wind speed $R^2$, the other variances were comparatively minor. This result suggests that from the perspective of in situ observations, the MFR results approached those of ERA5 and IFS.

Figure 11 shows the MAE values for the UV components of MFR-retrieved wind fields in the test dataset in different months. Errors in oceanic regions were higher than those in land areas, which was attributed to the prevalence of strong winds over oceans. This discrepancy represents a significant challenge to successful retrieval. Land area errors exhibited a distinct seasonal pattern, with lower errors in autumn and winter and higher errors in spring and summer. In contrast, seasonal patterns were less pronounced in oceanic regions, primarily due to the dominance of monsoon climates over much of the eastern Eurasian continent, where land wind field characteristics were closely linked to seasonal variation.

5. Discussion

To better illustrate the effectiveness of the different retrieval methods and wind field products, we conducted a case study of a wind field retrieval during the landfall of Super Typhoon Surigae in the Philippines on April 19, 2021 at 0:00 UTC (Figure 12). All models successfully retrieved the storm (columns 3–6), with data positions closely matching those of the ERA5 wind field data. The MFR (S2) results were closer to the ERA5 data in terms of intensity and detail compared to MFR (S1), OSR (CoordConv-Unet), and OSR (Swin-Unet). For example, for the U component near the Japanese islands and the V component in the eastern Japanese islands, MFR (S2) produced significantly better contour details and intensity than MFR (S1), OSR (CoordConv-Unet), and OSR (Swin-Unet), all of which produced noticeable underestimations in these two regions. However, in terms of contour details, the IFS initial field outperformed MFR (S2). HY-2B and CFOSAT captured the most intricate features; however, both suffered from insufficient data coverage, notably CFOSAT, which did not observe the storm area in the 6 h period before and after the time frame of the interpolation. This comparison of wind speeds clearly showed that in storm retrieval, the edge details of the IFS initial field were the most accurate and clear, followed by MFR (S2), with OSR (Swin-Unet) showing the worst performance.

To avoid interpretations based on a single case, we examined another wind field retrieval case for Super Typhoon Mindulle on 28 September 2021, 12:00 UTC (Figure 13). The IFS initial field produced the richest details and was closely aligned with ERA5 data for the U component in the eastern Japanese islands and the V component in the northeastern region. The event was closely represented by MFR (S2), with only minor deviations. MFR (S1), OSR (CoordConv-Unet), and OSR (Swin-Unet) produced noticeable over- and underestimations in the U and V components, respectively, in this area. In terms of wind speed, MFR (S2) was closely aligned with both the IFS initial field and ERA5; however, it still exhibited smoothing issues. In contrast, MFR (S1), OSR (CoordConv-Unet), and OSR (Swin-Unet) had smoothing issues and also produced significant underestimations, particularly in land areas and the eastern region of the storm itself, as shown in the vector diagram. Furthermore, HY-2B did not reproduce the storm area before and after the 6 h time frame.

Overall, the retrieval results of the MFR method showed good performance during both super typhoons, indicating its stability and reliability even under extreme weather conditions.

6. Conclusions

We developed an MFR wind field retrieval method. By analysing and modelling data at both the mesoscale and large scales, extracting wind field features from Himawari satellite cloud observations, and integrating data from different scales, we achieved high-quality wind field retrievals with minimal resource requirements. The MAE/${\mathrm{MAE}}_d$ of wind speed and wind direction in the retrieved wind field were 0.97 m/s and 23.31°, respectively. The accuracy of the retrieved wind field matched or exceeded that of some scatterometer satellite wind field products, while providing complete coverage of land and sea areas. In terms of in situ observations, the MFR results nearly attained the same level of accuracy as ERA5 and IFS. Additionally, we demonstrated the effectiveness of the MFR method in retrieving wind fields under extreme weather conditions, highlighting its robust performance. Wind fields obtained through this method can be used as initial fields for large-scale grid forecasting, facilitating the development of deep learning-based wind field forecasting models using observational data.

In future studies, we will explore the incorporation of multi-source observational data into the retrieval process. Single observational satellites have limitations in their observation methods and biases in data processing. Therefore, integrating additional satellite observational data, as well as data from buoys and observation stations, into the retrieval process will mitigate the limitations of single observation methods and enhance the quality of the final retrieved wind fields. We will also investigate a forecasting model based on the retrieved wind field, with the aim of achieving deep learning large-scale grid wind field forecasting based on observational data.